[Courier-imap] Compressing live Maildir(+) directories

[Ronan M. <ro...@io...>]

Hi,

Has anybody considered implementing compression within the Maildir
backend of the Courier IMAP server?  It strikes me that this might
be reasonably simple to implement.  Granted, it has major CPU
implications, but conversely could result in siginificant storage
savings.

I've been toying with the following ideas:

 - Modify the Maildir backend for something like Exim or Postfix
   to compress files as they're written to the Maildir.

   - possible refinement here - don't compress files which match
     certain criteria (e.g. compressed attachments) - differentiate
     un/compressed files by naming compressed files .gz for example

   - Another possible refinement - use and external script to
     compress old messages / folders

 - Modify the maildir backend for Courier IMAP to recognise these
   compressed files and uncompress them as they're opened.

The main overhead here is compression rather than de-compression, but
given that SMTP deliver isn't seen directly from the users point of
view I think this might be acceptable.  It might be possible to minimise
this overhead by enforcing a max message size limit to prevent a 100MB
file from hogging CPU.

This will no doubt have an impact on some IMAP functions - sorting
will be slowed down as each file that's accessed needs to be compressed,
but the typical 'show me message X' transaction shouldn't experience
much of an overhead.

I know this could be accomplished transparently using filesystem level
compression, but that's not an option for the scenario I have in mind.

Has this been attempted before?


-Ronan

______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
______________________________________________________________________