#13 Archiving large IMAP folders fails.

closed-fixed
nobody
None
5
2006-06-18
2004-04-26
No

Hi,

I'd like to archive some of my IMAP folders
(cyrus21-imapd).
I call archivemail like that:
/usr/bin/archivemail --suffix= --days 30 -o
Mail.archive/
imap://florian:pass@imap.foobar.lan/florian.debian.powerpc

That workes for some folders, but expecially on bigger
ones (about 3.000 messages and above) it fails with the
following traceback:

Deleting messages
Traceback (most recent call last):
File "/usr/bin/archivemail", line 1305, in ?
main()
File "/usr/bin/archivemail", line 623, in main
archive(mailbox_path)
File "/usr/bin/archivemail", line 1040, in archive
_archive_imap(mailbox_name, final_archive_name)
File "/usr/bin/archivemail", line 1258, in _archive_imap
'+FLAGS.SILENT', '\\Deleted')
File "/usr/lib/python2.3/imaplib.py", line 670, in store
typ, dat = self._simple_command('STORE',
message_set, command, flags)
File "/usr/lib/python2.3/imaplib.py", line 1000, in
_simple_command
return self._command_complete(name,
self._command(name, *args))
File "/usr/lib/python2.3/imaplib.py", line 832, in
_command_complete
raise self.abort('command: %s => %s' % (name, val))
imaplib.abort: command: STORE => socket error: EOF
cleaning up ...

The mailbox is archived correct, but the archived
messages in the folder are not deleted.
I'm not sure whether this is an archivemail- or
imaplib-bug. I hope to have reported it at the right place.

Regards,
Florian

Discussion

  • Morten Lied Johansen

    Logged In: YES
    user_id=574105

    I have had to deal with this problem too, and my tests
    suggests that the command to delete a message from the IMAP
    server gets a list of messages to delete, and that there is
    a limit to the lenght of that list.

    The solution could probably be to split the command into
    manageable chunks before sending it to the server?

     
  • Peter Poeml

    Peter Poeml - 2006-06-18
    • status: open --> closed-fixed
     
  • Peter Poeml

    Peter Poeml - 2006-06-18

    Logged In: YES
    user_id=78531

    I fixed it with the following patch:

    --- archivemail.py (revision 97)
    +++ archivemail.py (working copy)
    @@ -1279,9 +1279,14 @@
    if archive:
    archive.close()
    archive.finalise()
    - vprint("Deleting messages")
    - imap_srv.store(string.join(message_list, ','),
    - '+FLAGS.SILENT', '\\Deleted')
    + # do not delete more than a certain number of messages at a time, because the
    + # command length is limited. This avoids that servers terminate the connection with
    + # EOF or TCP RST.
    + vprint("Deleting %s messages" % len(message_list))
    + max_delete = 100
    + for i in range(0, len(message_list), max_delete):
    + imap_srv.store(string.join(message_list[i:i+max_delete], ','),
    + '+FLAGS.SILENT', '\\Deleted')
    imap_srv.close()
    imap_srv.logout()

     

Log in to post a comment.