[ archivemail-Bugs-1491229 ] archivemail dies w/ MemoryError

[SourceForge.net <no...@so...>]

Bugs item #1491229, was opened at 2006-05-18 23:31
Message generated for change (Comment added) made by nikosch
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=456910&aid=1491229&group_id=49630

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
>Assigned to: Nikolaus Schulz (nikosch)
Summary: archivemail dies w/ MemoryError

Initial Comment:
Archivemail dies with a "MemoryError" on a certain
mbox. I can provide more debugging information if you
let me know what to give. The archivemail process
itself doesn't seem to be sucking down much memory
during this, so I don't think it's a problem with the
system running out of memory / swap.

This is on a 345M folder, which may contain some big
messages.

archivemail v0.6.1
$ archivemail -v -D "30 apr 2006"
/home/xxxxx/mymaildir/PG_Reports

message is 1812.20 days older than the specified date
message is not flagged important
decision: archive message
saving message to file
'/tmp/@27975.0archivemail/@27975.1archive.gz'
Traceback (most recent call last):
  File "/home/william/bin/archivemail", line 1298, in ?
    main()
  File "/home/william/bin/archivemail", line 619, in main
    archive(mailbox_path)
  File "/home/william/bin/archivemail", line 1036, in
archive
    _archive_mbox(mailbox_name, final_archive_name)
  File "/home/william/bin/archivemail", line 1105, in
_archive_mbox
    msg = original.next()
  File "/usr/lib/python2.2/mailbox.py", line 34, in next
    return self.factory(_Subfile(self.fp, start, stop))
  File "/usr/lib/python2.2/rfc822.py", line 106, in
__init__
    self.readheaders()
  File "/usr/lib/python2.2/rfc822.py", line 153, in
readheaders
    line = self.fp.readline()
  File "/usr/lib/python2.2/mailbox.py", line 63, in
readline
    data = self.fp.readline(length)
MemoryError


----------------------------------------------------------------------

>Comment By: Nikolaus Schulz (nikosch)
Date: 2006-11-08 13:23

Message:
Logged In: YES 
user_id=1594781

I cannot reproduce this. My box has 256MB memory. I've disabled swap and hammered archivemail with a 370MB mbox containing 52000 messages and at least one large message (26 MB attachment). 274MB got archived. No problem. 
TTBOMK even no indication that the system's struggling with memory. 

Besides, it is the rfc822 module throwing the exception, so I guess it's the module's implementation that is overstretched here, not archivemail itself. 

But I'd be interested in mailboxes triggering that exception, so if you think you can give away one that hasn't too private content, feel free to email me about that. (Of course we would have to stick to FTP or HTTP for the transfer of such a large mailbox.)

Hmm, do I understand correctly that it depends on the total volume of the mailboxes an archivemail process is archiving?  Weird. I fail to see how multiple mailboxes can add up in any way to hit some limit. 

Nikolaus


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2006-06-13 23:22

Message:
Logged In: NO 

I think the problem is when pass a large amount of data to a
single archivemail process. I was able to work around it
using a little bit of bash:

for i in `ls /var/spool/mail`; do archivemail -d 60 -o
/var/spool/bak /var/spool/mail/$i; done

hope this helps.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2006-06-06 00:45

Message:
Logged In: NO 

I'm having the same exact problem. I corrected it by
removing the message it got stuck on (after seeing it with
the -v arg), but I'd rather have it go through without
erroring out randomly.

Micheal

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=456910&aid=1491229&group_id=49630