Files are not cleaned up
Status: Beta
Brought to you by:
waynemcdougall
Menu
▾
▴
#114 Files are not cleaned up
I seem to be getting the following error once every day
or two:
Unexpected error 7 Out of memory Connection x File y
I'm not too worried about the actual error (although I
can't understand why it would be out of memory, since
the system usually has > 500 MB memory free), but it
seems to be causing something weird...
I noticed that there were quite a few files in some of
the scan folders under Fluffy's (numbered) thread
directories. As the matter of fact, about 5 (out of 25)
scan folders have a few thousand zip files in them.
When I looked closer, it turned out that the oldest file in
each of these folders was timestamped exactly at the
time that the above error appears in the logs.
So it seems as if Fluffy doesn't cleanup any of the files
it creates after the unexpected error. I'm not sure if the
files get cleaned up if I shutdown Fluffy, I didn't have a
reason to shut it down for the last couple of weeks.
Surely I can manually delete the files, but I thought it
would be interesting to investigate before it erases its
tracks...
I'm using Flufy 1.4.92a. Note that we are processing
about 10k emails through Fluffy per day.
Robert
Fluffy release 1.4.94
Logged In: YES
user_id=660239
I think the most likely cause is that a particularly malformed
message causes Fluffy to go into an infinite loop rescursively
decomposing the message into MIME parts. I'd like to fix this,
but to do that I'd really need an exact copy of the message.
You could perhaps look at your logs to figure out what
message causes this error. Or using 1.4.95 I hope Fluffy will
save an offending message as infiniteloop.txt in the Fluffy
folder.
The left-over files cause no harm and are deleted when the
next incoming mail message uses that same connection
number.
Logged In: YES
user_id=898950
This is what I see:
<23>Dec 29, 17:47:16 relay1 Fluffy Open connection 13
206.156.53.150
<22>Dec 29, 17:47:16 relay1 Fluffy Immediate provisional
acceptance of delivery to SHUTCHINGS@MYDOMAIN.COM
from <SENDER@bloomberg.net> 206.156.53.150
<19>Dec 29, 17:47:17 relay1 Fluffy Unexpected error 0 Out
of memory Connection 13 File number 428
<23>Dec 29, 17:47:18 relay1 Fluffy Closing connection 13
206.156.53.150
In the folder <Fluffy>\13\scan\ there is a file 23139.zip (the
oldest one) which is timestamped 17:47:17 - the exact
date/time of the error.
The funny thing is that this file is 0 bytes. I've checked the
other 5 "problem folders" and it turns out that in all of them,
the oldest file is empty. That file always has the same
date/time as the corresponding error in the logs.
Furthermore, I should add that the left-over files are actually
not deleted (until Fluffy is restarted?). Instead it seems that
it adds a file for every new connection. At some point, I got a
run-time error 61 (Disk full), which may or may not be related
to this problem. Either way, the disk wasn't full at all. So I
wonder if this may have been caused by a limitation to the
number of files in the scan folder(s).
Logged In: YES
user_id=660239
1. The files in scan are of no interest. It is the file mail.zip in
the \13\ folder that I need to see.
2. Disk full will occur because the disk does get full (you don't
see the true file size until the file is closed, and it won't be
closed until Fluffy ends). However after that error Fluffy
recover.
3. The files do get purged eventually and cause no harm.
4. But the best solution is to stop the problem from occurring
at all. For which I need a copy of that malformed message to
find the error case I'm not handling correctly.
5. You're my best hope because you say you get this every
couple of days, where most people it might be every 2
months or so.
Logged In: YES
user_id=898950
OK, I will start using 1.4.95 (as soon as you release it) hoping
that it will catch the offending file. I've checked my most
recent logs and it seems that the error is still occuring once
every 2/3 days (and 4 times on 1 particular day). So I should
be able to catch it.
I will send you a couple of logs. Perhaps you can figure out
what is going on.
As for the disk full error, I'm sure that the disk was not full at
that time. As the matter of fact, it had several GB's free.
Many thanks for your help!
Logged In: YES
user_id=660239
1.4.94 (attached to this thread) will catch the offending file.
I've had a Gb fill up with a recursive error..as soon as the
error occurs the file is deleted and the GB all come back.
But maybe that's not the case here. I guess it could also be
an individual file size - can't be larger than 2 Gb or 4 Gb or
something, isn't it?
Logged In: YES
user_id=660239
This is fixed in patch release 1.5.0.
The cause is a mail message that contains a blank MIME
section that is marked as being encoded in Base64.
Microsoft reports a string conversion performed on an emtpy
byte array as an "out of memory error". Gee thanks.
Logged In: NO
Cool! I'm glad you found it. I have applied 1.5.0 a few days
ago and I have NOT seen the error anymore. Great job!