Re: [fetchmail-devel] Re: [fetchmail-users] Incorrect header line and lost mails

[Frederic M. <fre...@wo...>]

Matthias Andree wrote:
> Greetings,
>
> I think we'll shelve this problem and revisit in 6.4.X. I've thought
> about it again, and there is no good solution. It is unclear whether
> some mailer spoiled the blank line between header and body, and the
> incorrect header line delimits header from body, or whether it's just
> broken folding as observed by you.
>   
> I am very much in favor of Rob MacGregor's suggestion of putting the
> message in a message/rfc822 container, and if user addresses have been
> figured from the header, forward there, else to the fallback postmaster
> (usually the calling user, literally "postmaster" if run as root, or
> whichever is configured in the rcfile), and I'd be more comfortable if
> such major changes can get testing first before being deployed. If it
> works well, it might get backported to 6.3.X depending on how fast those
> evolve.
>
> I hope that's acceptable. If you want messages passed on in spite of
> problematic headers, and let other parts of the mail system sort things
> out, try the attached patch. It's highly experimental, may cause older
> bugs to reappear that were masked (which please report, particularly
> segfaults!), and isn't suitable for application in production or
> distributions.
>   

Thank you Matthias, but I patched it myself as I promised. I should have 
told you about it, but I haven't said a word because I'm experiencing 
some big UIDL troubles with version 6.3.2 (I back ported my fix to that 
version to try it in production) and I don't know if the problem is 
related to my fix or if it is a known problem in 6.3.2 (as the recent 
e-mails on this list tend to say).

Here is a suspicious case from my log with the "keep" option active:

Feb 20 17:36:26 localhost fetchmail[14623]: mise en sommeil à lun 20 fév 
2006 17:36:26 CET
Feb 20 17:41:26 localhost fetchmail[14623]: réveillé à lun 20 fév 2006 
17:41:26 CET
Feb 20 17:41:29 localhost fetchmail[14623]: 1353 messages (1353 déjà 
vus) pour xxx#yyy.be dans 192.168.100.1 (70802116 octets).
Feb 20 17:41:29 localhost fetchmail[14623]: mise en sommeil à lun 20 fév 
2006 17:41:29 CET
Feb 20 17:46:29 localhost fetchmail[14623]: réveillé à lun 20 fév 2006 
17:46:29 CET
Feb 20 17:46:31 localhost fetchmail[14623]: 1355 messages (1355 déjà 
vus) pour xxx#yyy.be dans 192.168.100.1 (70808100 octets).
Feb 20 17:46:31 localhost fetchmail[14623]: mise en sommeil à lun 20 fév 
2006 17:46:31 CET
Feb 20 17:51:31 localhost fetchmail[14623]: réveillé à lun 20 fév 2006 
17:51:31 CET
Feb 20 17:51:34 localhost fetchmail[14623]: 1358 messages (1358 déjà 
vus) pour xxx#yyy.be dans 192.168.100.1 (70825184 octets).
Feb 20 17:51:34 localhost fetchmail[14623]: mise en sommeil à lun 20 fév 
2006 17:51:34 CET

Why is it reporting 1353 messages seen, then 1355 and then 1358 without 
downloading any mail ? I have no answer and I can't figure out why it 
would be related to my patch. Moreover, and this is related to the same 
problem, Fetchmail keeps downloading some e-mails two or three times or 
it delays some e-mails until it eventually download them. It looks like 
the UIDL management is seriously broken but it may be on my version 
only. I just have to find out...

If I remove the "keep" option everything looks good and I can't see any 
mail loss but, on the other hand, I can't compare the mails on the 
server to those actually downloaded.

I also think my fix won't be acceptable to you because I let the message 
pass without any change (I can't encapsulate the offending e-mail in a 
container. It is beyond the time I can spend on this project) and I keep 
processing the e-mail until the first blank line is found even if it is 
in the body (I don't know what would happen if the end of the mail was 
reached without encountering a blank line).

Anyway, if you want to have a look at my patch in its current state, I 
can send it to you.

Frederic