fetchmail-devel — Development related discussion

[fetchmail-devel] Re: handle imap EXPUNGE better

[Sunil S. <sh...@bo...>]

[Sorry for the delayed reply]

Quoting from Matthias Andree's mail on Thu, Dec 29, 2005:
> Sunil Shetye <sh...@bo...> writes:
> I have some concerns about this:
> 
> > +		    /* Some servers do not report RECENT after an EXPUNGE.
> > +		     * For such servers, assume that the mail being
> > +		     * expunged is a recent one. For other servers, we
> > +		     * should get an updated RECENT report later and this
> > +		     * assumption will have no effect. */
> > +		    if (recentcount > 0)
> > +			recentcount--;
> > +		    actual_deletions++;
> 
> How do we know that the other client deleting messages is deleting
> recent ones?  A getmail client with "delete_after" or moldremover or
> such might remove old and seen messages and we're decreasing RECENT
> without good reason. It looks as though the assumption were a bit bold.

This issue affects only those imap servers who do not report RECENT on
EXPUNGE. All the good imap servers do report RECENT like this:

IMAP> A0158 EXPUNGE
IMAP< * 1 EXPUNGE
IMAP< * 3 EXISTS
IMAP< * 3 RECENT                 [ some imap servers do not report this line ]
IMAP< A0158 OK EXPUNGE completed

> It would be better to update the cache properly (watching out for
> eventual iterator variables that need to be decremented, too) and
> continue. From our own cached flags we should be able to deduce if the
> expunged message was recent or not. If we don't know and the server
> expunged a different mail.
> 
> Now, before you go delve into the code and hack away, take a step back
> from the blackboard to see the whole picture: the RECENT flags are just
> crap if several clients are accessing the mailbox. What happens to the
> RECENT count if we reselect the mailbox? Can we actually use it? Do we
> need to check \Seen flags instead, for lack of UID support in IMAP?

I plan to implement the recentcount as a difference between EXISTS
count at end and beginning of poll. I can work on that only after this
patch is accepted. At that time, your point will also get implemented.

> The long way to a fix, which is not going to be in 6.3.X, but 6.4.0 at
> the earliest, will be to rewrite the UID code to be
> protocol-independent, allow for storing attributes with messages (such
> as arrival or retrieval timestamp, size) in the .fetchids file, and
> store one file per (server, account, folder) tuple. We have your code
> from 2003 that splits the uid files, and I made some .fetchids
> reformatting change on top of that.  I still find uid.c ugly, and I
> doubt it's up to the job, so we'll have to rewrite that.
> 
> Next, we'd have to teach the IMAP code to use UID/UIDVALIDITY on servers
> that support it for client-side tracking and make that the default, and
> make POP3+UIDL default. We have IMAP patches that use the UID, but
> AFAICT they don't pay attention to the UIDVALIDITY and so require some
> work. Again, this is all 6.4.X stuff.
> 
> Comments, suggestions?

fetchmail uses the RECENT only to decide
a) whether to repoll at the end of mailbox poll
b) whether to come out of IDLE (or NOOP look) while waiting for new
mail.
The UID code is not going to help here as this would require getting
the full list of UIDs at the end of mailbox poll (and every minute,
with "idle"). The RECENT code is much cleaner as imap servers reports
the RECENT count consistently and correctly.

The RECENT count is not being used to decide which mails are new. It
is only being used at the end of the poll or while idling for new
mail.

-- 
Sunil Shetye.

Re: [fetchmail-devel] BUG (?) fetchmail 6.3.2 hangs with 'idle' on OpenBSD

[Casper G. <cas...@ko...>]

Well, without actually knowing what I'm doing (I think) this is 
how I fixed it:

-------------------------------------------------------------
*** imap_orig.c Mon Feb 20 02:26:35 2006
--- imap.c      Mon Feb 20 02:17:05 2006
***************
*** 110,115 ****
--- 110,118 ----

                    mytimeout = saved_timeout;
                    stage = STAGE_FETCH;
+
+                   recentcount = count;
+                   return PS_SUCCESS;
                }
            }
            else if (strstr(buf, " RECENT"))
-------------------------------------------------------------

Seems somehow the 'stage = STAGE_FETCH' never gets
any action if we don't do it like that. Otherwise on
the next round we'll just be stage = STAGE_IDLE again,
and nobody ever checks if we went to FETCH in between.

In fact as far as I could see changing the stage
variable has no effect whatsoever, it's the recentcount
that does the trick. It gets imap_getrange() out of
its stubborn imap_idle() loop, and then things start
to happen, and I get my mail.

Works for me. Could break something on a server
supporting IDLE though, not sure about that. Someone
might want to check that?

Casper

[fetchmail-devel] BUG (?) fetchmail 6.3.2 hangs with 'idle' on OpenBSD

[Casper G. <cas...@ko...>]

I include here full gdb, -V, and -v -v output. Here's a description
of the problem:

I'm running 6.3.2 on OpenBSD 3.3 to an IMAP server with 'idle' in the 
config file. The server does not support the IDLE command, so fetchmail
does the NOOP stuff.

Everything works fine until fetchmail enters the NOOP loop
_AND_ it receives a new incoming mail notification. I.e. the
IMAP server says:

fetchmail: IMAP< * 1 EXISTS

..then fetchmail hangs..it won't continue until a 300 second
timeout passes, and after that it reports:

fetchmail: socket error while fetching from ea...@po...
fetchmail: 6.3.2 querying pop.kolumbus.fi (protocol IMAP) at Sun, 19 Feb 2006 23:43:40 +0200 (EET): poll completed
fetchmail: Query status=2 (SOCKET)

..and then the normal cycle starts again. 

Here's all the debug info:

----------------------------------------------------------------------

Configurarion options:
> ./configure --enable-NTLM --with-ssl --disable-nls

----------------------------------------------------------------------

Config file:

set postmaster "casper"
set bouncemail
set no spambounce
set properties ""
set daemon 10
poll pop.kolumbus.fi with proto IMAP port 143
  user 'ea5540' there with password 'xxxxxxx' is 'casper' here options no keep idle

----------------------------------------------------------------------

> fetchmail -V
This is fetchmail release 6.3.2+NTLM+SSL.

Copyright (C) 2002, 2003 Eric S. Raymond
Copyright (C) 2004 Matthias Andree, Eric S. Raymond, Rob F. Funk, Graham Wilson
Copyright (C) 2005 Matthias Andree, Sunil Shetye
Copyright (C) 2006 Matthias Andree
Fetchmail comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. For details,
please see the file COPYING in the source or documentation directory.

Fallback MDA: (none)
OpenBSD zoidberg.homeip.net 3.3 GENERIC_NOIPV6#0 i386
Taking options from command line and /home/casper/.fetchmailrc
Poll interval is 10 seconds
Idfile is /home/casper/.fetchids
Fetchmail will forward misaddressed multidrop messages to casper.
Options for retrieving from ea...@po...:
  True name of server is pop.kolumbus.fi.
  Protocol is IMAP (using service 143).
  All available authentication methods will be tried.
  Server nonresponse timeout is 300 seconds (default).
  Default mailbox selected.
  Only new messages will be retrieved (--all off).
  Fetched messages will not be kept on the server (--keep off).
  Old messages will not be flushed before message retrieval (--flush off).
  Oversized messages will not be flushed before message retrieval (--limitflush off).
  Rewrite of server-local addresses is enabled (--norewrite off).
  Carriage-return stripping is disabled (stripcr off).
  Carriage-return forcing is disabled (forcecr off).
  Interpretation of Content-Transfer-Encoding is enabled (pass8bits off).
  MIME decoding is disabled (mimedecode off).
  Idle after poll is disabled (idle off).
  Nonempty Status lines will be kept (dropstatus off)
  Delivered-To lines will be kept (dropdelivered off)
  Fetch message size limit is 100 (--fetchsizelimit 100).
  Do binary search of UIDs during 9 out of 10 polls (--fastuidl 10).
  Messages will be SMTP-forwarded to: localhost (default)
  Single-drop mode: 1 local name recognized.
  No UIDs saved from this host.

------------------------------------------------------------------------
Running with -v -v + gdb session:

zoidberg [Sun 19 23:33] <~/tmp/fetchmail-6.3.2>: ./fetchmail -N -v -v
fetchmail: starting fetchmail 6.3.2 daemon
fetchmail: 6.3.2 querying pop.kolumbus.fi (protocol IMAP) at Sun, 19 Feb 2006 23:33:25 +0200 (EET): poll started
fetchmail: IMAP< * OK IMAP4 server (InterMail vM.6.01.04.01 201-2131-118-101-20041129) ready Sun, 19 Feb 2006 23:33:25 +0200 (EET)
fetchmail: IMAP> A0001 CAPABILITY
fetchmail: IMAP< * CAPABILITY IMAP4rev1 UIDPLUS NAMESPACE QUOTA
fetchmail: IMAP< A0001 OK CAPABILITY completed
fetchmail: Protocol identified as IMAP4 rev 1
fetchmail: will idle after poll
fetchmail: IMAP> A0002 LOGIN "ea5540" *
fetchmail: IMAP< A0002 OK LOGIN completed
fetchmail: selecting or re-polling default folder
fetchmail: IMAP> A0003 SELECT "INBOX"
fetchmail: IMAP< * 0 EXISTS
fetchmail: IMAP< * OK [UIDVALIDITY 1031331700] UIDs valid
fetchmail: IMAP< * OK [UIDNEXT 1000] Predicted next UID
fetchmail: IMAP< * FLAGS (\Answered \Flagged \Deleted \Draft \Seen)
fetchmail: IMAP< * OK [PERMANENTFLAGS (\* \Answered \Flagged \Deleted \Draft \Seen)] Permanent flags
fetchmail: IMAP< * 0 RECENT
fetchmail: IMAP< A0003 OK [READ-WRITE] SELECT completed
fetchmail: 0 messages waiting after first poll
fetchmail: IMAP> A0004 NOOP
fetchmail: IMAP< A0004 OK NOOP completed
fetchmail: IMAP> A0005 NOOP
fetchmail: IMAP< A0005 OK NOOP completed
fetchmail: IMAP< * 1 EXISTS
fetchmail: IMAP< * 2 EXISTS
fetchmail: IMAP< * 3 EXISTS
fetchmail: IMAP< * 4 EXISTS

Here fetchmail hangs..nothing happens for 300 secs (unless a new
"EXISTS" event arrives and then the 300 secs timer resets)..it's totally
dead. Sending SIGHUP doesn't help. So I enter GDB on another terminal:

(gdb) attach 14837
Attaching to program `/home/casper/tmp/fetchmail-6.3.2/fetchmail', process 14837
Reading symbols from /usr/libexec/ld.so...done.
Reading symbols from /usr/lib/libssl.so.7.0...done.
Reading symbols from /usr/lib/libcrypto.so.9.0...done.
Reading symbols from /usr/lib/libc.so.29.0...done.
0x401cb7af in recvfrom ()
(gdb) where
#0  0x401cb7af in recvfrom ()
#1  0x401a8eff in recv ()
#2  0x22cf in SockRead (sock=3, buf=0xcfbf6ae8 "* 2 EXISTS", len=8192)
    at socket.c:467
#3  0x1c5f9 in gen_recv (sock=3, buf=0xcfbf6ae8 "* 2 EXISTS", size=8193)
    at transact.c:1483
#4  0x6448 in imap_ok (sock=3, argbuf=0x0) at imap.c:60
#5  0x7626 in imap_idle (sock=3) at imap.c:637
#6  0x7ab0 in imap_getrange (sock=3, ctl=0x43000, folder=0x0,
    countp=0xcfbfabec, newp=0xcfbfabe8, bytes=0xcfbfabe4) at imap.c:717
#7  0x18294 in do_session (ctl=0x43000, proto=0x8b00, maxfetch=0)
    at driver.c:1307
#8  0x18e78 in do_protocol (ctl=0x43000, proto=0x8b00) at driver.c:1630
#9  0x8b6f in doIMAP (ctl=0x43000) at imap.c:1199
#10 0xdff8 in query_host (ctl=0x43000) at fetchmail.c:1413
#11 0xb81a in main (argc=4, argv=0xcfbfd1c4) at fetchmail.c:685
(gdb) up
#1  0x401a8eff in recv ()
(gdb) up
#2  0x22cf in SockRead (sock=3, buf=0xcfbf6ae8 "* 2 EXISTS", len=8192)
    at socket.c:467
467                 if ((n = fm_peek(sock, bp, len)) <= 0)

(gdb) attach 14837
Attaching to program `/home/casper/tmp/fetchmail-6.3.2/fetchmail', process 14837
Reading symbols from /usr/libexec/ld.so...done.
Reading symbols from /usr/lib/libssl.so.7.0...done.
Reading symbols from /usr/lib/libcrypto.so.9.0...done.
Reading symbols from /usr/lib/libc.so.29.0...done.
0x401cb7af in recvfrom ()
(gdb) where
#0  0x401cb7af in recvfrom ()
#1  0x401a8eff in recv ()
#2  0x22cf in SockRead (sock=3, buf=0xcfbf6ae8 "* 2 EXISTS", len=8192)
    at socket.c:467
#3  0x1c5f9 in gen_recv (sock=3, buf=0xcfbf6ae8 "* 2 EXISTS", size=8193)
    at transact.c:1483
#4  0x6448 in imap_ok (sock=3, argbuf=0x0) at imap.c:60
#5  0x7626 in imap_idle (sock=3) at imap.c:637
#6  0x7ab0 in imap_getrange (sock=3, ctl=0x43000, folder=0x0,
    countp=0xcfbfabec, newp=0xcfbfabe8, bytes=0xcfbfabe4) at imap.c:717
#7  0x18294 in do_session (ctl=0x43000, proto=0x8b00, maxfetch=0)
    at driver.c:1307
#8  0x18e78 in do_protocol (ctl=0x43000, proto=0x8b00) at driver.c:1630
#9  0x8b6f in doIMAP (ctl=0x43000) at imap.c:1199
#10 0xdff8 in query_host (ctl=0x43000) at fetchmail.c:1413
#11 0xb81a in main (argc=4, argv=0xcfbfd1c4) at fetchmail.c:685
(gdb) up
#1  0x401a8eff in recv ()
(gdb) up
#2  0x22cf in SockRead (sock=3, buf=0xcfbf6ae8 "* 2 EXISTS", len=8192)
    at socket.c:467
467                 if ((n = fm_peek(sock, bp, len)) <= 0)

It's stuck on the 'peek'??

(gdb) list
462             {
463
464     #ifdef __BEOS__
465                 if ((n = fm_read(sock, bp, 1)) <= 0)
466     #else
467                 if ((n = fm_peek(sock, bp, len)) <= 0)
468     #endif
469                     return (-1);
470     #ifdef FORCE_STUFFING
471                 maxavailable = n;
(gdb) up
#3  0x1c5f9 in gen_recv (sock=3, buf=0xcfbf6ae8 "* 2 EXISTS", size=8193)
    at transact.c:1483
1483        if (SockRead(sock, buf, size) == -1)
(gdb) list
1478    {
1479        int oldphase = phase;       /* we don't have to be re-entrant */
1480
1481        phase = SERVER_WAIT;
1482        set_timeout(mytimeout);
1483        if (SockRead(sock, buf, size) == -1)
1484        {
1485            set_timeout(0);
1486            phase = oldphase;
1487            if(is_idletimeout())
(gdb) print mytimeout
$1 = 300
(gdb) detach
Detaching from program: /home/casper/tmp/fetchmail-6.3.2/fetchmail process 14837

Timeout is indeed 300...so I detach and wait to see what happens,
and sure enough after 300 seconds it wakes up:

fetchmail: timeout after 300 seconds waiting for server pop.kolumbus.fi.
fetchmail: socket error while fetching from ea...@po...
fetchmail: 6.3.2 querying pop.kolumbus.fi (protocol IMAP) at Sun, 19 Feb 2006 23:43:40 +0200 (EET): poll completed
fetchmail: Query status=2 (SOCKET)
fetchmail: Deleting fetchids file.
fetchmail: sleeping at Sun, 19 Feb 2006 23:43:40 +0200 (EET)
fetchmail: awakened at Sun, 19 Feb 2006 23:43:50 +0200 (EET)
fetchmail: 6.3.2 querying pop.kolumbus.fi (protocol IMAP) at Sun, 19 Feb 2006 23:43:50 +0200 (EET): poll started
fetchmail: IMAP< * OK IMAP4 server (InterMail vM.6.01.04.01 201-2131-118-101-20041129) ready Sun, 19 Feb 2006 23:43:50 +0200 (EET)
fetchmail: IMAP> A0001 CAPABILITY
fetchmail: IMAP< * CAPABILITY IMAP4rev1 UIDPLUS NAMESPACE QUOTA
...

..and the mail is finally delivered. This cycle continues ad infinitum..
It hangs every time a new mail is reported by the IMAP server, and
then continues after 300 secs :(

Any ideas what could be wrong?

Thanks.

Regards,
  Casper

Re: [fetchmail-devel] Re: [fetchmail-users] Incorrect header line and lost mails

[Matthias A. <mat...@gm...>]

Frederic Marchal <fre...@wo...> writes:

> Matthias Andree wrote:
>> Frederic Marchal <fre...@wo...> writes:
>>
>>
>>> I expect this mail will be wrapped so here is the explanation. The
>>> References header is one long line and is wrapped at some point after
>>> the <43ea8d070602140508w6af9d34aq5c which leaves the
>>> 222...@ma...> all by itself on a line. You have
>>> received that e-mail and can compare it to your copy
>>>
>>> On my side, the References line is truncated at column 263. rfc821
>>> specifies a maximum line length of 1000 characters but an implementation
>>> should be prepared for longer lines or reject the mail. It is not what
>>> is happening here.
>>>
>>> The culprit may be Mercury/32 or the Wingate proxy which is not listed
>>> in the header.
>>>
>>
>> Is there any chance we might isolate the bug before we fix it?
> Isolate in what ? Mercury or fetchmail ?

Finding out if it's in Mercury/32 or Wingate or outside both.

> In fetchmail, the problem is clearly located in transact.c (I wouldn't
> call it a bug until I get some clarification from the one who wrote
> it). The part of the code that rejects the mail is preceded by a comment
> stating
>
>         * At least one brain-dead website (netmind.com) is known to
>         * send out robotmail that's missing the RFC822 delimiter blank
>         * line before the body! Without this check fetchmail segfaults.
>         * With it, we treat such messages as spam and refuse them.
>
> But the test after this comment doesn't test for the beginning of the
> body but for a header line not starting with a blank and not containing
> a colon.

Which is assuming that it's looking at a body line rather than a header,
on the assumption that all header lines either start with whitespace, or
with a sequence of non-space characters terminated with a colon (":").

I'm not sure if it's still needed, and the segfault had probably better
be fixed in the place where it actually occurs, but that's stuff for
6.4.0, I'm not going to attempt removing this workaround in 6.3.X to
avoid user astonishment.

-- 
Matthias Andree

Re: [fetchmail-devel] Incorrect README.svn

[Matthias A. <mat...@gm...>]

Frederic Marchal <fre...@wo...> writes:

> Hello,
>
> The file README.svn states that automake 1.7 or above is needed to run
> autogen.sh. I had automake 1.7.9 and it failed with the message
>
> Makefile.am:4: option `no-dist-gzip' not recognized
>
> I upgraded to automake 1.9.5 and it succeeded. Version 1.8 may be the
> minimum.

Indeed it is. I'm not sure which autoconf versions older than 2.59 work,
but since 2.59 is more than two years old, I don't care for older
versions.

I'm not sure about the gettext version. The current code expects 0.14.3,
it may or may not work with older versions, I'm not sure either.

Anyways, the new requirements (automake >= 1.8, autoconf >= 2.59) are in
README.svn.

Merci beaucoup de rapporter ce problème.

-- 
Matthias Andree

[fetchmail-devel] Incorrect README.svn

[Frederic M. <fre...@wo...>]

Hello,

The file README.svn states that automake 1.7 or above is needed to run 
autogen.sh. I had automake 1.7.9 and it failed with the message

Makefile.am:4: option `no-dist-gzip' not recognized

I upgraded to automake 1.9.5 and it succeeded. Version 1.8 may be the 
minimum.

Frederic

Re: [fetchmail-devel] Re: [fetchmail-users] Incorrect header line and lost mails

[Frederic M. <fre...@wo...>]

Matthias Andree wrote:
> Frederic Marchal <fre...@wo...> writes:
>
>   
>> I expect this mail will be wrapped so here is the explanation. The
>> References header is one long line and is wrapped at some point after
>> the <43ea8d070602140508w6af9d34aq5c which leaves the
>> 222...@ma...> all by itself on a line. You have
>> received that e-mail and can compare it to your copy
>>
>> On my side, the References line is truncated at column 263. rfc821
>> specifies a maximum line length of 1000 characters but an implementation
>> should be prepared for longer lines or reject the mail. It is not what
>> is happening here.
>>
>> The culprit may be Mercury/32 or the Wingate proxy which is not listed
>> in the header.
>>     
>
> Is there any chance we might isolate the bug before we fix it? 
>   
Isolate in what ? Mercury or fetchmail ?

I installed a Mercury/32 server on my computer and I made some tests. 
The header remains intact until it is fetched from the POP3 server by 
the client. This bug is not (yet) mentioned on the mailing list of the 
project. It is a closed source project and there seems to be only one 
very busy developer. I won't get any help from there.

In fetchmail, the problem is clearly located in transact.c (I wouldn't 
call it a bug until I get some clarification from the one who wrote it). 
The part of the code that rejects the mail is preceded by a comment stating

         * At least one brain-dead website (netmind.com) is known to
         * send out robotmail that's missing the RFC822 delimiter blank
         * line before the body! Without this check fetchmail segfaults.
         * With it, we treat such messages as spam and refuse them.

But the test after this comment doesn't test for the beginning of the 
body but for a header line not starting with a blank and not containing 
a colon. If the condition is true, refuse_mail is set to 1 and the 
function returns PS_REFUSED which delete the mail on the server.

Frederic

Re: [fetchmail-devel] Re: [fetchmail-users] Incorrect header line and lost mails

[Matthias A. <mat...@gm...>]

Frederic Marchal <fre...@wo...> writes:

> Matthias (and others),
>
> I have some more questions and requests for guidance...
>
> - Should I patch the source from 6.3.2 or the one in the SVN
> repository ?

The SVN's branches/BRANCH_6-3/ would be the most useful place at this
time for fixes, and trunk/ for feature patches.

> - I will simply ignore the invalid lines (unless someone has a better
> idea).

Do you mean, skip the header, or include it without complaining?

> I think it should be user selectable. Is it suitable to have that
> option in the rcfile and not available from the command line ?

Personally, I do not appreciate such inconsistencies.

> - The option would be set with a SET statement in the rcfile. Is there
>any reason to make it configurable for each server ?

That depends on whether it's switching off the complaint (in that case,
global might be enough) or actually deleting the header. The better
option in any case is server or user configurable.

> - I intent to add the variable of the option in struct runctl if it is
> global and to struct hostdata if it is local to a server. Is it the good
> way of doing it ?

It's the only reasonable way of doing it. :-)

> - Does anybody have a suitable name for that option ?

That depends on what it does. It should default to "off" for compatibility.

-- 
Matthias Andree

Re: [fetchmail-devel] Re: [fetchmail-users] Incorrect header line and lost mails

[Matthias A. <mat...@gm...>]

Frederic Marchal <fre...@wo...> writes:

> I expect this mail will be wrapped so here is the explanation. The
> References header is one long line and is wrapped at some point after
> the <43ea8d070602140508w6af9d34aq5c which leaves the
> 222...@ma...> all by itself on a line. You have
> received that e-mail and can compare it to your copy
>
> On my side, the References line is truncated at column 263. rfc821
> specifies a maximum line length of 1000 characters but an implementation
> should be prepared for longer lines or reject the mail. It is not what
> is happening here.
>
> The culprit may be Mercury/32 or the Wingate proxy which is not listed
> in the header.

Is there any chance we might isolate the bug before we fix it? 

Even if that isn't of immediate help for you, documenting this properly
might help other people solve their problem, and we may be able to write
a workaround in the least intrusive way.

> Anyway, I can't change any of them and nothing prevent the mail from
> being delivered. Moreover I can read it in thunderbird without any
> warning or inconvenience.

I think given the vast number of varieties how messages can be broken,
and one workaround might break another, the best bet is probably to wrap
the message up as a MIME message/rfc822 attachment that contains a short
introductory text that tells the end user the message contained broken
headers.

-- 
Matthias Andree

[fetchmail-devel] fetchmail in ibiblio (was: ibiblio action notice from the `keeper' program)

[Matthias A. <mat...@gm...>]

ke...@ib... writes:

> At about 2006-02-14 09:48:31 UCT, `keeper' moved the following files from
> Incoming to system/mail/pop/fetchmail (remote mail retrieval and forwarding utility intended for use over on-demand TCP/IP links):
>
> fetchmail-6.3.0.tar.bz2  Remote mail retrieval and forwarding utility intended for use over on-demand TCP/IP links.
> fetchmail.lsm            

Note I have uploaded a new fetchmail-6.3.2.tar.bz2,
which fixes security issues (denial of service) in 6.3.0.

Please remove these obsolete files.
After that, please consider if a separate fetchmail
directory is really necessary. I doubt it.

from the directory system/mail/pop/

File: fetchmail-5.9.13-1.i386.rpm  	375 KB  23.06.2002
File: fetchmail-5.9.13-1.src.rpm 	1005 KB 23.06.2002
File: fetchmail-5.9.13.tar.gz           994 KB 	23.06.2002
File: fetchmail-5.9.14-1.i386.rpm 	332 KB 	07.09.2002
File: fetchmail-5.9.14-1.lsm            1 KB 	09.09.2002
File: fetchmail-5.9.14-1.src.rpm 	972 KB 	07.09.2002
File: fetchmail-5.9.14.tar.gz           962 KB 	07.09.2002
File: fetchmail-6.1.0-1.i386.rpm 	377 KB 	23.09.2002
File: fetchmail-6.1.0-1.src.rpm 	1062 KB 23.09.2002
File: fetchmail-6.1.0.tar.gz            1051 KB 23.09.2002
File: fetchmail-6.1.1-1.i386.rpm 	382 KB 	19.10.2002
File: fetchmail-6.1.1-1.src.rpm 	1108 KB 19.10.2002
File: fetchmail-6.1.1.tar.gz            1097 KB 19.10.2002
File: fetchmail-6.1.2-1.i386.rpm 	383 KB 	01.11.2002
File: fetchmail-6.1.2-1.src.rpm 	1108 KB 01.11.2002
File: fetchmail-6.1.2.tar.gz            1097 KB 01.11.2002
File: fetchmail-6.2.2-1.i386.rpm 	385 KB 	01.03.2003
File: fetchmail-6.2.2-1.src.rpm 	1188 KB 01.03.2003
File: fetchmail-6.2.2.tar.gz            1177 KB 01.03.2003
File: fetchmail-FAQ.html                137 KB 	01.03.2003
File: fetchmail.README                  3 KB 	01.03.2003
File: fetchmail.lsm                     2 KB 	01.03.2003
File: fetchmail_pcre-5.0.5.README 	1 KB 	30.06.1999
File: fetchmail_pcre-5.2.8.lsm          1 KB 	18.02.2000
File: fetchmail_pcre-5.2.8.patch.gz 	17 KB 	18.02.2000

from the directory system/mail/pop/fetchmail/

File: fetchmail-6.1.2-1.i386.rpm  	383 KB  01.11.2002
File: fetchmail-6.1.2-1.src.rpm 	1108 KB 01.11.2002
File: fetchmail-6.1.2.tar.gz            1097 KB 01.11.2002
File: fetchmail-6.2.5.2.lsm             1 KB 	23.07.2005
File: fetchmail-6.2.5.2.tar.gz          1248 KB 23.07.2005
File: fetchmail-6.3.0.tar.bz2           1129 KB 04.12.2005
File: fetchmail-FAQ.html                136 KB 	01.11.2002
File: fetchmail.README                  3 KB 	01.11.2002
File: fetchmail.lsm                     2 KB 	04.12.2005

-- 
Matthias Andree

Re: [fetchmail-devel] Re: [fetchmail-users] Incorrect header line and lost mails

[Frederic M. <fre...@wo...>]

Matthias (and others),

I have some more questions and requests for guidance...

- Should I patch the source from 6.3.2 or the one in the SVN repository ?

- I will simply ignore the invalid lines (unless someone has a better 
idea). I think it should be user selectable. Is it suitable to have that 
option in the rcfile and not available from the command line ?

- The option would be set with a SET statement in the rcfile. Is there 
any reason to make it configurable for each server ?

- I intent to add the variable of the option in struct runctl if it is 
global and to struct hostdata if it is local to a server. Is it the good 
way of doing it ?

- Does anybody have a suitable name for that option ?

BTW, I already have a working version of the patch for 6.3.2 but I need 
to do some more testings and adapt it to the answers to the questions 
here above.

Frederic

Re: [fetchmail-devel] Re: [fetchmail-users] Incorrect header line and lost mails

[Frederic M. <fre...@wo...>]

Rob MacGregor wrote:
> On 2/14/06, Matthias Andree <mat...@gm...> wrote:
>   
> Personally, I'm curious to know what mail client is producing such
> borken emails, and what MTAs are passing them on.
>   
Here it is. I just got one from fetchmail-devel:

Received: from spooler by webmobile.be (Mercury/32 v4.01a); 14 Feb 2006 
14:47:07 +0100
X-Envelope-To: <fre...@wo...>
Return-path: <fet...@li...>
Received: from bat.berlios.de (195.37.77.135) by webmobile.be 
(Mercury/32 v4.01a) with ESMTP ID MG001FEF;
   14 Feb 2006 14:47:03 +0100
Received: from bat.berlios.de (localhost [127.0.0.1])
    by bat.berlios.de (8.11.3/8.11.3/SuSE Linux 8.11.1-0.5) with ESMTP 
id k1EDm2E08521;
    Tue, 14 Feb 2006 14:48:02 +0100
Received: from outmx024.isp.belgacom.be (outmx024.isp.belgacom.be 
[195.238.4.128])
    by bat.berlios.de (8.11.3/8.11.3/SuSE Linux 8.11.1-0.5) with ESMTP 
id k1EDlLE08506
    for <fet...@be...>; Tue, 14 Feb 2006 14:47:21 +0100
Received: from outmx024.isp.belgacom.be (localhost [127.0.0.1])
        by outmx024.isp.belgacom.be (8.12.11/8.12.11/Skynet-OUT-2.22) 
with ESMTP id k1EDlDjl009366
        for <fet...@be...>; Tue, 14 Feb 2006 14:47:13 +0100
        (envelope-from <fre...@wo...>)
Received: from [192.168.100.30] (167.191-201-80.adsl.skynet.be 
[80.201.191.167])
        by outmx024.isp.belgacom.be (8.12.11/8.12.11/Skynet-OUT-2.22) 
with ESMTP id k1EDl5Rq009276
        for <fet...@be...>; Tue, 14 Feb 2006 14:47:05 +0100
        (envelope-from <fre...@wo...>)
Message-ID: <43F...@wo...>
From: Frederic Marchal <fre...@wo...>
User-Agent: Thunderbird 1.5 (Windows/20051201)
MIME-Version: 1.0
To: fet...@be...
Subject: Re: [fetchmail-devel] Re: [fetchmail-users] Incorrect header line
 and lost mails
References: <43F...@wo...>    
 <43e...@ma...>    
 <m3h...@me...>    
 <43F...@wo...>    
 <m3o...@me...> <43ea8d070602140508w6af9d34aq5c
222...@ma...>
In-Reply-To: <43e...@ma...>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Sender: fet...@be...
Errors-To: fet...@be...
X-BeenThere: fet...@li...
X-Mailman-Version: 2.0.13
Precedence: bulk
List-Help: <mailto:fet...@li...?subject=help>
List-Post: <mailto:fet...@li...>
List-Subscribe: <http://lists.berlios.de/mailman/listinfo/fetchmail-devel>,
    <mailto:fet...@li...?subject=subscribe>
List-Id: <fetchmail-devel.lists.berlios.de>
List-Unsubscribe: 
<http://lists.berlios.de/mailman/listinfo/fetchmail-devel>,
    <mailto:fet...@li...?subject=unsubscribe>
List-Archive: <http://lists.berlios.de/pipermail/fetchmail-devel/>
Date: Tue, 14 Feb 2006 14:47:14 +0100
X-CC-Diagnostic: Body has "colon" (20)


I expect this mail will be wrapped so here is the explanation. The 
References header is one long line and is wrapped at some point after 
the <43ea8d070602140508w6af9d34aq5c which leaves the 
222...@ma...> all by itself on a line. You have 
received that e-mail and can compare it to your copy

On my side, the References line is truncated at column 263. rfc821 
specifies a maximum line length of 1000 characters but an implementation 
should be prepared for longer lines or reject the mail. It is not what 
is happening here.

The culprit may be Mercury/32 or the Wingate proxy which is not listed 
in the header. Anyway, I can't change any of them and nothing prevent 
the mail from being delivered. Moreover I can read it in thunderbird 
without any warning or inconvenience.

Frederic

Re: [fetchmail-devel] Re: [fetchmail-users] Incorrect header line and lost mails

[Frederic M. <fre...@wo...>]

Rob MacGregor wrote:
> On 2/14/06, Frederic Marchal <fre...@wo...> wrote:
>   
>> I believe it is not fetchmail's job to enforce rfc822 that way. It
>> should be left to a program dedicated to that task or, at least, it
>> should not be the default option. It would be a good idea to tag the
>> mail with a specific header though. It would make it easier to filter
>> out with a program such as procmail. I wouldn't divert the mail to the
>> postmaster either. It is likely that a busy postmaster will throw it
>> away without looking at it if he is not the obvious recipient.
>>     
>
> However, the postmaster defined by fetchmail doesn't have to be The
> Postmaster.  Of course, there's nothing to say that the option can't
> both have a global setting and a per-user override.
>   
In my case, that account (which is not The Postmaster) received 82 mails
for the last 6 hours and they ended up at the secretary that must deal
with 80% of spam from various addresses every day. I double check that
account from time to time and I salvage a few mails every month. Sending
a mail to an account where the user spend most of his/her time deleting
spams is unreliable at best :-).

With the limited experience I gained from our small particular net, I
think it is best to let the mail go unchanged if fetchmail can extract
enough information to determine the next route. It could also add a
valid X-Envelope-To if none was found but, as you said in a previous
post, anything beyond that would be risky...

Now, other e-mail processing configurations may have requirements I
don't see. So, feel free to tell me about it. It would help me to have a
clear picture of what fetchmail must deal with if I start changing the
code, preferably before I do it :-).

Frederic

Re: [fetchmail-devel] Re: [fetchmail-users] Incorrect header line and lost mails

[Frederic M. <fre...@wo...>]

Matthias Andree wrote:
> Frederic Marchal <fre...@wo...> writes:
>
>   
>> Well, I'm a C programmer and I may be able to help but I'm mainly a
>> windows programmer (yes, I know, shame on me :-) ) and I'm not yet all
>> too familiar with the Linux programming and especially with the patch
>> mechanism and none at all with the diff tool.
>>     
>
> The workflow is:
>
> 1. create a copy of all files you need to edit, for instance, 
>    copy foo.c to foo.c.orig
>
>    alternatively, copy the whole directory tree recursively
>
> 2. edit until you're happy with the code
>
> 3. test the edits (compile, install, run)
>
> 4. generate a unified (preferred) or context patch.  Not all diff
>    utilities support unified output, but all are to support context
>    format, as mandated by the relevant standard, IEEE Std 1003.1-2001.
>
>    Plain ed-style patches (unfortunately still the default in the diff
>    utility) are impractical so they are not usually accepted.
>
> The diff tool is quite simple actually, if you created file copies:
>
> diff -u OLDFILE NEWFILE >PATCHFILE (unified diff, preferred)
>
> diff -c OLDFILE NEWFILE >PATCHFILE (context diff, alternative)
>
> If you copied the whole directory before the edits:
>
> diff -u -r BACKUP_DIR EDITED_DIR >PATCHFILE    (or -c instead of -u)
>
> If you added new files to EDITED_DIR, you'd use -N but I don't think
> we'll need this here.
>
> diff, used either way, will produce a file (PATCHFILE) that you'd attach
> to a message and mail to me (or Rob, or preferably the -devel@ list) or
> upload to the BerliOS patch tracker.
>
> If I have the choice, I prefer the -devel@ list. That's public, archived
> and easiest for me since I don't need to fetch anything, but get it
> pushed into my inbox.
>
>
> The patch utility does the reverse and will usually just be used like
> this, it derives the file names from the PATCHFILE.
>
> patch <PATCHFILE
>
> perhaps with -d (to change directory) or -p (to strip leading path
> components from the paths shown in the patch file; used if patch cannot
> find the files).
>   
Thank you. It looks easy indeed but you saved me from a lot of 
man/info/howto readings... :-)


>> I'll also need some direction about the proper way of doing it for a
>> good integration in the current source.
>>     
>
> The bigger concern is that fetchmail shouldn't be emitting headers it
> knows are broken, so that the next hop (the MTA or MDA) does not get
> confused. I wonder if fetchmail should prefix the broken lines with
> X-Fetchmail-Escaped-Broken-Header: or something similar.
>   
At first glance it seemed reasonable, but after one day of thinking 
about it, I'm not so sure I understand why fetchmail should even alter 
the original header in any way.

Why is it fetchmail's concern if the next hop gets confused by an 
invalid header that was there in the first place ? Isn't it more useful 
to keep intact all the information from the original e-mail ? After all, 
an invalid header is also a signature for a spam or a virus and it could 
be used by some tool along the delivery chain to filter out the mail. In 
my case, a mail client could also detect the wrapping and restore the 
original header provided fetchmail doesn't mess it up.

Frederic

Re: [fetchmail-devel] Re: [fetchmail-users] Incorrect header line and lost mails

[Rob M. <rob...@gm...>]

On 2/14/06, Frederic Marchal <fre...@wo...> wrote:
>
> I believe it is not fetchmail's job to enforce rfc822 that way. It
> should be left to a program dedicated to that task or, at least, it
> should not be the default option. It would be a good idea to tag the
> mail with a specific header though. It would make it easier to filter
> out with a program such as procmail. I wouldn't divert the mail to the
> postmaster either. It is likely that a busy postmaster will throw it
> away without looking at it if he is not the obvious recipient.

However, the postmaster defined by fetchmail doesn't have to be The
Postmaster.  Of course, there's nothing to say that the option can't
both have a global setting and a per-user override.

I'd rather have the option of processing by email because at least in
the case of 3 fetchmail boxes I look after, procmail et all aren't
installed.  Nothing to say the 2 are exclusive however.

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-devel] Re: [fetchmail-users] Incorrect header line and lost mails

[Frederic M. <fre...@wo...>]

Rob MacGregor wrote:
> On 2/14/06, Matthias Andree <mat...@gm...> wrote:
>   
>> The bigger concern is that fetchmail shouldn't be emitting headers it
>> knows are broken, so that the next hop (the MTA or MDA) does not get
>> confused. I wonder if fetchmail should prefix the broken lines with
>> X-Fetchmail-Escaped-Broken-Header: or something similar.
>>     
>
> My 0.02$CURRENCY - when fetchmail finds a mail such as this it should
> create a new mail, addressed to the postmaster address, with the
> illegal email as a text/plain attachment.
>
> If you're feeling kind, insert a blank line before the offending
> illegal header and include that altered version as a message/rfc822
> attachment (but still have the text/plain attachment)
>
> There's no point in passing on the known broken email as is, and
> simply "fixing" it may result in more problems.  At least if the
> postmaster gets it as an attachment then they can decide how to handle
> it.
>
> Personally, I'm curious to know what mail client is producing such
> borken emails, and what MTAs are passing them on.
>
>   
This thread started in fetchmail-user.

My problem is that at least one good e-mail (I mean one I would have 
want to receive) had an invalid header line and was deleted by 
fetchmail. The mail was sent by hotmail  with a very long TO header (it 
is not rfc822 compliant) and further wrapped at some point. As a result, 
the header contained one line without colon and not starting with a 
blank character. The whole mail was simply deleted by fetchmail although 
it could have been delivered.

I believe it is not fetchmail's job to enforce rfc822 that way. It 
should be left to a program dedicated to that task or, at least, it 
should not be the default option. It would be a good idea to tag the 
mail with a specific header though. It would make it easier to filter 
out with a program such as procmail. I wouldn't divert the mail to the 
postmaster either. It is likely that a busy postmaster will throw it 
away without looking at it if he is not the obvious recipient.

The drawback is that if the mail is really junk, it will be downloaded 
and processed until it encounters the spam filter. It is a problem for a 
dialup link or even a DSL line with a low download limit. That's a 
reason to keep it an option the user can set.

Frederic

[fetchmail-devel] Re: [fetchmail-users] Incorrect header line and lost mails

[Rob M. <rob...@gm...>]

On 2/14/06, Matthias Andree <mat...@gm...> wrote:
>
> The bigger concern is that fetchmail shouldn't be emitting headers it
> knows are broken, so that the next hop (the MTA or MDA) does not get
> confused. I wonder if fetchmail should prefix the broken lines with
> X-Fetchmail-Escaped-Broken-Header: or something similar.

My 0.02$CURRENCY - when fetchmail finds a mail such as this it should
create a new mail, addressed to the postmaster address, with the
illegal email as a text/plain attachment.

If you're feeling kind, insert a blank line before the offending
illegal header and include that altered version as a message/rfc822
attachment (but still have the text/plain attachment)

There's no point in passing on the known broken email as is, and
simply "fixing" it may result in more problems.  At least if the
postmaster gets it as an attachment then they can decide how to handle
it.

Personally, I'm curious to know what mail client is producing such
borken emails, and what MTAs are passing them on.

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

[fetchmail-devel] Re: [fetchmail-users] Incorrect header line and lost mails

[Matthias A. <mat...@gm...>]

Frederic Marchal <fre...@wo...> writes:

> Well, I'm a C programmer and I may be able to help but I'm mainly a
> windows programmer (yes, I know, shame on me :-) ) and I'm not yet all
> too familiar with the Linux programming and especially with the patch
> mechanism and none at all with the diff tool.

The workflow is:

1. create a copy of all files you need to edit, for instance, 
   copy foo.c to foo.c.orig

   alternatively, copy the whole directory tree recursively

2. edit until you're happy with the code

3. test the edits (compile, install, run)

4. generate a unified (preferred) or context patch.  Not all diff
   utilities support unified output, but all are to support context
   format, as mandated by the relevant standard, IEEE Std 1003.1-2001.

   Plain ed-style patches (unfortunately still the default in the diff
   utility) are impractical so they are not usually accepted.

The diff tool is quite simple actually, if you created file copies:

diff -u OLDFILE NEWFILE >PATCHFILE (unified diff, preferred)

diff -c OLDFILE NEWFILE >PATCHFILE (context diff, alternative)

If you copied the whole directory before the edits:

diff -u -r BACKUP_DIR EDITED_DIR >PATCHFILE    (or -c instead of -u)

If you added new files to EDITED_DIR, you'd use -N but I don't think
we'll need this here.

diff, used either way, will produce a file (PATCHFILE) that you'd attach
to a message and mail to me (or Rob, or preferably the -devel@ list) or
upload to the BerliOS patch tracker.

If I have the choice, I prefer the -devel@ list. That's public, archived
and easiest for me since I don't need to fetch anything, but get it
pushed into my inbox.


The patch utility does the reverse and will usually just be used like
this, it derives the file names from the PATCHFILE.

patch <PATCHFILE

perhaps with -d (to change directory) or -p (to strip leading path
components from the paths shown in the patch file; used if patch cannot
find the files).

> Nevertheless, I have identified the core of the problem (its the call
> to readheaders in drive.c and the response to PS_REFUSED). I still
> have to check what happens if an invalid FROM or TO header is passed
> after readheaders but I think I can fix the program.

That would be great.

Basically they should not matter that much. fetchmail has some
deprecated code to guess from To:/Cc: or similar, but that is an
unreliable concept and doesn't deserve attention, From: shouldn't matter
as long as Return-Path: is given (and even then, From: shouldn't
matter). Missing or corrupted Return-Path: headers should be treated as
though the message had contained "Return-Path: <>".

> I'll also need some direction about the proper way of doing it for a
> good integration in the current source.

The bigger concern is that fetchmail shouldn't be emitting headers it
knows are broken, so that the next hop (the MTA or MDA) does not get
confused. I wonder if fetchmail should prefix the broken lines with
X-Fetchmail-Escaped-Broken-Header: or something similar.

-- 
Matthias Andree

[fetchmail-devel] ibiblio action notice from the `keeper' program

[<ke...@ib...>]

At about 2006-02-14 09:48:31 UCT, `keeper' moved the following files from
Incoming to system/mail/pop/fetchmail (remote mail retrieval and forwarding utility intended for use over on-demand TCP/IP links):

fetchmail-6.3.0.tar.bz2  Remote mail retrieval and forwarding utility intended for use over on-demand TCP/IP links.
fetchmail.lsm            

Thank you for your contribution of time, effort, and creativity.

This message was a form letter generated by keeper 1.55, but replying to
it will reach the human who told keeper what to do.  You got this note
because you're listed as a maintainer or author in the archive part involved.
If your package was actually uploaded by someone else, and you know who that
person is, please try to get that person to list him or herself in the LSM.
--
                        

[fetchmail-devel] patch 6.3.2.1 for fetchmail 6.3.2 fixes SEGFAULT with .netrc files

[Matthias A. <mat...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings,

Craig Leres identified a new bug in fetchmail 6.3.2 (that now tries to
erase the .netrc passwords before freeing its memory) that causes
fetchmail to crash right after reading the .netrc file.

Craig also provided a patch that I have accepted into the SVN
repository that will appear in fetchmail 6.3.3. I am including the patch
here (if your mailer is not GnuPG enabled, you need to manually edit all
lines that start with "- -" so that they start with "-" - i. e.  remove
the first minus and blank character on those lines).

I recommend to add this patch as an interim fix to all distribution
packages and on all sites that wish to use .netrc files.

The patch can also be downloaded:

http://download.berlios.de/fetchmail/patch-6.3.2.1-fix-netrc-SIGSEGV.diff

My GnuPG signature for the patch is available at:

http://download.berlios.de/fetchmail/patch-6.3.2.1-fix-netrc-SIGSEGV.diff.asc

Here is the patch:

.......................................................................
Craig Leres identified a problem that makes fetchmail 6.3.2 (only this
version) crash if the .netrc file does not contain a password for a
particular account.

This patch is mostly Craig Leres' work has been committed to the
SVN repository and should be applied to fetchmail 6.3.2 on all sites
that plan to use netrc files:

Index: netrc.c
===================================================================
- --- netrc.c	(Revision 4683)
+++ netrc.c	(Revision 4684)
@@ -314,8 +314,10 @@
 free_netrc(netrc_entry *a) {
     while(a) {
 	netrc_entry *n = a->next;
- -	memset(a->password, 0x55, strlen(a->password));
- -	xfree(a->password);
+	if (a->password != NULL) {
+		memset(a->password, 0x55, strlen(a->password));
+		free(a->password);
+	}
 	xfree(a->login);
 	xfree(a->host);
 	xfree(a);

Sorry for the inconvenience.  -- Matthias Andree, 2006-01-30
.......................................................................

Regards,

- -- 
Matthias Andree
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFD3dmkvmGDOQUufZURAv1+AKDYf5zB++Dyj6buzKS0Fz6W9B70bQCglnYI
F7gplc9LV+Ixh88mq0DSFNI=
=4UM8
-----END PGP SIGNATURE-----

[fetchmail-devel] [Patch #804] 6.3.2 dumps core if there is a .netrc entry with no password

[<no...@be...>]

Patch #804 has been updated. 

Project: fetchmail
Category: None
Status: Open
Submitted by: leres
Assigned to : none
Summary: 6.3.2 dumps core if there is a .netrc entry with no password

-------------------------------------------------------
For more info, visit:

http://developer.berlios.de/patch/?func=detailpatch&patch_id=804&group_id=1824

[fetchmail-devel] [Bug #6234] 6.3.2 dumps core if there is a .netrc entry with no password

[<no...@be...>]

Bug #6234, was updated on 2006-Jan-29 18:45
Here is a current snapshot of the bug.

Project: Community Fetchmail
Category: None
Status: Open
Resolution: None
Bug Group: None
Priority: 5
Submitted by: leres
Assigned to : none
Summary: 6.3.2 dumps core if there is a .netrc entry with no password

Details: If the users .netrc contains a line similar to this:

    machine lose login leres

fetchmail will dump core when free_netrc() attempts to memset() NULL.


For detailed info, follow this link:
http://developer.berlios.de/bugs/?func=detailbug&bug_id=6234&group_id=1824

[fetchmail-devel] fetchmail security announcement 2006-01 (CVE-2006-0321)

[Matthias A. <mat...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

fetchmail-SA-2006-01: crash when bouncing messages.

Topics:		#1 crash when bouncing a message
		#2 fetchmail 6.2.5.X end of life

Author:		Matthias Andree
Version:	1.0
Announced:	2006-01-22
Type:		free() with bogus pointer
Impact:		fetchmail crashes
Danger:		low
Credits:	Nathaniel W. Turner (bug report)
CVE Name:	CVE-2006-0321
URL:		http://fetchmail.berlios.de/fetchmail-SA-2006-01.txt
		http://bugs.debian.org/348747
Project URL:	http://fetchmail.berlios.de/

Affects:	fetchmail release >= 6.3.0
		fetchmail release <  6.3.2
		fetchmail release candidates 6.3.2-rc1, -rc2 and -rc3

Not affected:	fetchmail release candidate 6.3.2-rc4
		other versions not mentioned here or in the previous
		sections have not been checked

Corrected:	2006-01-19 fetchmail 6.3.2-rc4
		2006-01-22 fetchmail 6.3.2


0. Release history
==================

2006-01-19	internal review draft
2006-01-20	add CVE ID
2006-01-22	release 1.0


1. Background
=============

fetchmail is a software package to retrieve mail from remote POP2, POP3,
IMAP, ETRN or ODMR servers and forward it to local SMTP, LMTP servers or
message delivery agents.

fetchmail ships with a graphical, Python/Tkinter based configuration
utility named "fetchmailconf" to help the user create configuration (run
control) files for fetchmail.


2. Problem description and Impact
=================================

Fetchmail contains a bug that causes itself to crash when bouncing a
message to the originator or to the local postmaster. The crash happens
after the bounce message has been sent, when fetchmail tries to free the
dynamic array of failed addresses, and calls the free() function with an
invalid pointer.  This bug was introduced short before fetchmail 6.3.0
and is not present in the now discontinued 6.2.X series (see below).


3. Workaround
=============

None known at this time.


4. Solution
===========

Download and install fetchmail 6.3.2 or a newer stable release from
fetchmail's project site at
<http://developer.berlios.de/project/showfiles.php?group_id=1824>.


5. End of life announcement
===========================

The aged fetchmail 6.2.5.X branch is discontinued effective immediately.
No further releases from the 6.2.5.X branch will be made.

The new 6.3.X stable branch has been available since 2005-11-30
and will not change except for bugfixes, documentation and message
translations.


A. Copyright, License and Warranty
==================================

(C) Copyright 2006 by Matthias Andree, <mat...@gm...>.
Some rights reserved.

This work is licensed under the Creative Commons
Attribution-NonCommercial-NoDerivs German License. To view a copy of
this license, visit http://creativecommons.org/licenses/by-nc-nd/2.0/de/
or send a letter to Creative Commons; 559 Nathan Abbott Way;
Stanford, California 94305; USA.

THIS WORK IS PROVIDED FREE OF CHARGE AND WITHOUT ANY WARRANTIES.
Use the information herein at your own risk.

END OF fetchmail-SA-2006-01.txt
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFD04VrvmGDOQUufZURAuktAKD62llCNM2NlccXzI8NOqU9/sD1hgCeLfw0
fu8vMb8DiolqOUdQU6vVAP0=
=tfMu
-----END PGP SIGNATURE-----

[fetchmail-devel] fetchmail 6.3.2 stable release with security relevant bugfixes

[Matthias A. <mat...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings,

I am announcing the release of fetchmail 6.3.2.

This release fixes a denial of service bug/fetchmail crash after sending
a bounce, adds a Maillennium (Comcast) workaround and fixes other bugs.
(The security announcement will be mailed separately.)

This is a recommended upgrade for all users of any previous fetchmail versions.

The software is available from:
<https://developer.berlios.de/project/showfiles.php?group_id=1824&release_id=8784>

These are the relevant changes in 6.3.2 since 6.3.1,
unless otherwise noted, changes to this release were made by Matthias Andree.

# SECURITY FIX IN THIS RELEASE
* CVE-2006-0321: Fix segfault or bus error after bouncing a message.  This bug
  was introduced into 6.3.0 when removing alloca(); it caused fetchmail to free
  random memory.  Reported by Nathaniel W. Turner, Debian Bug#348747.
  See fetchmail-SA-2006-01.txt

# DEPRECATED FEATURES AND MAJOR INCOMPATIBLE CHANGE ADVANCE WARNINGS
* The --enable-fallback (fall back to MDA if MTA unavailable) may be removed
  from a future fetchmail release.

# INCOMPATIBLE CHANGES
* Automatically disable the POP3 TOP command if the greeting string contains
  "Maillennium POP3/PROXY server", which is used by comcast and known to
  truncate messages after 80 kByte. Fall back to RETR, and complain if we had
  used TOP otherwise (the warning is printed only once per server in daemon
  mode).  Suggested by Ed Wilts.
  *Note* that this means messages are marked read on these servers, which is a
  deviation from how 6.3.1 behaved, but we have no alternative, comcast haven't
  fixed this bug in years.  Preventing the loss of the remainder of the message
  justifies this incompatible fix.
* fetchmail, since 6.3.0, requires write permission to the directory holding the
  idfile. See the amendment in the 6.3.0 MAJOR INCOMPATIBLE CHANGES section
  below for details. The manual page was updated.

# CHANGES RELEVANT TO PACKAGERS
* The outdated BUGS document was removed from the distribution.
* Added fetchmail-SA-2006-01.txt to the distribution.

# BUG FIXES
* SMTP/LMTP cleanup to fix these two bugs:
  - switch back to SMTP after having tried LMTP hosts (multiple smtphost hosts)
  - switch back to LMTP after sending a bounce.
  The patch removes the global state variable that was the root of this problem.
  Patch by Sunil Shetye. (MA)
* Don't complain about fetchall keep in --configdump mode. Bug introduced in
  6.3.0.
* fetchmailconf.py: Fix novice help for Poll interval and fetchall.
  Reported by Justin Pryzby, Debian Bug #344978.
* Some verbose output disappeared in debug mode. Adding further -v options would
  alternate between verbose and debug mode. debug mode now comprises all verbose
  output, and adding more -v options does not switch back from debug to verbose
  mode.
* fetchmail.man: Fix accented characters in Héctor García's name. Merged from
  downstream debian/patches/01_man_page.dpatch.
* Add missing --help text for "--sslcertck" option.
* fetchmailconf.py: Accept --help and --version.
* fetchmail --version  now prints the copyright notice.
* don't complain about READ-ONLY IMAP folders in --fetchall --keep mode.
  Reported Alexander Zangerl, Debian Bug#348964.
* the RPM .spec file now generates a -debuginfo package on newer RPM versions.

Regards,

- -- 
Matthias Andree
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFD04UzvmGDOQUufZURApk+AJ46zXfGAd0jHsbxziZ7JQpPugjJKwCeM2xW
DSAxh7uWlnM7Teolv4wF+BE=
=hrZ0
-----END PGP SIGNATURE-----

[fetchmail-devel] New Polish PO file for `fetchmail'

[Translation P. R. <tra...@ir...>]

Hello, gentle maintainer. This is a message from the Translation
Project robot.

A revised PO file, for programs using the textual domain `fetchmail',
has been submitted by the team of translators taking care of the
Polish language.  This particular file, along with all other PO files
pertaining to the same textual domain, is available as:
>    http://www.iro.umontreal.ca/translation/maint/fetchmail/pl.po

The file should soon be made available in mirror sites as:

>    ftp://ftp.unex.es/pub/gnu-i18n/po/maint/fetchmail/pl.po
>    http://translation.sf.net/maint/fetchmail/pl.po

This file has already been sent to you separately on 2006-01-18,
as a MIME invoice unpacking the file `fetchmail-6.3.2-rc3.pl.po'.

The following HTML page should also be updated by tomorrow.

>    http://www.iro.umontreal.ca/translation/HTML/domain-fetchmail.html

Please consider including all PO files, as they stand, in the `po/'
subdirectory of your next release of programs using that textual domain,
whether it is official or pretest.  Whenever you have a distribution ready
which holds a newer PO Template, please send the URL of this distribution
to the address below.  The distribution could be a pretest or a snapshot,
it does not even have to compile.  This is to be used by translators,
when they need to get some translation context from your sources.

Within the Translation Project, each PO Template file should have different
version numbers, but since it is not OK to have two different distributions
using same version numbers, this is not a problem in practice.

Contact me if any question arises.  Thanks for your collaboration,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                mailto:tra...@ir...