Re: [fetchmail-devel] POP3 UIDL behavior

[Seth M. <se...@ro...>]

Matthias Andree wrote:

> We have the following structure (roughly):
> 
>     while (fgets(buf, POPBUFSIZE, tmpfp) != (char *)NULL) {
>     // ...
>         /* find proper list and save it */
>         for (ctl = hostlist; ctl; ctl = ctl->next) {
>             if (strcasecmp(host, ctl->server.queryname) == 0
>                     && strcasecmp(user, ctl->remotename) == 0) {
>                 save_str(&ctl->oldsaved, id, UID_SEEN);
>                 break;
>             }
>         }
>     }
 >
> So if the "break;" breaks out of anything but the for ctl=... list,
> something's seriously wrong. The "break;" applies to the for... loop for
> as long as I remember C (that's nearly two decades now), not to the
> outer while... loop. Anything else is a hardware problem,
> compiler/optimizer bug or something is corrupting the heap or stack or
> unrelated variables. Can you check that? The latter may be revealed by
> running under valgrind (OTOH, OpenSSL reads uninitialized memory to
> harvest entropy, so expect a gazillion of complaints unless you have an
> ignore file for OpenSSL).

I completely agree; something weird is going on.


> What compiler, version, compilers flags, and operating system are you
> using? What platform/hardware does this happen on?

Vanilla Debian "stable" (kernel 2.6.18) on amd64.

fetchmail:/usr/local/src/fetchmail-6.3.9# gcc -v
Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v 
--enable-languages=c,c++,fortran,objc,obj-c++,treelang --prefix=/usr 
--enable-shared --with-system-zlib --libexecdir=/usr/lib 
--without-included-gettext --enable-threads=posix --enable-nls 
--program-suffix=-4.1 --enable-__cxa_atexit --enable-clocale=gnu 
--enable-libstdcxx-debug --enable-mpfr --enable-checking=release 
x86_64-linux-gnu
Thread model: posix
gcc version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)


fetchmail:/usr/local/src/fetchmail-6.3.9-roller# make
make  all-recursive
make[1]: Entering directory `/usr/local/src/fetchmail-6.3.9-roller'
Making all in m4
make[2]: Entering directory `/usr/local/src/fetchmail-6.3.9-roller/m4'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/usr/local/src/fetchmail-6.3.9-roller/m4'
Making all in po
make[2]: Entering directory `/usr/local/src/fetchmail-6.3.9-roller/po'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/usr/local/src/fetchmail-6.3.9-roller/po'
make[2]: Entering directory `/usr/local/src/fetchmail-6.3.9-roller'
gcc -DHAVE_CONFIG_H -DLOCALEDIR=\"/usr/local/share/locale\" -I. 
-I./libesmtp   -g -O2 -I/usr/kerberos/include -MT uid.o -MD -MP -MF 
.deps/uid.Tpo -c -o uid.o uid.c
mv -f .deps/uid.Tpo .deps/uid.Po
gcc  -g -O2 -I/usr/kerberos/include  -L/usr/lib -o fetchmail socket.o 
getpass.o fetchmail.o env.o idle.o options.o daemon.o driver.o 
transact.o sink.o smtp.o uid.o mxget.o md5ify.o cram.o gssapi.o opie.o 
interface.o netrc.o unmime.o conf.o checkalias.o lock.o rcfile_l.o 
rcfile_y.o norm_charmap.o  pop3.o imap.o etrn.o odmr.o     libfm.a 
strlcpy.o strlcat.o  -lcrypt  -lresolv -lssl -lcrypto
make[2]: Leaving directory `/usr/local/src/fetchmail-6.3.9-roller'
make[1]: Leaving directory `/usr/local/src/fetchmail-6.3.9-roller'


> 
>> This is the specific fetchmailrc file that it's currently running with:
>>
>> set daemon 60
>> set no syslog
>> set logfile /var/lib/fetchmail/test/log
>> set idfile /var/lib/fetchmail/test/uidl
>> set no bouncemail
>> set spambounce
> 
> Unrelated, but please don't to the latter. Spambouncing hurts innocent
> bystanders.

I've made several changes to sink.c so it will only send bounce notices 
to the defined postmaster address. ;)


> I'm not convinced it's actually fixing bugs in fetchmail, rather than
> working around them in strange ways. I do not dispute that the patch is
> effective, but I'm loathe to apply the patch before I understand why
> exactly it's needed. My configuration looks similar to yours as far as
> UIDs are concerned, and I don't see problems here.
> 
> I'm inclined to believe in compiler bugs for now, but there may be a
> fetchmail bug in a different location that corrupts innocent storage and
> just happens to hit uid.c.
> 

When I have time (probably next week) I'll collect the debug output from 
before and after the patch to illustrate. The event that started me on 
this quest was fetchmail re-downloading the entire contents of a "keep" 
POP3 box every time the daemon started even though I had uidl enabled.

-- 
Seth Mattinen		se...@ro...
Roller Network LLC