fetchmail-devel — Development related discussion

Re: [fetchmail-devel] Re: r3948 - trunk

[Matthias A. <ma...@dt...>]

Graham Wilson <gr...@mk...> writes:

>> I hadn't heard contradictions last time I posted my progress indicator
>> WRT the warning message fixes I deemed critical, so I'm a bit surprised
>> by your comments now.
>
> Sorry for that. Your right, I should have posted something sooner. I
> just think that for larger changes like this, we should be working on
> branches that we can then re-integrate into the trunk.

OK, I will figure how to branch and merge in Subversion and to that next
time for "sweeping" changes that aren't self-contained in a single
patch-set. It's only that I am not particularly fond of branches as most
of my development efforts are still CVS-based, and CVS's branch handling
is known to be inferior to SVN.

-- 
Matthias Andree

[fetchmail-devel] Re: r3948 - trunk

[Graham W. <gr...@mk...>]

On Wed, Oct 20, 2004 at 12:58:27PM +0200, Matthias Andree wrote:
> Practical consequence: if you're using strict mail server settings,
> SpamAssassin or similar, all fetchmail warnings are either rejected or
> filtered out as spam, with the user not seeing the warning. Whoops.

Yes, I realize the importance of fetchmail correctly encoding the
message if it is in a non-ASCII character set: there is actually a
Debian bug (#277324) asking for support for this. I also appreciate you
working on it.

> I hadn't heard contradictions last time I posted my progress indicator
> WRT the warning message fixes I deemed critical, so I'm a bit surprised
> by your comments now.

Sorry for that. Your right, I should have posted something sooner. I
just think that for larger changes like this, we should be working on
branches that we can then re-integrate into the trunk.

-- 
gram

[fetchmail-devel] Re: r3950 - in trunk: . po

[Graham W. <gr...@mk...>]

On Wed, Oct 20, 2004 at 12:59:04PM +0200, Matthias Andree wrote:
> Graham Wilson <gr...@mk...> writes:
> > On Tue, Oct 19, 2004 at 06:05:19PM -0500, sv...@de... wrote:
> >> Log:
> >> Update.
> >
> > Update?
> 
> Yup. Trivial updates, such as listing gettext requirements, fixups in
> the translation files after the recent changes.

What I meant was that the log message was not very descriptive. All the
other log messages I've seen lately seemed to have been fine though. So,
sorry for the noise.

-- 
gram

Re: [fetchmail-devel] Re: r3950 - in trunk: . po

[Matthias A. <ma...@dt...>]

Graham Wilson <gr...@mk...> writes:

> On Tue, Oct 19, 2004 at 06:05:19PM -0500, sv...@de... wrote:
>> Modified:
>>    trunk/configure.ac
>>    trunk/po/de.po
>>    trunk/po/fetchmail.pot
>>    trunk/po/fr.po
>> Log:
>> Update.
>
> Update?

Yup. Trivial updates, such as listing gettext requirements, fixups in
the translation files after the recent changes.

-- 
Matthias Andree

Re: [fetchmail-devel] Re: r3948 - trunk

[Matthias A. <ma...@dt...>]

Graham Wilson <gr...@mk...> writes:

> On Tue, Oct 19, 2004 at 05:51:19PM -0500, sv...@de... wrote:
>> Added:
>>    trunk/rfc2047e.c
>> Modified:
>>    trunk/Makefile.am
>>    trunk/fetchmail.h
>> Log:
>> Add RFC-2047 encoder for internationalized mail warnings.
>
> Shouldn't we be adding this stuff on a branch? Or have we decided that
> the next release will definitely support correctly encoded warning
> messages? Even then, shouldn't the code be on a branch until we're sure
> it's good?

The issue at hand is that fetchmail warnings, if translated, are usually
in violation of RFC-2822, by stuffing national characters (8-bit data)
into headers and bodies unencoded and without markup. I've chosen the
trivial approach of just listing the charset of the body and specifying
it's 8-bit data (which may fail on ancient MTAs, but I don't care).

Practical consequence: if you're using strict mail server settings,
SpamAssassin or similar, all fetchmail warnings are either rejected or
filtered out as spam, with the user not seeing the warning. Whoops.

I hadn't heard contradictions last time I posted my progress indicator
WRT the warning message fixes I deemed critical, so I'm a bit surprised
by your comments now.

I may need to change the integration though, we should interpolate
variables before encoding and not just assume the interpolated stuff is
US-ASCII, as we're still doing.

-- 
Matthias Andree

[fetchmail-devel] Re: r3948 - trunk

[Graham W. <gr...@mk...>]

On Tue, Oct 19, 2004 at 05:51:19PM -0500, sv...@de... wrote:
> Added:
>    trunk/rfc2047e.c
> Modified:
>    trunk/Makefile.am
>    trunk/fetchmail.h
> Log:
> Add RFC-2047 encoder for internationalized mail warnings.

Shouldn't we be adding this stuff on a branch? Or have we decided that
the next release will definitely support correctly encoded warning
messages? Even then, shouldn't the code be on a branch until we're sure
it's good?

-- 
gram

[fetchmail-devel] Re: r3950 - in trunk: . po

[Graham W. <gr...@mk...>]

On Tue, Oct 19, 2004 at 06:05:19PM -0500, sv...@de... wrote:
> Modified:
>    trunk/configure.ac
>    trunk/po/de.po
>    trunk/po/fetchmail.pot
>    trunk/po/fr.po
> Log:
> Update.

Update?

-- 
gram

[fetchmail-devel] warning message charset fix: progress indicator 2/3 (was: Re: warning message charset fix: progress indicator 1/3.)

[Matthias A. <ma...@dt...>]

Matthias Andree <ma...@dt...> writes:

> Remaining issues (I don't have time to work on this now, if someone wants 
> to take a stab at this, feel free, just mention you're working on it):
>
> 2/3. write (or take and integrate) RFC-2047 encoder/folder for the
>      headers.

I have written a new RFC-2047 encoder (please test this one hard and
audit as though you would not trust me, it may be b0rked) & hooked up.

We don't want this to become our brown paper bag issue of the
first-release-after-a-year release...

> 3/3. figure if nl_langinfo(CODESET) always specifies IANA-conformant
>      character sets or if we need to map these and possibly quench the
>      translation for those that aren't OK'd by IANA.
>      Reference: http://www.iana.org/assignments/character-sets

Still open.

-- 
Matthias Andree

Re: [fetchmail-devel] Rodolfo's hang (from fetchmail-friends)

[Matthias A. <ma...@dt...>]

Brian Candler <B.C...@po...> writes:

> In the event of any dispute about whether it's fetchmail's fault or the
> server's, then I'd say the best thing to do is to get a
>     tcpdump -i eth0 -n -s1500 -X tcp port 110
> output, and see exactly what is going on.

Right, I'd forgotten about tcpdump.

> But if you're going to read the mail line-by-line then you can just as
> easily check for the CR LF . CR LF termination sequence!
>
> I can't imagine that any other major POP3 client does anything other than
> that, and I don't see why fetchmail should.

I don't intend to change the POP3 client before I know it is buggy.

I may need to change the IMAP client though, it goofs up reading
literals.

-- 
Matthias Andree

Encrypted mail welcome: my GnuPG key ID is 0x052E7D95 (PGP/MIME preferred)

Re: [fetchmail-devel] Rodolfo's hang (from fetchmail-friends)

[Brian C. <B.C...@po...>]

On Fri, Oct 15, 2004 at 02:35:53PM +0200, Matthias Andree wrote:
> I'd asked Rodolfo if it was POP3 that hung and if so, I said his
> upstream server was broken. We'll see what he can figure next.

I thought I saw something about downloads hanging because they contained a
null (\0) character. That would be different.

In the event of any dispute about whether it's fetchmail's fault or the
server's, then I'd say the best thing to do is to get a
    tcpdump -i eth0 -n -s1500 -X tcp port 110
output, and see exactly what is going on.

> > qmail-pop3d gives a wrong size because it doesn't count end-of-line as two
> > bytes; however using the size would not work anyway even if it did, because
> > lines which start with the termination character use byte-stuffing but don't
> > count it twice in the size (see RFC1939, section 11)
> 
> That could be compensated for as the message is read, because it is read
> line-wise.

But if you're going to read the mail line-by-line then you can just as
easily check for the CR LF . CR LF termination sequence!

I can't imagine that any other major POP3 client does anything other than
that, and I don't see why fetchmail should.

Regards,

Brian.

Re: [fetchmail-devel] Rodolfo's hang (from fetchmail-friends)

[Matthias A. <ma...@dt...>]

Brian Candler <B.C...@po...> writes:

> On Fri, Oct 15, 2004 at 11:20:28AM +0200, Matthias Andree wrote:
>> re-reading Rodolfo's fetchmail hang, it appears as though his POP3
>> server is at fault.
>> 
>> I've looked into the reliability of the SIZE information given by POP3
>> servers, and the first one I checked, qmail-pop3d, failed horribly by
>> giving a size that is too short - we cannot use SIZE in POP3 to
>> determine how many bytes to read lest we risk reading only part of the
>> mail.
>
> ... and therefore losing command/response synchronisation.

Yes, which would be much worse.

> The end of a message in POP3 is determined by the sequence "CR LF . CR LF"
> and only by that. You are not permitted to perform "LIST n", look at the
> size returned, enter "RETR n", read exactly that number of bytes, and expect
> to be back in sync with the server at that point.

I don't expect such, but qmail hosing the SIZE figures is dangerous if
the client allocates a single linear chunk of memory on that
basis. qmail is full of warts, DJB doesn't care, but anyways, qmail
AFAIK gets the message termination right.

I'd asked Rodolfo if it was POP3 that hung and if so, I said his
upstream server was broken. We'll see what he can figure next.

> qmail-pop3d gives a wrong size because it doesn't count end-of-line as two
> bytes; however using the size would not work anyway even if it did, because
> lines which start with the termination character use byte-stuffing but don't
> count it twice in the size (see RFC1939, section 11)

That could be compensated for as the message is read, because it is read
line-wise.

-- 
Matthias Andree

Encrypted mail welcome: my GnuPG key ID is 0x052E7D95 (PGP/MIME preferred)

Re: [fetchmail-devel] Rodolfo's hang (from fetchmail-friends)

[Brian C. <B.C...@po...>]

On Fri, Oct 15, 2004 at 11:20:28AM +0200, Matthias Andree wrote:
> re-reading Rodolfo's fetchmail hang, it appears as though his POP3
> server is at fault.
> 
> I've looked into the reliability of the SIZE information given by POP3
> servers, and the first one I checked, qmail-pop3d, failed horribly by
> giving a size that is too short - we cannot use SIZE in POP3 to
> determine how many bytes to read lest we risk reading only part of the
> mail.

... and therefore losing command/response synchronisation.

The end of a message in POP3 is determined by the sequence "CR LF . CR LF"
and only by that. You are not permitted to perform "LIST n", look at the
size returned, enter "RETR n", read exactly that number of bytes, and expect
to be back in sync with the server at that point.

qmail-pop3d gives a wrong size because it doesn't count end-of-line as two
bytes; however using the size would not work anyway even if it did, because
lines which start with the termination character use byte-stuffing but don't
count it twice in the size (see RFC1939, section 11)

line on server        abcd\n
RFC1939 size          6 (a b c d \r \n)
Actual octets sent    6 (a b c d \r \n)

line on server        .abc\n
RFC1939 size          6 (. a b c \r \n)
Actual octets sent    7 (. . a b c \r \n)

Regards,

Brian.

[fetchmail-devel] Rodolfo's hang (from fetchmail-friends)

[Matthias A. <ma...@dt...>]

Hi,

re-reading Rodolfo's fetchmail hang, it appears as though his POP3
server is at fault.

I've looked into the reliability of the SIZE information given by POP3
servers, and the first one I checked, qmail-pop3d, failed horribly by
giving a size that is too short - we cannot use SIZE in POP3 to
determine how many bytes to read lest we risk reading only part of the
mail.

Bye,

-- 
Matthias Andree

Encrypted mail welcome: my GnuPG key ID is 0x052E7D95 (PGP/MIME preferred)

[fetchmail-devel] [BUG] IMAP client doesn't read literals properly

[Matthias A. <ma...@dt...>]

While debugging Rodolfo Borges' "no '\n' at the last line" bug-report on
fetchmail-friends, I found that the IMAP reader doesn't properly detect
the end of the mail; sample output:

fetchmail: IMAP> A0009 FETCH 372 BODY.PEEK[TEXT]
fetchmail: IMAP< * 372 FETCH (BODY[TEXT] {4}
fetchmail: IMAP< test)
fetchmail: IMAP< A0009 OK Fetch completed.

and the ) that is part of the untagged FETCH reply from the 2nd line
becomes part of the message.

The curly brace notation, {N} means that there will be N octets after
the end of the line - and the 4 is pretty clearly a 4, not a 5 -- see
RFC-3501 (IMAP v4) for details.

I presume one of the many workarounds broke the protocol for working
servers. Haven't had time to R&D a proper fix yet, apparently, fetchmail
ignores the length.

-- 
Matthias Andree

Encrypted mail welcome: my GnuPG key ID is 0x052E7D95 (PGP/MIME preferred)

[fetchmail-devel] warning message charset fix: progress indicator 1/3.

[Matthias A. <ma...@dt...>]

Hi,

I have taught fetchmail to specify the character encoding for the body
of (localized?) charset issues, please test. It works for me with
ISO-8859-15 and UTF-8 on Linux, I'll need to test on FreeBSD 5 which
names character sets without dash between ISO and 8859 for instance.

Remaining issues (I don't have time to work on this now, if someone wants 
to take a stab at this, feel free, just mention you're working on it):

2/3. write (or take and integrate) RFC-2047 encoder/folder for the
     headers.

3/3. figure if nl_langinfo(CODESET) always specifies IANA-conformant
     character sets or if we need to map these and possibly quench the
     translation for those that aren't OK'd by IANA.
     Reference: http://www.iana.org/assignments/character-sets

-- 
Matthias Andree

Encrypted mail welcome: my GnuPG key ID is 0x052E7D95 (PGP/MIME preferred)

[fetchmail-devel] [BUG] fetchmail hangs during pop3 pull after a mail with a null char

[David G. <da...@dg...>]

First : fetchmail is great - thanks  :) 

I sent this to fetchmail-friends a while back and it was suggested that I send it to the devlist.
I joined and lurked - and finally got round to sending this in...


It's been working for years with these occasional hangs that have been
fixed by popping the bad messages and manually filing them. I finally
had a bad message arrive when I was in a position to debug!

Summary : fetchmail hangs during pop3 pull after a mail with a null char.

The mail with a null char is pulled OK but then rejected by local
Cyrus lmtp and bounced to postmaster via exim4.20
The next pop3 pull then fails.
I've made an effort to trace and I think the hang occurs due to a
double call to SMTP_ok which is empty the second time. I am pretty
sure the second call originates at sink.c line 1433.

in the config expunge 1 fixes the problem (which makes sense)

general config is pop3->lmtp->local Cyrus IMAP

So I went through the FAQ G3 points:
1. OS: Linux RedHat 7.3 kernel 2.6.6
2. gcc version 2.96 20000731 (Red Hat Linux 7.3 2.96-110)
3. below
4. forwarding to lmtp listener on cyrus 2.2.3 (bounce goes to SMTP
exim 4.2)
5. -v -v -f /etc/fetchmailrc
6. at the end

nb: SMTP_ok loop start comments are my trace.

Aug 17 17:33:01 willow fetchmail[13648]: 6.2.5 querying pop3.ukfsn.org
(protocol POP3) at Tue 17 Aug 2004 05:33:01 PM BST: poll started
Aug 17 17:33:01 willow fetchmail[13648]: POP3< +OK
<aa5...@po...>
Aug 17 17:33:01 willow fetchmail[13648]: POP3> CAPA^M
Aug 17 17:33:01 willow fetchmail[13648]: POP3< +OK Capability list follows
Aug 17 17:33:01 willow fetchmail[13648]: POP3< PIPELINING
Aug 17 17:33:01 willow fetchmail[13648]: POP3< TOP
Aug 17 17:33:01 willow fetchmail[13648]: POP3< USER
Aug 17 17:33:01 willow fetchmail[13648]: POP3< UIDL
Aug 17 17:33:01 willow fetchmail[13648]: POP3< STLS
Aug 17 17:33:01 willow fetchmail[13648]: POP3< .
Aug 17 17:33:01 willow fetchmail[13648]: POP3> USER dgreaves^M
Aug 17 17:33:01 willow fetchmail[13648]: POP3< +OK Tell me your password.
Aug 17 17:33:01 willow fetchmail[13648]: POP3> PASS *^M
Aug 17 17:33:02 willow fetchmail[13648]: POP3< +OK Welcome aboard! You
have 55 messages.
Aug 17 17:33:05 willow fetchmail[13648]: POP3> STAT
Aug 17 17:33:05 willow fetchmail[13648]: POP3< +OK 55 429607
Aug 17 17:33:05 willow fetchmail[13648]: 55 messages for dgreaves at
pop3.ukfsn.org (429607 octets).



fetchmailrc:
set syslog
set postmaster "da...@dg..."
set nobouncemail
set properties ""
#set daemon 180
set idfile /var/run/fetchmail.ids

# The ukfsn accounts
poll pop3.ukfsn.org with proto POP3 tracepolls
~    user 'dgreaves' there with password 'xxxxxxx' is
da...@dg... here options fetchall lmtp smtp
/var/imap/socket/lmtp expunge 5
~    antispam 571 550 501 554

<more user accounts removed>

here is output from
~  ./fetchmail -v -v -f /etc/fetchmailrc

Aug 17 17:43:33 willow fetchmail[13675]: 6.2.5 querying pop3.ukfsn.org
(protocol POP3) at Tue 17 Aug 2004 05:43:33 PM BST: poll started
Aug 17 17:43:33 willow fetchmail[13675]: POP3< +OK
<40b...@po...>
Aug 17 17:43:33 willow fetchmail[13675]: POP3> CAPA^M
Aug 17 17:43:33 willow fetchmail[13675]: POP3< +OK Capability list follows
Aug 17 17:43:34 willow fetchmail[13675]: POP3< PIPELINING
Aug 17 17:43:34 willow fetchmail[13675]: POP3< TOP
Aug 17 17:43:34 willow fetchmail[13675]: POP3< USER
Aug 17 17:43:34 willow fetchmail[13675]: POP3< UIDL
Aug 17 17:43:34 willow fetchmail[13675]: POP3< STLS
Aug 17 17:43:34 willow fetchmail[13675]: POP3< .
Aug 17 17:43:34 willow fetchmail[13675]: POP3> USER dgreaves^M
Aug 17 17:43:34 willow fetchmail[13675]: POP3< +OK Tell me your password.
Aug 17 17:43:34 willow fetchmail[13675]: POP3> PASS *^M
Aug 17 17:43:34 willow fetchmail[13675]: POP3< +OK Welcome aboard! You
have 33 messages.
Aug 17 17:43:37 willow fetchmail[13675]: selecting or re-polling
default folder
Aug 17 17:43:37 willow fetchmail[13675]: POP3> STAT
Aug 17 17:43:37 willow fetchmail[13675]: POP3< +OK 33 186252
Aug 17 17:43:37 willow fetchmail[13675]: 33 messages for dgreaves at
pop3.ukfsn.org (186252 octets).
Aug 17 17:43:37 willow fetchmail[13675]: POP3> LIST 1
#**********************************************Aug 17 17:43:37 willow
fetchmail[13675]: POP3< +OK 1 9060
Aug 17 17:43:37 willow fetchmail[13675]: POP3> RETR 1
Aug 17 17:43:37 willow fetchmail[13675]: POP3< +OK Message follows
Aug 17 17:43:37 willow fetchmail[13675]: reading message
dgr...@po...:1 of 33 (9060 octets)
Aug 17 17:43:37 willow fetchmail[13675]: About to rewrite Return-Path:
<linux-kernel-owner+lkml=40d...@vg...>^M
Rewritten version is Return-Path:
<linux-kernel-owner+lkml=40d...@vg...>^M
Aug 17 17:43:38 willow fetchmail[13675]: About to rewrite From:
"O.Sezer" <se...@tt...>^M Rewritten version is From:
"O.Sezer" <se...@tt...>^M
Aug 17 17:43:38 willow fetchmail[13675]: About to rewrite To:
lin...@vg...^M Rewritten version is To:
lin...@vg...^M
Aug 17 17:43:38 willow fetchmail[13675]: About to rewrite Cc:
mar...@cy...^M Rewritten version is Cc:
mar...@cy...^M
Aug 17 17:43:38 willow fetchmail[13675]: About to rewrite Sender:
lin...@vg...^M Rewritten version is Sender:
lin...@vg...^M
Aug 17 17:43:38 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:38 willow fetchmail[13675]: LMTP< 220 willow LMTP Cyrus
v2.2.3 ready
Aug 17 17:43:38 willow fetchmail[13675]: LMTP> LHLO localhost
Aug 17 17:43:38 willow fetchmail[13675]: SMTP< 250-willow
Aug 17 17:43:38 willow fetchmail[13675]: SMTP< 250-8BITMIME
Aug 17 17:43:38 willow fetchmail[13675]: SMTP< 250-ENHANCEDSTATUSCODES
Aug 17 17:43:38 willow fetchmail[13675]: SMTP< 250-PIPELINING
Aug 17 17:43:38 willow fetchmail[13675]: SMTP< 250-SIZE
Aug 17 17:43:38 willow fetchmail[13675]: SMTP< 250-AUTH EXTERNAL
Aug 17 17:43:38 willow fetchmail[13675]: SMTP< 250 IGNOREQUOTA
Aug 17 17:43:38 willow fetchmail[13675]: forwarding to
/var/imap/socket/lmtp
Aug 17 17:43:38 willow fetchmail[13675]: LMTP> MAIL
FROM:<linux-kernel-owner+lkml=40d...@vg...>
SIZE=9060
Aug 17 17:43:38 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:38 willow fetchmail[13675]: LMTP< 250 2.1.0 ok
Aug 17 17:43:38 willow fetchmail[13675]: LMTP> RCPT
TO:<da...@dg...>
Aug 17 17:43:38 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:38 willow fetchmail[13675]: LMTP< 250 2.1.5 ok
Aug 17 17:43:38 willow fetchmail[13675]: LMTP> DATA
Aug 17 17:43:38 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:38 willow fetchmail[13675]: LMTP< 354 go ahead
********************************************************************************************************************************************************************Aug
17 17:43:38 willow fetchmail[13675]: message dgr...@po...:1
was not the expected length (9317 actual != 9060 expected)
Aug 17 17:43:38 willow fetchmail[13675]: LMTP>. (EOM)
Aug 17 17:43:38 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:38 willow fetchmail[13675]: LMTP< 250 2.1.5 Ok
Aug 17 17:43:38 willow fetchmail[13675]:  flushed
Aug 17 17:43:38 willow fetchmail[13675]: POP3> DELE 1^M
Aug 17 17:43:38 willow fetchmail[13675]: POP3< +OK Done.
Aug 17 17:43:38 willow fetchmail[13675]: POP3> LIST 2
Aug 17 17:43:38 willow fetchmail[13675]: POP3< +OK 2 5098
Aug 17 17:43:38 willow fetchmail[13675]: POP3> RETR 2
Aug 17 17:43:39 willow fetchmail[13675]: POP3< +OK Message follows
Aug 17 17:43:39 willow fetchmail[13675]: reading message
dgr...@po...:2 of 33 (5098 octets)
Aug 17 17:43:39 willow fetchmail[13675]: About to rewrite Return-Path:
<linux-kernel-owner+lkml=40d...@vg...>^M
Rewritten version is Return-Path:
<linux-kernel-owner+lkml=40d...@vg...>^M
#****************************************************************Aug
17 17:43:39 willow fetchmail[13675]: About to rewrite From: Christoph
Hellwig <hc...@in...>^M Rewritten version is From: Christoph
Hellwig <hc...@in...>^M
Aug 17 17:43:39 willow fetchmail[13675]: About to rewrite To: Markus
Lidel <Mar...@sh...>^M Rewritten version is To:
Markus Lidel <Mar...@sh...>^M
Aug 17 17:43:39 willow fetchmail[13675]: About to rewrite Cc:
Christoph Hellwig <hc...@in...>,^M ^IWarren Togami
<wt...@re...>, lin...@vg...^M Rewritten version
is Cc: Christoph Hellwig <hc...@in...>,^M ^IWarren Togami
<wt...@re...>, lin...@vg...^M
Aug 17 17:43:39 willow fetchmail[13675]: About to rewrite Sender:
lin...@vg...^M Rewritten version is Sender:
lin...@vg...^M
Aug 17 17:43:39 willow fetchmail[13675]: forwarding to
/var/imap/socket/lmtp
Aug 17 17:43:39 willow fetchmail[13675]: LMTP> MAIL
FROM:<linux-kernel-owner+lkml=40d...@vg...>
SIZE=5098
Aug 17 17:43:39 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:39 willow fetchmail[13675]: LMTP< 250 2.1.0 ok
Aug 17 17:43:39 willow fetchmail[13675]: LMTP> RCPT
TO:<da...@dg...>
Aug 17 17:43:39 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:39 willow fetchmail[13675]: LMTP< 250 2.1.5 ok
Aug 17 17:43:39 willow fetchmail[13675]: LMTP> DATA
Aug 17 17:43:39 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:39 willow fetchmail[13675]: LMTP< 354 go ahead
Aug 17 17:43:39 willow fetchmail[13675]: message
dgr...@po...:2 was not the expected length (5209 actual !=
5098 expected)
Aug 17 17:43:39 willow fetchmail[13675]: LMTP>. (EOM)
Aug 17 17:43:39 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:39 willow fetchmail[13675]: LMTP< 250 2.1.5 Ok
Aug 17 17:43:39 willow fetchmail[13675]:  flushed
Aug 17 17:43:39 willow fetchmail[13675]: POP3> DELE 2^M
Aug 17 17:43:39 willow fetchmail[13675]: POP3< +OK Done.
Aug 17 17:43:39 willow fetchmail[13675]: POP3> LIST 3
Aug 17 17:43:39 willow fetchmail[13675]: POP3< +OK 3 2147
Aug 17 17:43:39 willow fetchmail[13675]: POP3> RETR 3
Aug 17 17:43:40 willow fetchmail[13675]: POP3< +OK Message follows
Aug 17 17:43:40 willow fetchmail[13675]: reading message
dgr...@po...:3 of 33 (2147 octets)
Aug 17 17:43:40 willow fetchmail[13675]: About to rewrite Return-Path:
<reiserfs-list-return-20355-david=dgr...@na...>^M
Rewritten version is Return-Path:
<reiserfs-list-return-20355-david=dgr...@na...>^M
#********#****************************Aug 17 17:43:40 willow
fetchmail[13675]: About to rewrite From: elliott
<aur...@se...>^M Rewritten version is From: elliott
<aur...@se...>^M
Aug 17 17:43:40 willow fetchmail[13675]: About to rewrite To:
rei...@na...^M Rewritten version is To:
rei...@na...^M
Aug 17 17:43:40 willow fetchmail[13675]: forwarding to
/var/imap/socket/lmtp
Aug 17 17:43:40 willow fetchmail[13675]: LMTP> MAIL
FROM:<reiserfs-list-return-20355-david=dgr...@na...>
BODY=8BITMIME SIZE=2147
Aug 17 17:43:40 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:40 willow fetchmail[13675]: LMTP< 250 2.1.0 ok
Aug 17 17:43:40 willow fetchmail[13675]: LMTP> RCPT
TO:<da...@dg...>
Aug 17 17:43:40 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:40 willow fetchmail[13675]: LMTP< 250 2.1.5 ok
Aug 17 17:43:40 willow fetchmail[13675]: LMTP> DATA
Aug 17 17:43:40 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:40 willow fetchmail[13675]: LMTP< 354 go ahead
Aug 17 17:43:40 willow fetchmail[13675]: message
dgr...@po...:3 was not the expected length (2194 actual !=
2147 expected)
Aug 17 17:43:40 willow fetchmail[13675]: LMTP>. (EOM)
Aug 17 17:43:40 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:40 willow fetchmail[13675]: LMTP< 554 5.6.0 Message
contains NUL characters
Aug 17 17:43:40 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:40 willow fetchmail[13675]: SMTP< 220 willow ESMTP Exim
4.20 Tue, 17 Aug 2004 17:43:40 +0100
Aug 17 17:43:40 willow fetchmail[13675]: SMTP> HELO localhost
Aug 17 17:43:40 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:40 willow fetchmail[13675]: SMTP< 250 willow Hello
[AdFSF0x5m0a0vDWoZf1oPXMI8ichzbi7] at localhost.localdomain [127.0.0.1]
Aug 17 17:43:40 willow fetchmail[13675]: SMTP> MAIL
FROM:<FET...@wi...>
Aug 17 17:43:40 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:40 willow fetchmail[13675]: SMTP< 250 OK
Aug 17 17:43:40 willow fetchmail[13675]: SMTP> RCPT
TO:<da...@dg...>
Aug 17 17:43:40 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:40 willow fetchmail[13675]: SMTP< 250 Accepted
Aug 17 17:43:40 willow fetchmail[13675]: SMTP> DATA
Aug 17 17:43:40 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:40 willow fetchmail[13675]: SMTP< 354 Enter message,
ending with "." on a line by itself
Aug 17 17:43:40 willow fetchmail[13675]: SMTP: (bounce-message body)
Aug 17 17:43:40 willow fetchmail[13675]: SMTP>. (EOM)
Aug 17 17:43:40 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:40 willow fetchmail[13675]: SMTP< 250 OK id=1Bx73k-0003Ya-FK
Aug 17 17:43:40 willow fetchmail[13675]: SMTP> QUIT
Aug 17 17:43:40 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:40 willow fetchmail[13675]: SMTP< 221 willow closing
connection
Aug 17 17:43:40 willow fetchmail[13675]:  flushed
Aug 17 17:43:40 willow fetchmail[13675]: POP3> DELE 3^M
Aug 17 17:43:40 willow fetchmail[13675]: POP3< +OK Done.
Aug 17 17:43:40 willow fetchmail[13675]: POP3> LIST 4
Aug 17 17:43:40 willow fetchmail[13675]: POP3< +OK 4 4406
Aug 17 17:43:40 willow fetchmail[13675]: POP3> RETR 4
Aug 17 17:43:40 willow fetchmail[13675]: POP3< +OK Message follows
Aug 17 17:43:40 willow fetchmail[13675]: reading message
dgr...@po...:4 of 33 (4406 octets)
Aug 17 17:43:41 willow fetchmail[13675]: About to rewrite Return-Path:
<net...@os...>^M Rewritten version is Return-Path:
<net...@os...>^M
Aug 17 17:43:41 willow fetchmail[13675]: About to rewrite From:
Wensong Zhang <we...@li...>^M Rewritten version is From:
Wensong Zhang <we...@li...>^M
Aug 17 17:43:41 willow fetchmail[13675]: About to rewrite To:
ne...@os...^M Rewritten version is To: ne...@os...^M
Aug 17 17:43:41 willow fetchmail[13675]: About to rewrite Cc: Julian
Anastasov <ja...@ss...>^M Rewritten version is Cc: Julian Anastasov
<ja...@ss...>^M
Aug 17 17:43:41 willow fetchmail[13675]: About to rewrite Sender:
net...@os...^M Rewritten version is Sender:
net...@os...^M
Aug 17 17:43:41 willow fetchmail[13675]: forwarding to
/var/imap/socket/lmtp
Aug 17 17:43:41 willow fetchmail[13675]: SMTP> MAIL
FROM:<net...@os...> SIZE=4406
Aug 17 17:43:41 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:41 willow fetchmail[13675]: SMTP< 250 2.1.0 ok
Aug 17 17:43:41 willow fetchmail[13675]: SMTP> RCPT
TO:<da...@dg...>
Aug 17 17:43:41 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:41 willow fetchmail[13675]: SMTP< 250 2.1.5 ok
Aug 17 17:43:41 willow fetchmail[13675]: SMTP> DATA
Aug 17 17:43:41 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:41 willow fetchmail[13675]: SMTP< 354 go ahead
*******************************************************Aug 17 17:43:41
willow fetchmail[13675]: message dgr...@po...:4 was not the
expected length (4533 actual != 4406 expected)
Aug 17 17:43:41 willow fetchmail[13675]: SMTP>. (EOM)
Aug 17 17:43:41 willow fetchmail[13675]: SMTP_ok loop start
Aug 17 17:43:41 willow fetchmail[13675]: SMTP< 250 2.1.5 Ok

## 5 minute hang

Aug 17 17:48:41 willow fetchmail[13675]: smtp listener protocol error 2
Aug 17 17:48:41 willow fetchmail[13675]:  not flushed
Aug 17 17:48:41 willow fetchmail[13675]: POP3> LIST 5
Aug 17 17:48:41 willow fetchmail[13675]: POP3< -ERR Client has been
idle for too long.
Aug 17 17:48:41 willow fetchmail[13675]: Client has been idle for too
long.
Aug 17 17:48:41 willow fetchmail[13675]: POP3> QUIT^M
Aug 17 17:48:41 willow fetchmail[13675]: client/server protocol error
while fetching from pop3.ukfsn.org
Aug 17 17:48:41 willow fetchmail[13675]: 6.2.5 querying pop3.ukfsn.org
(protocol POP3) at Tue 17 Aug 2004 05:48:41 PM BST: poll completed
Aug 17 17:48:41 willow fetchmail[13675]: Query status=4 (PROTOCOL)

# ./fetchmail -V -v -v -f /etc/fetchmailrc
This is fetchmail release 6.2.5+NLS
Fallback MDA: (none)
Linux willow 2.6.6 #1 Wed Jun 2 12:15:21 BST 2004 i586 unknown
Taking options from command line and /etc/fetchmailrc
Idfile is /var/run/fetchmail.ids
Progress messages will be logged via syslog
Fetchmail will show progress dots even in logfiles.
Fetchmail will forward misaddressed multidrop messages to
da...@dg....
Fetchmail will direct error mail to the postmaster.
Options for retrieving from dgr...@po...:
~  True name of server is pop3.ukfsn.org.
~  This host will be queried when no host is specified.
~  Password = "xxxxxx".
~  Protocol is POP3 (using default port).
~  All available authentication methods will be tried.
~  Server nonresponse timeout is 300 seconds (default).
~  Default mailbox selected.
~  All messages will be retrieved (--all on).
~  Fetched messages will not be kept on the server (--keep off).
~  Old messages will not be flushed before message retrieval (--flush
off).
~  Rewrite of server-local addresses is enabled (--norewrite off).
~  Carriage-return stripping is disabled (stripcr off).
~  Carriage-return forcing is disabled (forcecr off).
~  Interpretation of Content-Transfer-Encoding is enabled (pass8bits off).
~  MIME decoding is disabled (mimedecode off).
~  Idle after poll is disabled (idle off).
~  Nonempty Status lines will be kept (dropstatus off)
~  Delivered-To lines will be kept (dropdelivered off)
~  No received-message limit (--fetchlimit 0).
~  Fetch message size limit is 100 (--fetchsizelimit 100).
~  Do binary search of UIDs during 9 out of 10 polls (--fastuidl 10).
~  No SMTP message batch limit (--batchlimit 0).
~  Deletion interval between expunges forced to 5 (--expunge 5).
~  Messages will be LMTP-forwarded to: /var/imap/socket/lmtp
~  Recognized listener spam block responses are: 571 550 501 554
~  No pre-connection command.
~  No post-connection command.
~  Single-drop mode: 1 local name(s) recognized.
~        da...@dg...
~  No interface requirement specified.
~  No monitor interface specified.
~  No plugin command specified.
~  No plugout command specified.
~  1 UIDs saved.
~        0bcc1e7633bb91ec04fbf4e1505b377d
~  Poll trace information will be added to the Received header.

other account info removed

David Greaves

Re: [fetchmail-devel] Not Yet Reviewed Queue is Empty!

[Matthias A. <ma...@dt...>]

Rob Funk <rf...@fu...> writes:

> There's still work to be done for building a release, converting Eric's 
> scripts over to the non-ESR Berlios environment.

The tarball currently generated by "make dist" appears to be
self-contained, as "make distcheck" will confirm. Make distcheck cannot,
however, check for documentation we want in that isn't listed in a
Makefile.am.

> Since Eric has regained control of the old mailing lists, it would be good 
> to coordinate with him to get those moved over.

OK. Can we also copy over the archives with little effort (Mailman on
lists.ccil.org stores the while archive in mbox format, we'd need this
rather than downloading one per each month as the web interface of
pipermail offers.)

-- 
Matthias Andree

Encrypted mail welcome: my GnuPG key ID is 0x052E7D95 (PGP/MIME preferred)

Re: [fetchmail-devel] Re: Not Yet Reviewed Queue is Empty!

[Matthias A. <ma...@dt...>]

Graham Wilson <gr...@mk...> writes:

> Once we've figured out how were going to build release tarballs, I'd
> like to look at the differences between the 6.2.5 tarball and the 6.2.6
> tarball, and make sure we are aware of all of the changes.
>
> Hmm... come to think of it, should this release be 6.3.0? This is the
> first release in a while, and the maintainers have changed. Seems to me
> to warrant more than a patch version number increased.

I'm reading the numer as major.minor.patchlevel.

The maintainer change doesn't warrant the minor bump, but the automake
switchover does, so I'd also vote for 6.3.0.

I'd suggest we bump minor if we add features or make large compatible
changes; and we bump major for incompatible changes, and patchlevel for
bugfixes.

We absolutely MUST refrain from doing ESR's nonsense of calling .0
patchlevels "gold". The latest patchlevel with the same major.minor is
"gold", currently 6.2.5, rather than 6.2.0 - and we need to communicate
this change in versioning to our users.

I'd also suggest that we do bugfix patches only on patchlevels
henceforth, similar to what the GCC team do (they are a bit more radical
and will just fix regressions, not even regular bugs).

-- 
Matthias Andree

Encrypted mail welcome: my GnuPG key ID is 0x052E7D95 (PGP/MIME preferred)

[fetchmail-devel] Re: Not Yet Reviewed Queue is Empty!

[Graham W. <gr...@mk...>]

On Tue, Oct 12, 2004 at 10:54:32PM -0400, Rob Funk wrote:
> Matthias Andree wrote:
> > - charset of localized bounce mails
> > - update localization (.po files) via the translation project
> >   (I'll handle this as soon as we know we're not making any more code or
> >   message changes)
> > - do smoke compile tests on the various machines we have access to
> > - package a release candidate and call for testers in the public
> >
> > Anything missing?
> 
> There's still work to be done for building a release, converting Eric's 
> scripts over to the non-ESR Berlios environment.

Once we've figured out how were going to build release tarballs, I'd
like to look at the differences between the 6.2.5 tarball and the 6.2.6
tarball, and make sure we are aware of all of the changes.

Hmm... come to think of it, should this release be 6.3.0? This is the
first release in a while, and the maintainers have changed. Seems to me
to warrant more than a patch version number increased.

-- 
gram

Re: [fetchmail-devel] Not Yet Reviewed Queue is Empty!

[Rob F. <rf...@fu...>]

Matthias Andree wrote:
> apart from the norman-fix_config_reload I queried in a mail a few
> minutes ago, our "not yet reviewed" queue is empty.

Awesome, thanks!

> - charset of localized bounce mails
> - update localization (.po files) via the translation project
>   (I'll handle this as soon as we know we're not making any more code or
>   message changes)
> - do smoke compile tests on the various machines we have access to
> - package a release candidate and call for testers in the public
>
> Anything missing?

There's still work to be done for building a release, converting Eric's 
scripts over to the non-ESR Berlios environment.

Since Eric has regained control of the old mailing lists, it would be good 
to coordinate with him to get those moved over.

-- 
==============================|   "A microscope locked in on one point
 Rob Funk <rf...@fu...> |Never sees what kind of room that it's in"
 http://www.funknet.net/rfunk |    -- Chris Mars, "Stuck in Rewind"

[fetchmail-devel] Not Yet Reviewed Queue is Empty!

[Matthias A. <ma...@dt...>]

OK,

apart from the norman-fix_config_reload I queried in a mail a few
minutes ago, our "not yet reviewed" queue is empty. Most of the stuff
was a feature patch one way or another, or was invasive, which Rob
deemed inappropriate for 6.2.6 - which should go out the door this year
preferably... details are in the Wiki at
http://openfacts.berlios.de/index-en.phtml?title=FetchmailPatchReview

So the remaining issue are:

- charset of localized bounce mails
- update localization (.po files) via the translation project
  (I'll handle this as soon as we know we're not making any more code or
  message changes)
- do smoke compile tests on the various machines we have access to
- package a release candidate and call for testers in the public

Anything missing?

-- 
Matthias Andree

Encrypted mail welcome: my GnuPG key ID is 0x052E7D95 (PGP/MIME preferred)

[fetchmail-devel] Patches continued... includes call for testing

[Matthias A. <ma...@dt...>]

...it's been silent for a while, and I've been looking at some more
patches.

I have committed  2003-10-23-6.2.5-mauermann-bounce.diff  with a minor
change (i. e. I dropped the hunk that changed logging in sink.c:361
(version 3939)) because it looked sane, but I do not have a setup that
allows me to test these bounces, can someone of you check if bounces
after SMTP delivery failed still work?


I have also moved Sunil's  2003-10-11-6.2.4-shetye-limitflush.diff
to the "post 6.2.6" list as it adds a new feature.


I don't know how to proceed WRT
2003-10-29-6.2.4-norman-fix_config_reload.diff, it doesn't apply.


The patch reads:
-    /* avoid parsing the config file if all we're doing is killing a daemon */ 
-    if (!quitmode && !pid)
+    /*
+     * Avoid parsing the config file if all we're doing is killing a daemon,
+     * unless this appears to be a self-restarted instance because we detected an 
+     * "under-our-noses" configuration file change, in which case we definitely want
+     * to reload the config file.
+     */
+    if ( ( !quitmode && !pid ) || pid == getpid() )

The code now reads:

    /* avoid parsing the config file if all we're doing is killing a daemon */ 
    if (!(quitmode && argc == 2))
        implicitmode = load_params(argc, argv, optind);

I wonder if the current shape is correct, and doubt it - assuming we're
just killing a daemon based on -q and just one argument is
bold. Opinions solicited.

-- 
Matthias Andree

Encrypted mail welcome: my GnuPG key ID is 0x052E7D95 (PGP/MIME preferred)

[fetchmail-devel] Re: web page & 6.2.5 release

[Graham W. <bo...@de...>]

On Mon, Aug 09, 2004 at 03:28:48AM -0400, Rob Funk wrote:
> I set up an actual web page at http://fetchmail.berlios.de/, based on ESR's 
> fetchmail pages but without some stuff that seemed extraneous.  I also 
> went through and tried edited the pages to change the voice (e.g. 
> s/I/we/g), but I gave up about a third of the way through the FAQ.  And I 
> didn't check any of it into SVN, just edited the HTML on the site.

Thanks for doing this Rob. If we are interested interested, I can create
a separate repository for web page stuff.

-- 
gram

[fetchmail-devel] Re: SVN SSL problem?

[Graham W. <bo...@de...>]

On Mon, Sep 13, 2004 at 10:20:18AM +0100, Brian Candler wrote:
> On Sun, Sep 12, 2004 at 12:27:10PM -0400, Rob Funk wrote:
> > Did I miss a change?
> 
> $ openssl s_client -connect decoy.wox.org:443
> CONNECTED(00000003)
> depth=1 /C=US/ST=Texas/L=Dallas/O=decoy.wox.org/CN=Certificate authority/emailAddress=ca...@de...
> verify error:num=19:self signed certificate in certificate chain
> verify return:0
> 15428:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure:/usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_pkt.c:1052:SSL alert number 40
> 15428:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:/usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/s23_lib.c:226:
> $ 
> 
> So this server, whatever it is, is badly broken. (openssl s_client is
> normally quite happy to connect to sites with self-signed certificates)

The server works fine. It is, however, set up to always check client
certificates. I tested using s_client with a client certificate and
everything seemed to work fine.

-- 
gram

Re: [fetchmail-devel] SVN SSL problem?

[Rob F. <rf...@fu...>]

Matthias Andree wrote:
> Yes, I believe so. I have no problem accessing the repo. Graham had
> asked for a new key and CSR to be generated and should have sent you a
> new signed certificate in response (in private mail), which needs to be
> installed.
>
> The old key and certificate have expired and are invalid.

Yeah, I got it taken care of.  Oops.

-- 
==============================|   "A microscope locked in on one point
 Rob Funk <rf...@fu...> |Never sees what kind of room that it's in"
 http://www.funknet.net/rfunk |    -- Chris Mars, "Stuck in Rewind"