fetchmail-devel — Development related discussion

Re: [fetchmail-devel] [PATCH] fix listener mode

[Matthias A. <mat...@gm...>]

Sunil Shetye <sh...@bo...> writes:

> Quoting from Matthias Andree's mail on Tue, Dec 13, 2005:
>> > [OFFTOPIC] While debugging this, I found a weird bug relating to this
>> > SMTP/LMTP mode. It seems that once fetchmail switches to LMTP mode,
>> > there is no going back to SMTP mode. Try this simple fetchmail
>> > command:
>> >
>> > $ fetchmail -v --smtphost /this/socket/is/down,localhost
>> >
>> > fetchmail sends LHLO to the localhost SMTP server. Note that SMTP
>> > cannot be forced as the user could be running this configuration:
>
> Here is a patch which fixes this issue. This basically removes the
> global smtp_mode variable and passes the desired mode in every smtp
> function.

Wow, that patch is huge.  I'll happily merge it or something similar
onto the trunk (cleanup is always good), but for 6.3.1 I'd prefer a
smaller patch, even if it means hacking the state structures in some
places (for instance, upon sending a bounce and completion of the bounce
function).

I wonder if bouncing messages is the right thing at all. There should be
no reason to ever bounce in singledrop mode, and multidrop mode should
also only bounce for unknown users (but actually, that's the task of the
upstream servers; catchall OTOH is not a good idea any more nowadays).

> You may compare the output of
>
> $ fetchmail -v --smtphost /this/socket/is/down,localhost
>
> before and after the patch.

-- 
Matthias Andree

Re: [fetchmail-devel] Re: RFC: Patch to fix Berlios Bug #5849

[Matthias A. <mat...@gm...>]

Sunil Shetye <sh...@bo...> writes:

> Quoting from Matthias Andree's mail on Tue, Dec 13, 2005:
>> Sunil Shetye <sh...@bo...> writes:
>> 
>> > $ fetchmail -v --lmtp --smtphost localhost/port,/this/socket/is/up
>> >
>> > where localhost is indeed meant to be an LMTP server.
>> >
>> > The situation gets worse if fetchmail sends a bounce message in
>> > between!
>> 
>> One more thing: the whole LMTP implementation is a mess. Why would the
>> kind of socket (UNIX vs. INET) determine if SMTP or LMTP is spoken by
>> the service?
>
> It looks like it is assumed that there are no SMTP UNIX sockets.

Right, but my personal feeling is that a separate global --lmtp option
is plain ugly and undesirable, and LMTP hosts do not belong in
"--smtphost" for consistency reasons. 

I'd rather call this --mailsink (or perhaps, if someone has a better
name) and use tuples of {PROTOCOL}host/port, {PROTOCOL}[host]/port or
similar (delimiters around host similar to Postfix which goes out of the
IPv6 colon confusion when using :port).

PROTOCOL might default to SMTP if not given.

> I think, it is even more complex to give the port and type with every
> inet host and type with every unix socket. And there is no way of
> specifying different smtp options (esmtpname, esmtppassword) per
> smtphost.
>
> This should be a TODO.
>
> Any ideas, comments on this?

Yup, 6.4.0 stuff. This falls into a similar category as a hierarchical
rcfile parser with option inheritance. The current idea would be to
scope settings similar to C auto variables, and the result will then
resemble the ISC DHCP and ISC BIND configuration files.

I've added something to the TODO on the trunk.

-- 
Matthias Andree

Re: [fetchmail-devel] [Bug #5887] Spurious(?) 553 leads to lost mail

[Matthias A. <mat...@gm...>]

no...@be... writes:

> Project: Community Fetchmail
> Category: None
> Status: Open
> Resolution: None
> Bug Group: None
> Priority: 5
> Submitted by: dronis
> Assigned to : none
> Summary: Spurious(?) 553 leads to lost mail

I've closed this bug as invalid. Fetchmail behaves as documented,
problem is in the SMTP listener.

-- 
Matthias Andree

[fetchmail-devel] fetchmail 6.3.1 release candidate - please test

[Matthias A. <mat...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings,

I have released the final 6.3.1 release candidate,
dubbed fetchmail-6.3.1-rc1. It is available from
http://home.pages.de/~mandree/fetchmail/

Unless regressions since 6.3.0 are found, this will be released
as 6.3.1 somewhen next week.

The French translation lacks six messages I cannot translate properly by
myself, help will be appreciated (just send the missing msgid/msgstr
pairs to me either on- or off-list).

Please follow up to <fet...@li...> only.

Note that the fetchmail-friends@ list is supposed to be discontinued,
please resubscribe at BerliOS, for information, please see
<https://lists.berlios.de/mailman/listinfo/fetchmail-users>

These are the changes since 6.3.0:

>-------------------------------------------------------------------------------
* The multidrop To/Cc guessing code along with the fragile duplicate suppressor
  is deprecated and may be removed from a future release. (MA)
>-------------------------------------------------------------------------------
* Fix broken default port in POP2. Patch by Stanislav Brabec, SUSE [CZ]. (MA)
* Fix manual page, some lines starting with ' were escaped by \&.
  Reported by Simon Barner. (MA)
* Ship with gettext-0.14.3 again, as 6.2.9-rc10 did. Found by Sunil Shetye. (MA)
* Actually set default SSL certificate path if --sslcertpath is unset.
  Reported by Heino Tiedemann and Rob MacGregor. (MA)
* Remove bogus Netscape IMAP4rev1 Service >= 3.6 warning about BODY[TEXT]
  that we are not using. Patch by Sunil Shetye. (MA)
* Plug potential memory and socket leak when polling multiple folders or when
  the upstream sends bogus message sizes.  Patch by Sunil Shetye. (MA)
* Fix segfault (null pointer dereference) in multidrop mode with headerless
  email.  Reported by Daniel Drake, patch by Sunil Shetye. (MA)
* Update Catalan translation, by Ernest Adrogué Calveras. (MA)
* Fix segfault (null pointer dereference) on some operating systems with
  fetchmail's obsolete DNS MX/host alias lookups in multidrop mode.
  Patch by Dr.-Ing. Andreas Haakh. (MA)
* Close SMTP sockets early, to reduce resource usage, trigger earlier delivery
  with some MTAs and avoid SIGPIPE (SIG 13) when the SMTP listener gets bored
  and drops the connection after timeout.  Patch by Sunil Shetye. (MA)
* Don't treat hitting a fetch limit as error.  Patch by Sunil Shetye. (MA)
* Fix negative "messages left on server" on idle/repoll with fetchlimit.
  Patch by Sunil Shetye. (MA)
* Properly track logout stage.  Patch by Sunil Shetye. (MA)
* Preserve error conditions across postconnect script.  Sunil Shetye. (MA)
* Do not trash destination domain if multiple messages are forwarded into the
  same SMTP/LMTP connection. Reported by Joachim Feise, Berlios Bug #5849. (MA)
  (This patch had not been committed to SVN at the time of the 6.3.1-pre1
  release on 2005-12-13.)
* Manual page: Add "-md5" to "openssl x509" example in --sslfingerprint
  documentation.  Suggested by Jason White. (MA)
* Cope with servers that return UID information in response to non-UID
  RFC822.{SIZE|HEADER} requests. Reported by Jason White.
  Patch suggestion by by Sunil Shetye, simplified by MA.
>-------------------------------------------------------------------------------

- -- 
Matthias Andree
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFDpUR3vmGDOQUufZURAguXAJ9tslzlbNY3Xn7LiWwQdOqZfUXIlACgi6mJ
9glYchTf96xevABPML/kByE=
=o61G
-----END PGP SIGNATURE-----

Re: [fetchmail-devel] Re: RFC: Patch to fix Berlios Bug #5849

[Matthias A. <mat...@gm...>]

Sunil Shetye <sh...@bo...> writes:

> One thing that had stumped me then was the FALLBACK_MDA feature. After
> splitting, implementing FALLBACK_MDA would be difficult (but not
> impossible). Should this feature be continued to be supported?

I don't like this feature, and ESR agreed to change the default to be
"none" when I urged in that direction, so we've been without default
fallback for a while now.

Such fallback layers are rarely tested thoroughly and will often behave
differently enough from the "first attempt" code that I'd call such
fallback layers "nondeterministic" (for practical purposes), and I'd
rather suggest that people make sure their primary delivery target, be
it SMTP, LMTP, BSMTP or MDA, is reliable.  The last thing I want is that
people search for their mail because some fallback layer delivered it to
another place that the user has long forgotten about.

In short: I have no problem with removing the fallback code altogether.

> Once splitting is completed, making MDA work in multidrop should not
> be a problem.
>
> Here would be the aim of this development:
>
> - Split sink.c into sink-smtp.c, sink-bsmtp.c, sink-mda.c (and
>   possibly, sink-lmtp.c).

I'm not sure if sink-smtp.c, sink-bsmtp.c and sink-lmtp.c are
sufficiently different to warrant three separate modules. They share
quite a lot of code with some minor differences, but in the end it's
always an SMTP dialect. If the differences between these can be factored
out with common code shared, that'll be good. For instance, BSMTP and
SMTP are quite similar, we'll just replace a backend writer function (or
file/socket descriptor) and pretend all incoming response codes had been
the proper OK code for the respective stage (20X, 250, 354). LMTP is
also quite similar to ESMTP so that shared code is probably more
maintainable in the long run.

If C++ is of advantage (class inheritance), I don't mind either, we'll
slap "7.0.0" (rather than 6.4.0) on the new code then.

> - Make a structure (similar to struct method) to handle the sending of
>   messages.
> - Make the mda code handle the exit code in accordance with
>   <sysexits.h>.
> - Make the mda code work in multidrop mode.
> - Make the parser accept options per sink.
> - Make all sink options (like esmtpname) per sink.

> - Make FALLBACK_MDA work.
>
> Potential issues involved:
> - Cross platform:
>   * Does <sysexits.h> exists on all platforms?

Ah well... it's mail system specific, originated AFAICS in BSD4.3 and
was also present in System V.  It should therefore be fairly portable,
but we might want to peek at Postfix's source code to see if sysexits.h
specific workarounds are there.

>   * Do all platforms have at least EX_OK, EX__BASE, EX__MAX?
>   * Do all platforms have the common exit codes that need to be
>     handled specially (like EX_NOUSER)?
> - How to specify options per smtphost?

Well, a configuration system with inheritance was already suggested, but
would be a somewhat larger change, too.

> - Any other issues?

How to bounce (or what is other good non-bounce behavior) in MDA mode?

> Any comments on this?

-- 
Matthias Andree

[fetchmail-devel] [Bug #5887] Spurious(?) 553 leads to lost mail

[<no...@be...>]

Bug #5887, was updated on 2005-Dec-14 22:21
Here is a current snapshot of the bug.

Project: Community Fetchmail
Category: None
Status: Open
Resolution: None
Bug Group: None
Priority: 5
Submitted by: dronis
Assigned to : none
Summary: Spurious(?) 553 leads to lost mail

Details: I've just started using fetchmail to bring down mail from fetchmail  6.2.5.2+SSL+NLS to and from on a linux box.  Itt seemed to work, however, I've just discovered that some mail was simply being dropped.  Running with -v shows:

fetchmail: POP3> LIST 3
fetchmail: POP3< +OK 3 2543
fetchmail: POP3> RETR 3
fetchmail: POP3< +OK 2543 octets
reading message margaret@localhost:3 of 13 (2543 octets)
fetchmail: SMTP> MAIL FROM:<na...@fu...> BODY=7BIT SIZE=2543
fetchmail: SMTP< 553 5.1.8 <na...@fu...>... Domain of sender address
 na...@fu... does not exist
fetchmail: SMTP error: 553 5.1.8 <na...@fu...>... Domain of sender address na...@fu... does not exist
fetchmail: SMTP> RSET
fetchmail: SMTP< 250 2.0.0 Reset state
 flushed
fetchmail: POP3> DELE 3
fetchmail: POP3< +OK Message 3 has been deleted.

The domain and user both exist (and in any case, the e-mail gets to a very heavily spam-protected site I'm using as my mailhost).

host -a fukuoka-pu.ac.jp
Trying "fukuoka-pu.ac.jp"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53810
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 2, ADDITIONAL: 1

;; QUESTION SECTION:
;fukuoka-pu.ac.jp.              IN      ANY

;; ANSWER SECTION:
fukuoka-pu.ac.jp.       86400   IN      MX      10 fpu-vs.fukuoka-pu.ac.jp.
fukuoka-pu.ac.jp.       86400   IN      SOA     sparc.fukuoka-pu.ac.jp. postmaster.fukuoka-pu.ac.jp. 20030308 10800 3600 604800 86400
fukuoka-pu.ac.jp.       86400   IN      NS      sparc.fukuoka-pu.ac.jp.
fukuoka-pu.ac.jp.       86400   IN      NS      fwugate.fwu.ac.jp.

;; AUTHORITY SECTION:
fukuoka-pu.ac.jp.       86400   IN      NS      sparc.fukuoka-pu.ac.jp.
fukuoka-pu.ac.jp.       86400   IN      NS      fwugate.fwu.ac.jp.

;; ADDITIONAL SECTION:
fpu-vs.fukuoka-pu.ac.jp. 86400  IN      A       202.236.111.232

Notice the existence of the MX section of the lookup.  Shouldn't that be enough not to trigger a 553?

In any event,  is there a way to disable this "feature"  There's already tons of spam filtering upstream of me and my mailclient will filter it again.

David



For detailed info, follow this link:
http://developer.berlios.de/bugs/?func=detailbug&bug_id=5887&group_id=1824

[fetchmail-devel] Re: RFC: Patch to fix Berlios Bug #5849

[Sunil S. <sh...@bo...>]

Quoting from Matthias Andree's mail on Tue, Dec 13, 2005:
> Well, one thing that astonished me a bit is that on one hand the input
> side is clearly encapsulated with object-oriented practices, while the
> output side (SMTP/LMTP; MDA; BSMTP) is not nearly as clearly separated.
> On the other hand, we use POP3 sibling "protocols" where a distinction
> per "authentication" would be sufficient.

...

> Sending bounce messages is a sore spot. AFAIR, no bounce messages are
> ever sent if bogofilter is in MDA mode. MDA mode doesn't work for
> multidrop, as you'll probably recall from past discussions over at
> fetchmail-friends. I don't know about BSMTP restrictions.

I had worked on splitting up sink.c. I gave up when there were too
many local changes and not even read access to the repository then.
Also, by the time I had almost finished, a new version of fetchmail
had come out which had too many changes in sink.c.

One thing that had stumped me then was the FALLBACK_MDA feature. After
splitting, implementing FALLBACK_MDA would be difficult (but not
impossible). Should this feature be continued to be supported?

Once splitting is completed, making MDA work in multidrop should not
be a problem.

Here would be the aim of this development:

- Split sink.c into sink-smtp.c, sink-bsmtp.c, sink-mda.c (and
  possibly, sink-lmtp.c).
- Make a structure (similar to struct method) to handle the sending of
  messages.
- Make the mda code handle the exit code in accordance with
  <sysexits.h>.
- Make the mda code work in multidrop mode.
- Make the parser accept options per sink.
- Make all sink options (like esmtpname) per sink.
- Make FALLBACK_MDA work.

Potential issues involved:
- Cross platform:
  * Does <sysexits.h> exists on all platforms?
  * Do all platforms have at least EX_OK, EX__BASE, EX__MAX?
  * Do all platforms have the common exit codes that need to be
    handled specially (like EX_NOUSER)?
- How to specify options per smtphost?
- Any other issues?

Any comments on this?

-- 
Sunil Shetye.

[fetchmail-devel] Re: RFC: Patch to fix Berlios Bug #5849

[Sunil S. <sh...@bo...>]

Quoting from Matthias Andree's mail on Tue, Dec 13, 2005:
> Sunil Shetye <sh...@bo...> writes:
> 
> > $ fetchmail -v --lmtp --smtphost localhost/port,/this/socket/is/up
> >
> > where localhost is indeed meant to be an LMTP server.
> >
> > The situation gets worse if fetchmail sends a bounce message in
> > between!
> 
> One more thing: the whole LMTP implementation is a mess. Why would the
> kind of socket (UNIX vs. INET) determine if SMTP or LMTP is spoken by
> the service?

It looks like it is assumed that there are no SMTP UNIX sockets.

I think, it is even more complex to give the port and type with every
inet host and type with every unix socket. And there is no way of
specifying different smtp options (esmtpname, esmtppassword) per
smtphost.

This should be a TODO.

Any ideas, comments on this?

-- 
Sunil Shetye.

[fetchmail-devel] [PATCH] fix listener mode [WAS: Re: RFC: Patch to fix Berlios Bug #5849]

[Sunil S. <sh...@bo...>]

Quoting from Matthias Andree's mail on Tue, Dec 13, 2005:
> > [OFFTOPIC] While debugging this, I found a weird bug relating to this
> > SMTP/LMTP mode. It seems that once fetchmail switches to LMTP mode,
> > there is no going back to SMTP mode. Try this simple fetchmail
> > command:
> >
> > $ fetchmail -v --smtphost /this/socket/is/down,localhost
> >
> > fetchmail sends LHLO to the localhost SMTP server. Note that SMTP
> > cannot be forced as the user could be running this configuration:

Here is a patch which fixes this issue. This basically removes the
global smtp_mode variable and passes the desired mode in every smtp
function.

You may compare the output of

$ fetchmail -v --smtphost /this/socket/is/down,localhost

before and after the patch.

-- 
Sunil Shetye.

Re: [fetchmail-devel] Re: RFC: Patch to fix Berlios Bug #5849

[Matthias A. <mat...@gm...>]

Sunil Shetye <sh...@bo...> writes:

> $ fetchmail -v --lmtp --smtphost localhost/port,/this/socket/is/up
>
> where localhost is indeed meant to be an LMTP server.
>
> The situation gets worse if fetchmail sends a bounce message in
> between!

One more thing: the whole LMTP implementation is a mess. Why would the
kind of socket (UNIX vs. INET) determine if SMTP or LMTP is spoken by
the service?

-- 
Matthias Andree

Re: [fetchmail-devel] Re: RFC: Patch to fix Berlios Bug #5849

[Matthias A. <mat...@gm...>]

Sunil Shetye <sh...@bo...> writes:

> Quoting from Matthias Andree's mail on Tue, Dec 13, 2005:
>> Matthias Andree <mat...@gm...> writes:
>> > Joachim Feise reported that his smtphost setting is lost/ignored since
>> > 6.3.0. I could reproduce this.
>> > <http://developer.berlios.de/bugs/?func=detailbug&bug_id=5849&group_id=1824>
>> >
>> > However, I use neither LMTP nor multidrop, so I need help determining if
>> > the attached patch that I made is safe.
>> 
>> Hm. This is the same sink.c patch but without space changes and should
>> thus be easier to review. Please comment:
>
> Oops, my mistake.

Is it really?

> Yes, this patch is certainly required. This will also avoid an
> xfree/xstrdup on every mail. I would even recommend moving the
> declaration of parsed_host into the if block.
>
> [OFFTOPIC] While debugging this, I found a weird bug relating to this
> SMTP/LMTP mode. It seems that once fetchmail switches to LMTP mode,
> there is no going back to SMTP mode. Try this simple fetchmail
> command:
>
> $ fetchmail -v --smtphost /this/socket/is/down,localhost
>
> fetchmail sends LHLO to the localhost SMTP server. Note that SMTP
> cannot be forced as the user could be running this configuration:

Well, one thing that astonished me a bit is that on one hand the input
side is clearly encapsulated with object-oriented practices, while the
output side (SMTP/LMTP; MDA; BSMTP) is not nearly as clearly separated.
On the other hand, we use POP3 sibling "protocols" where a distinction
per "authentication" would be sufficient.

We certainly have some tasks ahead before fetchmail 7.0.0 some day - and
I wonder if we should talk to the compilers in C++ rather than C.

> $ fetchmail -v --lmtp --smtphost localhost/port,/this/socket/is/up
>
> where localhost is indeed meant to be an LMTP server.
>
> The situation gets worse if fetchmail sends a bounce message in
> between!

Sending bounce messages is a sore spot. AFAIR, no bounce messages are
ever sent if bogofilter is in MDA mode. MDA mode doesn't work for
multidrop, as you'll probably recall from past discussions over at
fetchmail-friends. I don't know about BSMTP restrictions.

-- 
Matthias Andree

[fetchmail-devel] Re: RFC: Patch to fix Berlios Bug #5849

[Sunil S. <sh...@bo...>]

Quoting from Matthias Andree's mail on Tue, Dec 13, 2005:
> Matthias Andree <mat...@gm...> writes:
> > Joachim Feise reported that his smtphost setting is lost/ignored since
> > 6.3.0. I could reproduce this.
> > <http://developer.berlios.de/bugs/?func=detailbug&bug_id=5849&group_id=1824>
> >
> > However, I use neither LMTP nor multidrop, so I need help determining if
> > the attached patch that I made is safe.
> 
> Hm. This is the same sink.c patch but without space changes and should
> thus be easier to review. Please comment:

Oops, my mistake. Yes, this patch is certainly required. This will
also avoid an xfree/xstrdup on every mail. I would even recommend
moving the declaration of parsed_host into the if block.

[OFFTOPIC] While debugging this, I found a weird bug relating to this
SMTP/LMTP mode. It seems that once fetchmail switches to LMTP mode,
there is no going back to SMTP mode. Try this simple fetchmail
command:

$ fetchmail -v --smtphost /this/socket/is/down,localhost

fetchmail sends LHLO to the localhost SMTP server. Note that SMTP
cannot be forced as the user could be running this configuration:

$ fetchmail -v --lmtp --smtphost localhost/port,/this/socket/is/up

where localhost is indeed meant to be an LMTP server.

The situation gets worse if fetchmail sends a bounce message in
between!

-- 
Sunil Shetye.

Re: [fetchmail-devel] fetchmail 6.3.1-pre1 snapshot - please test

[Matthias A. <mat...@gm...>]

Matthias Andree <mat...@gm...> writes:

> Greetings,
>
> I have released a first 6.3.1 beta quality release, 
                         
make that a "release candidate", as this line further down:

> I plan to release this as 6.3.1 next week-end.

already suggested. Sorry for the mess.

-- 
Matthias Andree

[fetchmail-devel] fetchmail 6.3.1-pre1 snapshot - please test

[Matthias A. <mat...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings,

I have released a first 6.3.1 beta quality release, 
dubbed fetchmail-6.3.1-pre1. It is available from
http://home.pages.de/~mandree/fetchmail/

I am calling it "beta quality" because some of the changes at the bottom
of the list require testing in multidrop and/or LMTP configurations
that I do not currently have available.

Feedback is sought, particularly from those who reported bugs against
6.3.0 and haven't yet responded if a patch they had been sent fixed
their bug.

I plan to release this as 6.3.1 next week-end.

These are the changes since 6.3.0:

- --------------------------------------------------------------------------------
* The multidrop To/Cc guessing code along with the fragile duplicate suppressor
  is deprecated and may be removed from a future release. (MA)
- --------------------------------------------------------------------------------
* Fix broken default port in POP2. Patch by Stanislav Brabec, SUSE [CZ]. (MA)
* Fix manual page, some lines starting with ' were escaped by \&.
  Reported by Simon Barner. (MA)
* Ship with gettext-0.14.3 again, as 6.2.9-rc10 did. Found by Sunil Shetye. (MA)
* Actually set default SSL certificate path if --sslcertpath is unset.
  Reported by Heino Tiedemann and Rob MacGregor. (MA)
* Remove bogus Netscape IMAP4rev1 Service >= 3.6 warning about BODY[TEXT]
  that we are not using. Patch by Sunil Shetye. (MA)
* Plug potential memory and socket leak when polling multiple folders or when
  the upstream sends bogus message sizes.  Patch by Sunil Shetye. (MA)
* Fix segfault (null pointer dereference) in multidrop mode with headerless
  email.  Reported by Daniel Drake, patch by Sunil Shetye. (MA)
* Update Catalan translation, by Ernest Adrogué Calveras. (MA)
* Fix segfault (null pointer dereference) on some operating systems with
  fetchmail's obsolete DNS MX/host alias lookups in multidrop mode.
  Patch by Dr.-Ing. Andreas Haakh. (MA)
* Close SMTP sockets early, to reduce resource usage, trigger earlier delivery
  with some MTAs and avoid SIGPIPE (SIG 13) when the SMTP listener gets bored
  and drops the connection after timeout.  Patch by Sunil Shetye. (MA)
* Don't treat hitting a fetch limit as error.  Patch by Sunil Shetye. (MA)
* Fix negative "messages left on server" on idle/repoll with fetchlimit.
  Patch by Sunil Shetye. (MA)
* Properly track logout stage.  Patch by Sunil Shetye. (MA)
* Preserve error conditions across postconnect script.  Sunil Shetye. (MA)
* Do not trash destination domain if multiple messages are forwarded into the
  same SMTP/LMTP connection. Reported by Joachim Feise, Berlios Bug #5849. (MA)
  (This patch had not been committed to SVN at the time of the 6.3.1-pre1
  release on 2005-12-13.)
- --------------------------------------------------------------------------------

- -- 
Matthias Andree
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFDnh6zvmGDOQUufZURAhWbAJ9ODJXjkSsOK6shLHJTMnxaOC1iCwCfVHhF
FhNN8hyq5w8R3OsQDVBwIZY=
=IzUo
-----END PGP SIGNATURE-----

Re: [fetchmail-devel] RFC: Patch to fix Berlios Bug #5849

[Matthias A. <mat...@gm...>]

Matthias Andree <mat...@gm...> writes:

> Greetings,
>
> Joachim Feise reported that his smtphost setting is lost/ignored since
> 6.3.0. I could reproduce this.
> <http://developer.berlios.de/bugs/?func=detailbug&bug_id=5849&group_id=1824>
>
> However, I use neither LMTP nor multidrop, so I need help determining if
> the attached patch that I made is safe.

Hm. This is the same sink.c patch but without space changes and should
thus be easier to review. Please comment:


Index: sink.c
===================================================================
--- sink.c	(Revision 4524)
+++ sink.c	(Arbeitskopie)
@@ -121,11 +121,9 @@
 
 	    ctl->smtphost = idp->id;  /* remember last host tried. */
 	    if(ctl->smtphost[0]=='/')
-		ctl->listener = LMTP_MODE;
-
-	    if (ctl->smtphost[0]=='/')
 	    {
-		parsed_host = NULL;
+		ctl->listener = LMTP_MODE;
+		xfree(parsed_host);
 		if ((ctl->smtp_socket = UnixOpen(ctl->smtphost))==-1)
 		    continue;
 	    }
@@ -195,7 +193,6 @@
 	}
 	set_timeout(0);
 	phase = oldphase;
-    }
 
     /*
      * RFC 1123 requires that the domain name part of the
@@ -214,11 +211,13 @@
        localhost as a domain part. */
     else
 	ctl->destaddr = xstrdup("localhost");
+	xfree(parsed_host);
+    }
+    /* end if (ctl->smtp_socket == -1) */
 
     if (outlevel >= O_DEBUG && ctl->smtp_socket != -1)
 	report(stdout, GT_("forwarding to %s\n"), ctl->smtphost);
 
-    xfree(parsed_host);
     return(ctl->smtp_socket);
 }
 

-- 
Matthias Andree

[fetchmail-devel] RFC: Patch to fix Berlios Bug #5849

[Matthias A. <mat...@gm...>]

Greetings,

Joachim Feise reported that his smtphost setting is lost/ignored since
6.3.0. I could reproduce this.
<http://developer.berlios.de/bugs/?func=detailbug&bug_id=5849&group_id=1824>

However, I use neither LMTP nor multidrop, so I need help determining if
the attached patch that I made is safe.

Looking forward to your reviews.

-- 
Matthias Andree

[fetchmail-devel] [PATCH] close smtp socket early

[Sunil S. <sh...@bo...>]

Hi,

The attached patch fixes the following bugs:

1. fetchmail should close smtp socket as soon as its work is done.
This matters when fetchmail is polling multiple hosts and the smtp
sockets just pile up till the end of the run. A SIGPIPE can also be
triggered if there is a lot of time interval between the end of poll
of the first mailserver and the end of run.

Running fetchmail -v, the output can be seen as:

6.3.0 querying mailserver1 (protocol IMAP) at ...: poll completed
...
6.3.0 querying mailserver2 (protocol IMAP) at ...: poll completed
...
6.3.0 querying mailserver3 (protocol IMAP) at ...: poll completed
SMTP> QUIT
SMTP< 221 2.0.0 localhost closing connection
SMTP> QUIT
SMTP< 221 2.0.0 localhost closing connection
SMTP> QUIT
SMTP< 221 2.0.0 localhost closing connection

After this patch, the output will be:

SMTP> QUIT
SMTP< 221 2.0.0 localhost closing connection
6.3.0 querying mailserver1 (protocol IMAP) at ...: poll completed
...
SMTP> QUIT
SMTP< 221 2.0.0 localhost closing connection
6.3.0 querying mailserver2 (protocol IMAP) at ...: poll completed
...
SMTP> QUIT
SMTP< 221 2.0.0 localhost closing connection
6.3.0 querying mailserver3 (protocol IMAP) at ...: poll completed

2. PS_MAXFETCH is being treated as an error condition. It should be
treated as a successful run instead.

3. On reaching the fetchlimit, a negative count is shown of mails
remaining on the server when fetchmail IDLEs (or repolls) atleast once
with this configuration before reaching the fetchlimit:

poll mailserver
    protocol imap
    fetchlimit 150
    idle

The line looks like this:

fetchlimit 150 reached; -99 messages left on server mailserver account user

4. stage should be set to STAGE_LOGOUT on a normal logout. Also, a new
stage STAGE_PREAUTH has been added to handle the server greeting line
(currently unused). stage is now an enum.

5. The value of err is overwritten if there is a postconnect script or
during normal logout. It should be preserved as far as possible.

-- 
Sunil Shetye.

[fetchmail-devel] New Catalan PO file for `fetchmail'

[Translation P. R. <tra...@ir...>]

Hello, gentle maintainer. This is a message from the Translation
Project robot.

A revised PO file, for programs using the textual domain `fetchmail',
has been submitted by the team of translators taking care of the
Catalan language.  This particular file, along with all other PO files
pertaining to the same textual domain, is available as:
>    http://www.iro.umontreal.ca/translation/maint/fetchmail/ca.po

The file should soon be made available in mirror sites as:

>    ftp://ftp.unex.es/pub/gnu-i18n/po/maint/fetchmail/ca.po
>    http://translation.sf.net/maint/fetchmail/ca.po
>    ftp://tiger.informatik.hu-berlin.de/pub/po/maint/fetchmail/ca.po

This file has already been sent to you separately on 2005-12-10,
as a MIME invoice unpacking the file `fetchmail-6.3.0.ca.po'.

The following HTML page should also be updated by tomorrow.

>    http://www.iro.umontreal.ca/translation/HTML/domain-fetchmail.html

Please consider including all PO files, as they stand, in the `po/'
subdirectory of your next release of programs using that textual domain,
whether it is official or pretest.  Whenever you have a distribution ready
which holds a newer PO Template, please send the URL of this distribution
to the address below.  The distribution could be a pretest or a snapshot,
it does not even have to compile.  This is to be used by translators,
when they need to get some translation context from your sources.

Within the Translation Project, each PO Template file should have different
version numbers, but since it is not OK to have two different distributions
using same version numbers, this is not a problem in practice.

Contact me if any question arises.  Thanks for your collaboration,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                mailto:tra...@ir...

[fetchmail-devel] [Patch #748] Patch to fix bug 5849, smtphost ignored

[<no...@be...>]

Patch #748 has been updated. 

Project: fetchmail
Category: None
Status: Open
Submitted by: jfeise
Assigned to : none
Summary: Patch to fix bug 5849, smtphost ignored

-------------------------------------------------------
For more info, visit:

http://developer.berlios.de/patch/?func=detailpatch&patch_id=748&group_id=1824

Re: [fetchmail-devel] Re: fetchmail segfault on 'empty' mail

[Matthias A. <mat...@gm...>]

Sunil Shetye <sh...@bo...> writes:

> On a side note, I feel that the empty messages point to a problem of
> mail delivery error on your ISP side. For even if somebody sends you
> an empty mail, your ISP must be adding atleast two headers
> (Return-Path: and Delivered-To:). Otherwise, your multidrop setup
> would not have worked at all. So, even a minimal mail cannot be
> headerless.

It is also a violation of the 23 year old RFC-822 which requires that at
least From: and Date: and either of To: or Bcc: must be present.

-- 
Matthias Andree

Re: [fetchmail-devel] [PATCH] Re: fetchmail segfault on 'empty' mail

[Matthias A. <mat...@gm...>]

Daniel Drake <ds...@ge...> writes:

>> @@ -936,7 +928,7 @@
>>       * to break it in a way that blackholed mail.  Better to pass
>>       * the occasional duplicate than to do that...
>>       */
>> -    if (MULTIDROP(ctl))
>> +    if (MULTIDROP(ctl) && msgblk.headers)
>>      {
>>  	MD5_CTX context;
>>
>
> I also think it probably makes more sense than my change did. Would be great 
> to see it included in the next versions of fetchmail :)

Thanks for testing it. The change is already in the SVN trunk and
BRANCH_6-3, so expect to see it in fetchmail-6.3.1 somewhen next week.

-- 
Matthias Andree

[fetchmail-devel] Re: fetchmail segfault on 'empty' mail

[Sunil S. <sh...@bo...>]

Hi Daniel,

Quoting from Daniel Drake's mail on Thu, Dec 08, 2005:
> Thanks.
> 
> I haven't looked at the freeing changes, but I can confirm that this hunk 
> alone would have fixed the problem:

...

> I also think it probably makes more sense than my change did. Would be 
> great to see it included in the next versions of fetchmail :)

Thanks for following up on this issue.

On a side note, I feel that the empty messages point to a problem of
mail delivery error on your ISP side. For even if somebody sends you
an empty mail, your ISP must be adding atleast two headers
(Return-Path: and Delivered-To:). Otherwise, your multidrop setup
would not have worked at all. So, even a minimal mail cannot be
headerless.

-- 
Sunil Shetye.

Re: [fetchmail-devel] [PATCH] Re: fetchmail segfault on 'empty' mail

[Matthias A. <mat...@gm...>]

Sunil Shetye <sh...@bo...> writes:

> The problem is specific to multidrop mailbox only where the
> duplicate-message killer cannot handle a headerless mail. The attached
> patch should fix this issue. 
>
> The patch also cleans up the code by calling free() at one point
> only for static variables.

I've merged it with trivial changes (I've left the braces in because the
patches are easier to review this way and ambiguities are avoided).

I cannot test the new code (no multidrop here, let alone broken
multidrop that needs to parse headers), I hope it doesn't break
anything, and I'm looking forward to hear from Daniel and others who use
multidrop configurations if this patch works OK for them.

It appears as though all regression and new bug reports had been
addressed and we might see 6.3.1 soon.

-- 
Matthias Andree

Re: [fetchmail-devel] [PATCH] Re: fetchmail segfault on 'empty' mail

[Daniel D. <ds...@ge...>]

Sunil Shetye wrote:
> The problem is specific to multidrop mailbox only where the
> duplicate-message killer cannot handle a headerless mail. The attached
> patch should fix this issue. 
> 
> The patch also cleans up the code by calling free() at one point
> only for static variables.

Thanks.

I haven't looked at the freeing changes, but I can confirm that this hunk 
alone would have fixed the problem:


> @@ -936,7 +928,7 @@
>       * to break it in a way that blackholed mail.  Better to pass
>       * the occasional duplicate than to do that...
>       */
> -    if (MULTIDROP(ctl))
> +    if (MULTIDROP(ctl) && msgblk.headers)
>      {
>  	MD5_CTX context;
>  

I also think it probably makes more sense than my change did. Would be great 
to see it included in the next versions of fetchmail :)

Thanks,
Daniel

[fetchmail-devel] [PATCH] Re: fetchmail segfault on 'empty' mail

[Sunil S. <sh...@bo...>]

Hi,

Quoting from Daniel Drake's mail on Wed, Dec 07, 2005:
> I don't have any more of these mails right now, so this is fairly 
> fabricated, but more or less accurate.
> 
> Before:
> 
> fetchmail: 6.2.5.2 querying aap.businessserve.co.uk (protocol POP3) at Wed, 
> 07 Dec 2005 13:20:02 +0000 (GMT): poll started
> fetchmail: POP3< +OK <..>
> fetchmail: POP3> CAPA
> fetchmail: POP3< -ERR authorization first
> fetchmail: authorization first
> fetchmail: Repoll immediately on xxx
> fetchmail: POP3< +OK <..>
> fetchmail: POP3> USER xxx
> fetchmail: POP3< +OK
> fetchmail: POP3> PASS *
> fetchmail: POP3< +OK
> fetchmail: POP3> STAT
> fetchmail: POP3< +OK 26 82969
> fetchmail: POP3> LIST 1
> fetchmail: POP3< +OK 1 1329
> fetchmail: POP3> RETR 1
> fetchmail: POP3< +OK 1329 octets follow.
> reading message xyz:1 of 26 (1329 octets)
> Segmentation fault

The problem is specific to multidrop mailbox only where the
duplicate-message killer cannot handle a headerless mail. The attached
patch should fix this issue. 

The patch also cleans up the code by calling free() at one point
only for static variables.

-- 
Sunil Shetye.