fetchmail-devel — Development related discussion

Re: [fetchmail-devel] [Bug #15103] Fetchmail Fails with ASSP (http://assp.sourceforge.net/)

[Matthias A. <mat...@gm...>]

Frederic Marchal schrieb:

[in response to BerliOS Bug #15103]

> I solved this issue by running ASSP in test mode (check option "Bayesian 
> Test Mode") and prefixing the subject with something like [SPAM] (in 
> option "Prepend Spam Subject"). It means ASSP won't stop mails seen as 
> bayesian spam but will tag the subject. The mails can then be filtered 
> out automatically by the final recipient.

This, and my being bogofilter's co-maintainer (bogofilter also does
Bayesian filtering), prompted me to have a glimpse at ASSP, and I must say
that the promises it makes are way too bold. ASSP's self-advertising as the
best tool aside, the assertion that Bayes filter were to intelligently
decide is just crap. Bayes filters are making statistical decisions based
on past training. "Maintenance free" doesn't work for them.

> Since you are downloading your mails with fetchmail, you can't simply 
> reject them because it is too late to do that. The purpose in rejecting 
> them is to leave the bad mails on the hand of the one that accepted them 
> in the first place. But in your case, you already have accepted them and 
> you have to deal with them...

True enough. I had, before reading your message, already closed the bug
report as invalid as ASSP violates the SMTP protocol. You've just handed me
another good reason not to change fetchmail, because the whole setup is
just twisted.

Beyond that, the assertion of the original were that ASSP were inefficient
(the wording was like 'resource use goes through the roof' or similar) if
it were to look at the whole message -- and that's a particular drawback or
implementation issue of ASSP that fetchmail certainly isn't meant to fix,
and I'm definitely not meaning to.

> If your server is also accepting external mails through direct 
> connections via port 25 and you want ASSP to reject those mails, then 
> you have to open some kind of backdoor on your mail server to let 
> fetchmail inject the mails it is reading. Be sure to properly firewall 
> that port to prevent anybody but fetchmail to inject mails.

Well, while a firewall certainly isn't a bad idea, it's not required
either. Most SMTP servers can be made to listen on particular addresses,
which for most of them is equivalent to binding them to particular
interfaces. Binding the SMTP server to interfaces that are only locally
accessible, such as the loopback interface or its address 127.0.0.1, is
sufficient to prevent access from the outside. It's doable with Postfix and
Exim. I don't know about Sendmail or Courier. It's also possible with qmail
running off tcpsvd or tcpserver, but I'd recommend against using qmail for
the delayed-bounce behaviour and dozens of other reasons beyond this
discussion.

> By the way, I don't advise to let ASSP delete or reject blindly bayesian 
> spams because, as you noticed, it doesn't take the whole mail into 
> account and, in general, it does a poor job at filtering spams. It tends 
> to have a high rate of false positives and you will loose good e-mails 
> if you don't keep an eye on what it does.

But then it seems to me that if you're using fetchmail or getmail, that
integrating bogofilter, spamprobe or qsf in the mail system is more
efficient than ASSP if it's prone to high resource use.

At least bogofilter and SpamProbe are much faster than SpamAssassin's bayes
mode. :-)

HTH

-- 
Matthias Andree

Re: [fetchmail-devel] [Bug #15103] Fetchmail Fails with ASSP (http://assp.sourceforge.net/)

[Frederic M. <fre...@wo...>]

ad...@be... a écrit :
> Bug #15103, was updated on 2009-Jan-23 07:36
> Here is a current snapshot of the bug.
>
> Project: Community Fetchmail
> Category: None
> Status: Open
> Resolution: None
> Bug Group: None
> Priority: 5
> Submitted by: darthasp
> Assigned to : none
> Summary: Fetchmail Fails with ASSP (http://assp.sourceforge.net/)
>
> Details: Hi There,
>
> I am trying to download messages from a catchall mailbox and forward them on to a mail server that sits been an ASSP installation.
>
> The problem I am having is that ASSP only looks at the first 2000 bytes of the message in order to determine whether it is SPAM or not.
>
> If the message that Fetchmail is sending is larger than this size and ASSP considers it to be SPAM - ASSP closes the connection with a 550: response before Fetchmail has finished copying the mail.
>
> Fetchmail then errors with Query status=6 (IOERR)
>
> It does not delete the message or mark it as seen.
>
> next time fetchmail logs in try's to send the message again with the same result.
>
> I can increase the amount of bytes that ASSP looks at and while this solves the issue temporary it is not a permanent solution.  ASSP was not designed to look that the whole message.
> When I increase the amount of bytes ASSP looks at the system resources go through the roof.
>
> Is it possible that Fetchmail could somehow recognise the 550 response from ASSP half way through message sending.
>
> I have enclosed log file for you to look at
>
> Hope you can help 
>
> Regards
>
> Phil 
>
> fetchmail: SMTP> MAIL FROM:<externaluser> SIZE=861742
> fetchmail: SMTP< 250 2.1.0 externaluser....Sender OK
> fetchmail: SMTP> RCPT TO:<internaluser>
> fetchmail: SMTP< 250 2.1.5 internaluser
> fetchmail: SMTP> DATA
> fetchmail: SMTP< 354 Start mail input; end with <CRLF>.<CRLF>
> fetchmail: error writing message text
> fetchmail: POP3> QUIT
> fetchmail: POP3< O07etM6Coi+kFL/DyxZouBPCFDIAEiwZWHF6pjPuZt2vAR+/F0pCL8Re3Qva/m6kfYL7EsD+Vd52
> fetchmail: MDA error while fetching from shrelay@do...@ma...main
> fetchmail: 6.3.9 querying server1 (protocol POP3) at Fri Jan 23 07:26:46 2009: poll completed
> fetchmail: Query status=6 (IOERR)
> fetchmail: normal termination, status 6
> fetchmail: 1 message for shrelay@domain at mail.domain (861742 octets).
> fetchmail: reading message shrelay@do...@ma...main:1 of 1 (861742 octets) (log message incomplete)fetchmail: error writing message text
> fetchmail: MDA error while fetching from shrelay@do...@ma...main
> fetchmail: Query status=6 (IOERR)
>   

I solved this issue by running ASSP in test mode (check option "Bayesian 
Test Mode") and prefixing the subject with something like [SPAM] (in 
option "Prepend Spam Subject"). It means ASSP won't stop mails seen as 
bayesian spam but will tag the subject. The mails can then be filtered 
out automatically by the final recipient.

Since you are downloading your mails with fetchmail, you can't simply 
reject them because it is too late to do that. The purpose in rejecting 
them is to leave the bad mails on the hand of the one that accepted them 
in the first place. But in your case, you already have accepted them and 
you have to deal with them...

If your server is also accepting external mails through direct 
connections via port 25 and you want ASSP to reject those mails, then 
you have to open some kind of backdoor on your mail server to let 
fetchmail inject the mails it is reading. Be sure to properly firewall 
that port to prevent anybody but fetchmail to inject mails.

By the way, I don't advise to let ASSP delete or reject blindly bayesian 
spams because, as you noticed, it doesn't take the whole mail into 
account and, in general, it does a poor job at filtering spams. It tends 
to have a high rate of false positives and you will loose good e-mails 
if you don't keep an eye on what it does.

Frederic

Re: [fetchmail-devel] POP3 UIDL behavior

[Matthias A. <mat...@gm...>]

Matthias Andree schrieb am 2009-01-23:

> > 2) Everything is duplicated in the scratchlist even for UID entries we 
> > should know about so I added a boolean flag to skips duplicates. 
> > Otherwise when the UID file is written it'll include oldlist plus 
> > duplicates in scratchlist. Is it supposed to duplicate everything into 
> > the scratchlist for some reason? This isn't a functional problem, but it 
> > does waste memory and disk space if you're thinking about large deployments.
> 
> This appears to be a side effect of your commenting out the "break;",
> since the for loop you made complete now terminates with uid == NULL.

Read that as "ctl == NULL".

-- 
Matthias Andree

Re: [fetchmail-devel] POP3 UIDL behavior

[Matthias A. <mat...@gm...>]

Seth Mattinen schrieb am 2009-01-19:

> I was working on developing fetchmail for use in a large multiuser 
> environment and came across some issues with the POP3 UID lists. I'm 
> very new to fetchmail's code, so I don't know of these things are 
> intentional or expected behavior, but they seemed "broken" to me.

Hi Seth,

you're scratching a sore spot ;-) Except for minimal streamlining and
minor fixes, the uid.c code is what we inherited from Eric S. Raymond,
who was very loathe to touch anything of it since it broke all too
easily.

A while back, Sunil and I had exchanged patches to save UIDs in one file
per account, but I didn't merge that code into fetchmail, to make
fetchmail 6.3.X as drop-in compatible with 6.2.5 so that nobody would
have excuses for not upgrading from 6.2.X to 6.3.X.

Let me beforehand state that the data structures for storing the UID
lists are very inefficient, and costly at run-time. Several hundred kept
messages bog down old computers, and several thousands bog down modern
computers. We have cascaded linear lists, and we store the file very
inefficiently on-disk (which is less of an issue today).

The whole UID issue is for reconsideration when I really start working
on 6.4.X. At least, we need efficient memory and on-disk structures and
we'd preferably save to one uid file per account, so we can do away with
all the "load unrelated host info to scratchlist" stuff. It may be
useful to store everything in some lightweight/embedded database instead
(such as TokyoCabinet, Berkeley DB, SQLite). No decisions made yet, and
maybe we need to benchmark our options here during development.

> 1) The old UID list is never loaded when the daemon starts or restarts 
> and the whole POP3 box downloaded every time for "keep" sources. My fix 
> was to comment out the break after save_str() in uid.c because as far as 
> I can tell, having that break in there causes it to never load the id 
> file and it'll self-break when the for loop hits a null anyway.

I do not observe such behaviour - which version of fetchmail are you
looking at?

As far as I can see (it's around line 218 in my current uid.c version as
of post-6.3.9 SVN, in initialize_saved_lists()), it just breaks out of
"find the correct list" (there's a for loop three lines above) AFTER it
has saved the UID.

So if commenting out that break; at line 219 fixes anything for you,
your fix is for the symptoms, but not the root cause. We should then
find the latter.

> 2) Everything is duplicated in the scratchlist even for UID entries we 
> should know about so I added a boolean flag to skips duplicates. 
> Otherwise when the UID file is written it'll include oldlist plus 
> duplicates in scratchlist. Is it supposed to duplicate everything into 
> the scratchlist for some reason? This isn't a functional problem, but it 
> does waste memory and disk space if you're thinking about large deployments.

This appears to be a side effect of your commenting out the "break;",
since the for loop you made complete now terminates with uid == NULL.

Make sure the spelling on the command line matches the one in the rcfile
exactly (including case) for a test and see if that helps.

Could you show me your configuration file (remove/mask passwords!) and
your command line? Please do not edit host names, they are crucial here.
If you're concerned about disclosing that in public, send the material
to me GnuPG encrypted off-list (my key is 0x052e7d95), but please again
without passwords.

HTH

-- 
Matthias Andree

[fetchmail-devel] [Bug #15103] Fetchmail Fails with ASSP (http://assp.sourceforge.net/)

[<ad...@be...>]

Bug #15103, was updated on 2009-Jan-23 07:36
Here is a current snapshot of the bug.

Project: Community Fetchmail
Category: None
Status: Open
Resolution: None
Bug Group: None
Priority: 5
Submitted by: darthasp
Assigned to : none
Summary: Fetchmail Fails with ASSP (http://assp.sourceforge.net/)

Details: Hi There,

I am trying to download messages from a catchall mailbox and forward them on to a mail server that sits been an ASSP installation.

The problem I am having is that ASSP only looks at the first 2000 bytes of the message in order to determine whether it is SPAM or not.

If the message that Fetchmail is sending is larger than this size and ASSP considers it to be SPAM - ASSP closes the connection with a 550: response before Fetchmail has finished copying the mail.

Fetchmail then errors with Query status=6 (IOERR)

It does not delete the message or mark it as seen.

next time fetchmail logs in try's to send the message again with the same result.

I can increase the amount of bytes that ASSP looks at and while this solves the issue temporary it is not a permanent solution.  ASSP was not designed to look that the whole message.
When I increase the amount of bytes ASSP looks at the system resources go through the roof.

Is it possible that Fetchmail could somehow recognise the 550 response from ASSP half way through message sending.

I have enclosed log file for you to look at

Hope you can help 

Regards

Phil 

fetchmail: SMTP> MAIL FROM:<externaluser> SIZE=861742
fetchmail: SMTP< 250 2.1.0 externaluser....Sender OK
fetchmail: SMTP> RCPT TO:<internaluser>
fetchmail: SMTP< 250 2.1.5 internaluser
fetchmail: SMTP> DATA
fetchmail: SMTP< 354 Start mail input; end with <CRLF>.<CRLF>
fetchmail: error writing message text
fetchmail: POP3> QUIT
fetchmail: POP3< O07etM6Coi+kFL/DyxZouBPCFDIAEiwZWHF6pjPuZt2vAR+/F0pCL8Re3Qva/m6kfYL7EsD+Vd52
fetchmail: MDA error while fetching from shrelay@do...@ma...main
fetchmail: 6.3.9 querying server1 (protocol POP3) at Fri Jan 23 07:26:46 2009: poll completed
fetchmail: Query status=6 (IOERR)
fetchmail: normal termination, status 6
fetchmail: 1 message for shrelay@domain at mail.domain (861742 octets).
fetchmail: reading message shrelay@do...@ma...main:1 of 1 (861742 octets) (log message incomplete)fetchmail: error writing message text
fetchmail: MDA error while fetching from shrelay@do...@ma...main
fetchmail: Query status=6 (IOERR)

For detailed info, follow this link:
http://developer.berlios.de/bugs/?func=detailbug&bug_id=15103&group_id=1824

[fetchmail-devel] POP3 UIDL behavior

[Seth M. <se...@ro...>]

I was working on developing fetchmail for use in a large multiuser 
environment and came across some issues with the POP3 UID lists. I'm 
very new to fetchmail's code, so I don't know of these things are 
intentional or expected behavior, but they seemed "broken" to me.

1) The old UID list is never loaded when the daemon starts or restarts 
and the whole POP3 box downloaded every time for "keep" sources. My fix 
was to comment out the break after save_str() in uid.c because as far as 
I can tell, having that break in there causes it to never load the id 
file and it'll self-break when the for loop hits a null anyway.

2) Everything is duplicated in the scratchlist even for UID entries we 
should know about so I added a boolean flag to skips duplicates. 
Otherwise when the UID file is written it'll include oldlist plus 
duplicates in scratchlist. Is it supposed to duplicate everything into 
the scratchlist for some reason? This isn't a functional problem, but it 
does waste memory and disk space if you're thinking about large deployments.

I'm still muddling through this so I can roll these changes plus some 
other weirdness I'm seeing into a patch when I'm finished, if interested.

-- 
Seth Mattinen		se...@ro...
Roller Network LLC

Re: [fetchmail-devel] [PATCH] don't copy between overlapping buffers with snprintf

[Matthias A. <mat...@gm...>]

Nico Golde schrieb am 2009-01-04:

> we are currently checking all of the Debian packages for 
> sprintf/snprintf usage that is undefined. Fetchmail is 
> copying between overlapping buffers in interface.c.

Huh, that piece of code is going to be dropped in 6.4.X for
non-portability anyways. :-)

> Attached is a patch to fix this.

Applied & scheduled to appear in 6.3.10, thanks. 

Best

-- 
Matthias Andree

[fetchmail-devel] [PATCH] don't copy between overlapping buffers with snprintf

[Nico G. <ni...@ng...>]

Hi,
we are currently checking all of the Debian packages for 
sprintf/snprintf usage that is undefined. Fetchmail is 
copying between overlapping buffers in interface.c.

Attached is a patch to fix this.

Cheers
Nico

-- 
Nico Golde - http://www.ngolde.de - ni...@ja... - GPG: 0x73647CFF
For security reasons, all text in this mail is double-rot13 encrypted.

[fetchmail-devel] [Patch #2625] Fix autotools cross-compiling support

[<ad...@be...>]

Patch #2625 has been updated. 

Project: fetchmail
Category: None
Status: Open
Submitted by: flameeyes
Assigned to : none
Summary: Fix autotools cross-compiling support

-------------------------------------------------------
For more info, visit:

http://developer.berlios.de/patch/?func=detailpatch&patch_id=2625&group_id=1824

[fetchmail-devel] New Indonesian PO file for 'fetchmail' (version 6.3.9-rc3)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'fetchmail' has been submitted
by the Indonesian team of translators.  The file is available at:

    http://translationproject.org/latest/fetchmail/id.po

(This file, 'fetchmail-6.3.9-rc3.id.po', has just now been sent to you in
a separate email.)

All other PO files for your package are available in:

    http://translationproject.org/latest/fetchmail/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    http://translationproject.org/domain/fetchmail.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

[fetchmail-devel] New Dutch PO file for 'fetchmail' (version 6.3.9-rc3)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'fetchmail' has been submitted
by the Dutch team of translators.  The file is available at:

    http://translationproject.org/latest/fetchmail/nl.po

(This file, 'fetchmail-6.3.9-rc3.nl.po', has just now been sent to you in
a separate email.)

All other PO files for your package are available in:

    http://translationproject.org/latest/fetchmail/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    http://translationproject.org/domain/fetchmail.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

[fetchmail-devel] [Feature #4396] record headers in oversized-msg warnmail

[<ad...@be...>]

Feature Request #4396, was updated on 2008-Nov-26 18:38
You can respond by visiting: 
http://developer.berlios.de/feature/?func=detailfeature&feature_id=4396&group_id=1824

Category: None
Status: Open
Priority: 5
Summary: record headers in oversized-msg warnmail

By: m-a
Date: 2008-Nov-26 18:38

Message:
Logged In: YES 
user_id=2007
Browser: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4

Bernhard Boxhorn suggests to record sender (or other
headers, perhaps Subject) in the
oversized-messages-kept/deleted mail. Should be doable
for POP3, we must have the header when we see a reject
of MAIL FROM:<foo@bar.example> SIZE=123456789 else we
wouldn't have the envelope sender, quote:

"So, this remains as feature request, when set the
--limitflush option then headers are downloaded after
realizing that the mail exceeds the limitflush size,
deletes the mail and informs the recipient (btw, why
not the sender) that sender xy...@ab... sent am mail
with X MB, this was 
bigger than $limitflush and the mail got deleted.
This would save the bandwith to download and then
reject the Mail and as well we could turn on
softbounce=yes for protecting us a bit when doing
configuration errors."

(softbounce referring to Postfix's soft_bounce feature.)

----------------------------------------------------------------------
You can respond by visiting: 
http://developer.berlios.de/feature/?func=detailfeature&feature_id=4396&group_id=1824

[fetchmail-devel] New Italian PO file for 'fetchmail' (version 6.3.9-rc3)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'fetchmail' has been submitted
by the Italian team of translators.  The file is available at:

    http://translationproject.org/latest/fetchmail/it.po

(This file, 'fetchmail-6.3.9-rc3.it.po', has just now been sent to you in
a separate email.)

All other PO files for your package are available in:

    http://translationproject.org/latest/fetchmail/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    http://translationproject.org/domain/fetchmail.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

[fetchmail-devel] fetchmail 6.3.9 Italian Translation available

[Matthias A. <mat...@gm...>]

Greetings,

Vincenzo Campanella has provided an Italian Translation for fetchmail
6.3.9. Since I've only received it after the release, I've made a patch
from his translation, to be applied upon the unpacked tarball.

You can download the patch from:
<http://mandree.home.pages.de/fetchmail/fetchmail-6.3.9-Italian.patch.bz2>
<http://prdownload.berlios.de/fetchmail/fetchmail-6.3.9-Italian.patch.bz2>

Thanks, Vincenzo!

Best regards
Matthias

[fetchmail-devel] New Italian PO file for 'fetchmail' (version 6.3.9-rc3)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'fetchmail' has been submitted
by the Italian team of translators.  The file is available at:

    http://translationproject.org/latest/fetchmail/it.po

(This file, 'fetchmail-6.3.9-rc3.it.po', has just now been sent to you in
a separate email.)

All other PO files for your package are available in:

    http://translationproject.org/latest/fetchmail/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    http://translationproject.org/domain/fetchmail.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

[fetchmail-devel] fetchmail 6.3.9 release available, with security fixes

[Matthias A. <mat...@gm...>]

Greetings,

I am announcing the release of fetchmail 6.3.9. This new stable version
of fetchmail fixes two security and two critical bugs since the previous
version, and makes various other changes, see below for details.

The software is available from:
<http://developer.berlios.de/project/showfiles.php?group_id=1824>

The fetchmail home pages are:
<http://www.fetchmail.info/>  or  <http://fetchmail.berlios.de/>

These are the relevant changes in 6.3.9 since 6.3.8;
unless otherwise noted, changes to this release were made by Matthias Andree:

# ADVANCE WARNING OF FEATURES TO BE REMOVED OR CHANGED IN FUTURE VERSIONS
(There are no plans to remove these features from a 6.3.X release, but they may
be removed from a 6.4.0 or newer release.)
* The MX and host alias DNS lookups that fetchmail performs in multidrop mode
  are based on assumptions that are rarely met in practice, somewhat defective,
  deprecated and may be removed from a future fetchmail version.  They have
  never supported IPv6 (including IPv6-mapped IPv4).
  Non-DNS based alias keywords such as "aka" will remain in fetchmail.
* The monitor and interface options may be removed from a future fetchmail
  version as they are not reasonably portable.
* POP2 is obsolete, support will be removed from a future fetchmail version.
* RPOP is obsolete, support will be removed from a future fetchmail release.
* --sslcertck will become a default setting in a future fetchmail version.
* The multidrop To/Cc guessing code along with the fragile duplicate suppressor
  is deprecated and may be removed from a future release.
* The "envelope Received" option may be removed from a future release, because
  the Received header was never meant to be machine-readable, the format varies
  widely, and various other differences in behavior make parsing Received an
  unreliable undertaking. The envelope option as such will remain though, in
  order to support Delivered-To, X-Envelope-To, X-Original-To and similar.
  See also <http://home.pages.de/~mandree/mail/multidrop>.
* The --enable-fallback (fall back to MDA if MTA unavailable) will be removed
  from a future fetchmail release, because it makes fetchmail's behavior
  inconsistent and confusing.
* The "protocol auto" default inside fetchmail may be removed from a future
  fetchmail release. Explicit configuration of the protocol is recommended.
* Kerberos IV support may be removed from a future fetchmail release.
* SIGHUP wakeup support may be removed from a future fetchmail release and
  cause fetchmail to terminate - it was broken for many years.
* Support for operating systems that are not sufficiently POSIX compliant may be
  removed or operation on such systems may be suboptimal for future releases.

# SECURITY AND CRITICAL BUG FIXES:
* CVE-2007-4565: Denial of service: When fetchmail tries to inject a warning
  message it created itself, and the message is refused by the SMTP listener,
  fetchmail dereferences a NULL pointer and crashes. Report & fix by Earl Chew.
  Note while this is theoretically a remote denial of service attack vector,
  fetchmail by default talks SMTP to the localhost, so the overall risk is
  rather low.
  This bug was apparently introduced on 1998-11-27 when the bouncemail facility
  was modularized. The bug then made its appearance in fetchmail release 4.6.8.
  See also fetchmail-SA-2007-02.txt.
* CVE-2008-2711: Denial of service: When fetchmail logs data blobs
  (for instance, a To: header in -v -v verbose mode) in excess of 2048
  bytes, it will crash, because it hands an uninitialized argument
  pointer (not the format string though) to vsnprintf and reads a
  random memory location (it calls va_arg() too often without
  resetting it with va_start()). Based on a patch (BerliOS patch #2492)
  by Petr Uzel, fixes Novell Bug #354291.
  Note 6.3.9-rc1 did not completely fix this issue, so it was redrawn a few 
  hours after its release.
  See also fetchmail-SA-2008-01.txt.
* When expunging, mark the right messages as seen to avoid message loss in "keep
  flush" configurations.  Workaround for previous versions: "expunge 0".
  Report and patch by Alexander Cherepanov - thanks a lot, Berlios Bug #11797,
  "imap_mark_seen doesn't consider expunged messages".
* SSL fix: close memory leak when SSL connection fails; fetchmail used to forget
  calling SSL_free() on the SSL context, leaking in excess of 500 kB RAM on a
  x86_64 system per failed SSL connection attempt.
  Bug reported and patch provided by Seiichi Ikarashi, Fujitsu.

# BUG FIXES:
* The configure script will additionally check for 'dn_skipname', to fix build
  failures with µClibc. The new check still recognizes the resolver libraries on
  Ubuntu 7.04, openSUSE 10.2, Solaris 8, NetBSD 4.0_BETA2 and FreeBSD 6.2.
  Fixes Gentoo bug #134187.
  NOTE: this is a bit of a hack, since we twist the HAVE_RES_SEARCH result, but
  res_search() and dn_skipname() are only used together and scheduled for
  removal in future versions, so this is probably fine.
* No longer complain about invalid sslproto "" when POP3 CAPA probe fails.
  Fixes Debian Bug#421446 (Holger Leskien), Novell Bug #247233 (Jon Nelson).
  Thanks to Matthias Strauß for a configuration to reproduce the issue.
* Allow .fetchmailrc and .fetchids to be symlinks, as the manpage does not
  document they aren't allowed - fixes Debian Bug #452907 (Roger Leigh).
  TOCTOU race persists.
* fetchmailconf quotes mailbox (folder) names when writing the configuration.
  Fixes BerliOS Bug #13207 (reported + fix suggested by Terry Brown).
* Only print "Deleting fetchids file" if there actually is one.
  Fixes Debian Bug#374514, reported by Dan Jacobson.
* SSL fix: check and report if SSL_set_fd fails.

# CHANGES:
* autoconf 2.60 is now required to build fetchmail; it uses
  AC_USE_SYSTEM_EXTENSIONS to replace AC_AIX, AC_MINIX, and the like.
* Removed dead FETCHMAIL_DEBUG code from fetchmail.h that was disabled by 
  default with no switches in configure to enable it. However, the macro would 
  have been prone to a symlink attack. Found by Nico Golde.
* Removed dead FORCE_STUFFING code from socket.c that was disabled by default
  with no switches in configure to enable it.
* Include the typedef for int16 in the #ifndef _AIX in smbencrypt.c (Peter
  O'Gorman)
* Correct check for u_int32_t in configure.ac (seems to be typedef'ed in
  namser.h on some platforms.) (Peter O'Gorman)
* In configure.ac change all CPFLAGS to CPPFLAGS, CEFLAGS to CFLAGS and LDEFLAGS
  to LDFLAGS otherwise the results of some tests (additional -L and -I flags) do
  not get used for later tests causing incorrect configure results. Makefile.am
  was also changed to reflect this. (Peter O'Gorman)
* m4/gethostbyname_r.m4 does AC_TRY_COMPILE, which unfortunately can pass even
  if there is no gethostbyname_r. Changed to AC_TRY_LINK. (Peter O'Gorman)
* Revise getnameinfo check to ensure NULL is defined and the result is properly
  evaluated, to avoid bogus results on for instance FreeBSD and redefinitions of
  NI_* at compile time. (Matthias Andree).
* __attribute__ ((unused)) is a gccism, removed from libesmtp/gethostbyname.c.
  (Peter O'Gorman)
* In KAME/getnameinfo.c it's best to use the correct argument to inet_ntoa.
  (Peter O'Gorman)
* In verbose mode, log if --check mode is enabled.
* Add sslcommonname option (rcfile and commandline) as a way to work around
  misconfigured upstream SSL servers that use the wrong certificate name. It
  specifies which CommonName fetchmail expects and logs. (Daniel Richard G.)
* Changed CRLF to LF line endings in contrib/delete-later (reporter: Petr Uzel)
* SSL change: enable all workarounds with SSL_CTX_set_options(ctx,SSL_OP_ALL)
* All translations have been re-enabled, in an attempt to rekindle translator or
  user interest.

# DOCUMENTATION:
* Add fetchmail-SA-2007-02.txt and fetchmail-SA-2008-01.txt.
* Re-add two lines to the manual page that had accidentally become comments 
  to nroff. One was part of the --sslproto documentation, and one in the 
  "Awakening the background daemon" section.
* The manual page no longer asserts that .fetchids were for exclusive POP3 use, 
  since it is planned to use the file with IMAP4 later.
* Add grammar fixes from Dan Jacobson to fetchmail.man. Debian Bug #461642.
* The manual page now mentions that user descriptions need to come before user
  options. Reported by Francensco Pontortì, to fix Debian Bug #467010.
* The manual page no longer hints that multi-user declarations per server were
  only useful in daemon mode running as root, to avoid hinting people to doing
  that.
* Several manual page rcfile examples now include "ssl".
* The manual page hints that option arguments beginning with numbers can be
  enclosed in quotes.
* The manual page now mentions that the --logfile must already exist before
  fetchmail is run.
* The FAQ now recommends (#I9) not to use Google Mail for their disregard to the
  protocols they claim to support.
* Documentation and program output now /consistently/ claim that the rcfile must
  not have more than 0700 (u=rwx,g=,o=) permissions, but fetchmail will still
  silently accept additional g=x permissions for compatibility with previous
  6.2.X and 6.3.X versions.
  Inconsistency (program 0710, manpage 0600) reported by Petr Uzel.
* The --logfile documentation is now clearer about requiring detached daemon 
  mode.

# TRANSLATION UPDATES AND ADDITIONS (ordered by language name):
* [sq]    Albanian (Besnik Bleta)
* [zh_CN] Chinese, simplified (Ji Zheng-Yu)
* [cs]    Czech (Petr Pisar)
* [da]    Danish (Byrial Ole Jensen) - outdated, but newer than in 6.3.8
* [nl]    Dutch (Tony Vroon, Benno Schulenberg)
* [en_GB] English, British
* [fi]    Finnish (Lauri Nurmi)
* [de]    German
* [id]    Indonesian (Andhika Padmawan)
* [ja]    Japanese (Takeshi Hamasaki)
* [pl]    Polish (Jakub Bogusz)
* [ru]    Russian (Pavel Maryanov)
* [es]    Spanish (Javier Fernández-Sanguino Peña, Matthias Andree)
* [tr]    Turkish (Engin Gündüz) - outdated, but newer than in 6.3.8
* [vi]    Vietnamese (Clytie Siddall)

-- 
Matthias Andree

[fetchmail-devel] fetchmail 6.3.9-rc3 release candidate

[Matthias A. <mat...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings,

I have uploaded a new fetchmail 6.3.9 release candidate to the usual
download location: <http://home.pages.de/~mandree/fetchmail/>.

It contains minor fixes that have accumulated since -rc2 (including a
fix that I'd promised to Petr Uzel, but made only after -rc2), and
translation updates - details below.

I've announced -rc3 to the translation project in order to collect
translations, and hope to release 6.3.9 later this month.

Happy fetching
Matthias

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Changes in fetchmail 6.3.9-rc3 from -rc2 (2008-11-04):

# BUGFIXES
* Only print "Deleting fetchids file" if there actually is one.
  Fixes Debian Bug#374514, reported by Dan Jacobson.
* SSL fix: check and report if SSL_set_fd fails.

# CHANGES:
* Changed CRLF to LF line endings in contrib/delete-later (reporter: Petr Uzel)
* SSL change: enable all workarounds with SSL_CTX_set_options(ctx,SSL_OP_ALL)
* All translations have been re-enabled, in an attempt to rekindle translator or
  user interest.

# DOCUMENTATION:
* Documentation and program output now /consistently/ claim that the rcfile must
  not have more than 0700 (u=rwx,g=,o=) permissions, but fetchmail will still
  silently accept additional g=x permissions for compatibility with previous
  6.2.X and 6.3.X versions.
  Inconsistency (program 0710, manpage 0600) reported by Petr Uzel.
* The --logfile documentation is now clearer about requiring detached daemon 
  mode.

# TRANSLATION UPDATES AND ADDITIONS (ordered by language name):
* [sq]    Albanian (Besnik Bleta)
* [zh_CN] Chinese, simplified (Ji Zheng-Yu)
* [cs]    Czech (Petr Pisar)
* [da]    Danish (Byrial Ole Jensen) - outdated, but newer than in 6.3.8
* [nl]    Dutch (Tony Vroon, Benno Schulenberg)
* [en_GB] English, British
* [fi]    Finnish (Lauri Nurmi)
* [de]    German
* [id]    Indonesian (Andhika Padmawan)
* [ja]    Japanese (Takeshi Hamasaki)
* [pl]    Polish (Jakub Bogusz)
* [ru]    Russian (Pavel Maryanov)
* [es]    Spanish (Javier Fernández-Sanguino Peña, Matthias Andree)
* [tr]    Turkish (Engin Gündüz) - outdated, but newer than in 6.3.8
* [vi]    Vietnamese (Clytie Siddall)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkkQYXIACgkQvmGDOQUufZVkIwCfdrx21btsTNF5PcCiMH3FuvEw
BxIAoJXprJPu7dLy100uznMhfM/UChoS
=keCZ
-----END PGP SIGNATURE-----

[fetchmail-devel] [Feature #4318] add --forceRETR option

[<ad...@be...>]

Feature Request #4318, was updated on 2008-Sep-22 08:58
You can respond by visiting: 
http://developer.berlios.de/feature/?func=detailfeature&feature_id=4318&group_id=1824

Category: None
Status: Open
Priority: 5
Summary: add --forceRETR option

By: jas67
Date: 2008-Sep-22 08:58

Message:
Logged In: YES 
user_id=48008
Browser: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1

Comcast is still using Maillenium, but no longer
displays the banner that triggers fetchmail to force
use of RETR instead of TOP.

I would like to not use --fetchall, but still force RETR.

I have added this to my own copy of fetchmail, and am
using it.     Here is a link to my patch to the
production 6.3.8 code:

http://gdx.dynalias.com/fetchmail_comcast_patch

I would love to have this feature added, so that I do
not need to patch future versions.    Plus, I'm sure
there are lots of comcast users out there wondering why
they are no getting truncated attachments.

Thanks,
Jay


----------------------------------------------------------------------
You can respond by visiting: 
http://developer.berlios.de/feature/?func=detailfeature&feature_id=4318&group_id=1824

Re: [fetchmail-devel] Merge MAPI code things

[Yangyan L. <yan...@gm...>]

Hi Matthias,

Thank you for your warmhearted reply!

> congratulations on finishing the project. I haven't yet reviewed your code;
> how about your availability for collaboration on code refinements should I
> deem them helpful - could you work on that if issues arise during the merge?
I've tried to limit the change made to fetchmail's original code as
few as possible. So I think it won't be too hard and I am confident
of resolving the issues:-)

>
>
> With respect to the merge, the fetchmail trunk is currently out of date,
> unmaintained and disabled.
>
> My tentative roadmap is as follows (but not fixed, I welcome any
> suggestions for improvement)
>
> - release 6.3.9 from BRANCH_6-3.
>
>  Could you check (dry-run) if merging the BRANCH_6-3 changes since you
>  copied that branch when forking BRANCH_MAPI is possible without major
>  effort? If so, I'd appreciate if you could synch up with 6.3.9 before we
>  try the merge-to-trunk effort.
BRANCH_MAPI was synchronized up with BRANCH_6-3 now.

>
> - merge BRANCH_6-3 into trunk and dub it 6.4.X for the nonce
>  (this may mean to delete trunk and recreate it from BRANCH_6-3,
>   and redo changes from the BRANCH_6-3 branch point to the point where
>   trunk fell unmaintained - I'll see)
>
> - The manual page should at least contain pointers to README.mapi where
>  adequate. Something like "--protocol mapi" (fill in whatever you chose):
>  OpenChange support, see README.mapi for details.
Could you have a look at the README.mapi and suggest which way is
better: merge it into manual or add some pointers to the README.mapi
at appropriate places in the manual?

>
> - merge MAPI support into trunk, perhaps with code revisions as I review
>  code.
>
> - perhaps clean up trunk, open ends for trunk (not all of them are needed
>  for the merge):
>
>  + sink.c is a mess. It does something like RTTI and under-the-hood
>    behaviour switching for SMTP, LMTP and MDA. Now, with MAPI as fourth
>    sink => eek. It needs abstraction, much like the "--protocol" feature.
My code did not touch sink.c by now. If the abstraction will take
place, I will handle the MAPI part.

>
>  + protocol vs. authenticator messup. KPOP (if it deserves to live) and
>    APOP are by no means protocols, but are just POP3 authenticators.
>    Probably unrelated to your work.
>
>  + SSL configuration is an intransparent mess, hard to understand, with
>    several "automatic"s under the hood, that I'd like to clean up.
>    Requires changing rcfile format.
>
>    Does MAPI require SSL support in fetchmail?
>    If not, it's independent of your work.
SSL is not needed by MAPI. I've write code to bypass the SSL settings
if users configure SSL for MAPI.

Best regards!
Yangyan

Re: [fetchmail-devel] documentation inconsistency - fetchmailrc permissions

[Matthias A. <mat...@gm...>]

Petr Uzel schrieb:

[rcfile modes]
> it seems that this change is missing in 6.3.9-rc2 - is it intentional?

Indeed it's missing from -rc2; I made this change 6 days after tagging
-rc2. The relevant change will still be part of 6.3.9. Sorry.

Should I issue -rc3 or just release 6.3.9? Any opinions?

Best regards,

-- 
Matthias Andree

Life is what happens to you while you're busy making other plans.
  						-- John Lennon, 1980

Re: [fetchmail-devel] documentation inconsistency - fetchmailrc permissions

[Petr U. <pet...@su...>]

On Monday 30 of June 2008 16:33:25 you wrote:
> > > I'm not sure which maximum permissions are correct - whether 600 or
> > > 710. (600 make better sense to me as I do not see any reason why
> > > fetchmailrc should have execute permission).
>
> > 6.3.9 will document 0700 but silently permit g+x for the nonce.

Hi,
it seems that this change is missing in 6.3.9-rc2 - is it intentional?

Thanks,


-- 
Best regards / s pozdravem

Petr Uzel, Packages maintainer
---------------------------------------------------------------------
SUSE LINUX, s.r.o.                          e-mail: pet...@su...
Lihovarská 1060/12                          tel: +420 284 028 964
190 00 Prague 9                             fax: +420 284 028 951
Czech Republic                              http://www.suse.cz   

Re: [fetchmail-devel] Merge MAPI code things

[Matthias A. <mat...@gm...>]

Yangyan Li schrieb:
> Hi Matthias,
> 
> I've finished the GSoC project "add MAPI support  for fetchmail, what
> can I do to merge/help merge it into fetchmail trunk?
> 
> I didn't update fetchmail's manual now, I write a separate
> ReadMe.mapi. I will merge the ReadMe.mapi into fetchmail's manual
> after I have some feedback on both the MAPI plugin and how to write
> the new manual for easy understanding.
> 
> I am always welcoming for opinions.

Hi Yangyan,

congratulations on finishing the project. I haven't yet reviewed your code;
how about your availability for collaboration on code refinements should I
deem them helpful - could you work on that if issues arise during the merge?


With respect to the merge, the fetchmail trunk is currently out of date,
unmaintained and disabled.

My tentative roadmap is as follows (but not fixed, I welcome any
suggestions for improvement)

- release 6.3.9 from BRANCH_6-3.

  Could you check (dry-run) if merging the BRANCH_6-3 changes since you
  copied that branch when forking BRANCH_MAPI is possible without major
  effort? If so, I'd appreciate if you could synch up with 6.3.9 before we
  try the merge-to-trunk effort.

- merge BRANCH_6-3 into trunk and dub it 6.4.X for the nonce
  (this may mean to delete trunk and recreate it from BRANCH_6-3,
   and redo changes from the BRANCH_6-3 branch point to the point where
   trunk fell unmaintained - I'll see)

- The manual page should at least contain pointers to README.mapi where
  adequate. Something like "--protocol mapi" (fill in whatever you chose):
  OpenChange support, see README.mapi for details.

- merge MAPI support into trunk, perhaps with code revisions as I review
  code.

- perhaps clean up trunk, open ends for trunk (not all of them are needed
  for the merge):

  + sink.c is a mess. It does something like RTTI and under-the-hood
    behaviour switching for SMTP, LMTP and MDA. Now, with MAPI as fourth
    sink => eek. It needs abstraction, much like the "--protocol" feature.

  + protocol vs. authenticator messup. KPOP (if it deserves to live) and
    APOP are by no means protocols, but are just POP3 authenticators.
    Probably unrelated to your work.

  + SSL configuration is an intransparent mess, hard to understand, with
    several "automatic"s under the hood, that I'd like to clean up.
    Requires changing rcfile format.

    Does MAPI require SSL support in fetchmail?
    If not, it's independent of your work.

Feel free to ask further questions, make suggestions, whatever, and many
thanks for your work!

Best regards

-- 
Matthias Andree

[fetchmail-devel] Merge MAPI code things

[Yangyan L. <yan...@gm...>]

Hi Matthias,

I've finished the GSoC project "add MAPI support  for fetchmail, what
can I do to merge/help merge it into fetchmail trunk?

I didn't update fetchmail's manual now, I write a separate
ReadMe.mapi. I will merge the ReadMe.mapi into fetchmail's manual
after I have some feedback on both the MAPI plugin and how to write
the new manual for easy understanding.

I am always welcoming for opinions.

Best regards!
Yangyan

[fetchmail-devel] [Feature #4277] COMSAT-support in fetchmail

[<ad...@be...>]

Feature Request #4277, was updated on 2008-Aug-22 10:15
You can respond by visiting: 
http://developer.berlios.de/feature/?func=detailfeature&feature_id=4277&group_id=1824

Category: None
Status: Open
Priority: 5
Summary: COMSAT-support in fetchmail 

By: jhagg
Date: 2008-Aug-22 10:15

Message:
Logged In: YES 
user_id=47514
Browser: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.1) Gecko/2008071420 Iceweasel/3.0.1 (Debian-3.0.1-1)

This would really be nice to have.

I have procmail to do final delivery on my mail server,
and it sends
a COMSAT message to my desktop (where I read my mail)
which runs fetchmail against the mail server
to download my mail locally. (Yes, I want to download
my mail, not store
it on the server. :-)

I know that fetchmail can be awakened by SIGUSR1 and I
have written
a short perl script that listens on the COMSAT port and
kicks
fetchmail whenever a new mail arrives.

This makes e-mail delivery almost instantenous, I don't
have to wait
for fetchmail to time out.

So, would it be possible to make fetchmail listen on
the COMSAT port directly?
Then I would'nt need an extra process to manage. :-)



----------------------------------------------------------------------
You can respond by visiting: 
http://developer.berlios.de/feature/?func=detailfeature&feature_id=4277&group_id=1824

[fetchmail-devel] [Bug #14257] Incorrect parsing of fetchmailrc

[<ad...@be...>]

Bug #14257, was updated on 2008-Jul-23 17:07
Here is a current snapshot of the bug.

Project: Community Fetchmail
Category: None
Status: Open
Resolution: None
Bug Group: None
Priority: 5
Submitted by: juanvaqueroponc
Assigned to : none
Summary: Incorrect parsing of fetchmailrc

Details: The username in question is being parsed as 'sername "aaaaaaaaaaaaa@aaaaaaaaaaaaa'

It is quite difficult to trigger this bug.
If I change some lines the bug does not trigger.
I have got to the attached file that triggers the bug, but
if I remove the last line the same bug seems not to trigger.


For detailed info, follow this link:
http://developer.berlios.de/bugs/?func=detailbug&bug_id=14257&group_id=1824