fetchmail-users — User questions and discussion

Re: [fetchmail-users] IMAP problem (fetchmail 6.3.1-pre1)

[Matthias A. <mat...@gm...>]

Jason White <jas...@in...> writes:

> Dec 16 18:40:03 jdc fetchmail[13415]: IMAP> A0006 FETCH 1 RFC822.HEADER 
> Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< * 1 FETCH (UID 16
> RFC822.HEADER {1360} 

Well, this tells fetchmail that 1360 bytes (octets) will
follow. Since this works for me with a Dovecot upstream server, I wonder
if it's really fetchmail's fault or perhaps SurgeMail's.

Could I possibly get a test mail account on that server to find out
which software is wrong?  If so, please send the IMAP server, mail
address (where I would send test messages), account and password or
other credentials off-list. The X-PGP-Key header mentions my GnuPG key
so encrypted communication is possible (and should be used).

If that is not possible, can you change your password, run ethereal,
tethereal or tcpdump -s5000 to capture to a file (the tcpdump filter
rule would be "host mail.internode.on.net and port 143"), run fetchmail
once, then change your password again and mail me the resulting tcpdump
or tethereal capture file? It contains the password, hence the hassle
with changing it. Looking at the tcpdump/tethereal dump should also give
us an idea which of the two sides miscounted the message size.

> Dec 16 18:40:33 jdc fetchmail[13415]: timeout after 30 seconds waiting for server mail.internode.on.net. 
> Dec 16 18:40:33 jdc fetchmail[13415]: socket error while fetching from jas...@ma... 

Thank you for retrying with 6.3.1-pre1 and reporting your problem.

-- 
Matthias Andree

[fetchmail-users] IMAP problem (fetchmail 6.3.1-pre1)

[Jason W. <jas...@in...>]

I initially raised this question on the fetchmail-friends list, but was wisely
advised to test with 6.3.1-pre1 and bring the problem here.

I am quite confident this is a protocol issue, but I don't know IMAP and
Fetchmail well enough to debug it. For some reason, it appears that Fetchmail
doesn't detect the final "OK" message from the server and simply times out
after retrieving the headers. Using an MUA (e.g., muttng), I can successfully
retrieve messages via IMAP from this server. For the moment, I have configured
fetchmail to use pop3 instead as a work-around so I can read mail easily from
this account.

From syslog:

Dec 16 18:40:02 jdc fetchmail[13415]: 6.3.1-pre1 querying mail.internode.on.net (protocol IMAP) at Fri Dec 16 18:40:02 2005: poll started 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< * OK internode.on.net bld-mail04 Ready 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP> A0001 CAPABILITY 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< * CAPABILITY IMAP4 IMAP4REV1 NAMESPACE QUOTA UIDPLUS IDLE XFLDDATA SURGEMAIL 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< A0001 OK CAPABILITY completed 
Dec 16 18:40:03 jdc fetchmail[13415]: Protocol identified as IMAP4 rev 1 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP> A0002 LOGIN "jasonjgw" * 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< A0002 OK LOGIN completed 
Dec 16 18:40:03 jdc fetchmail[13415]: selecting or re-polling default folder 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP> A0003 SELECT "INBOX" 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< * 1 EXISTS 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< * 0 RECENT 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< * OK [UIDVALIDITY 1134522857] UIDs valid 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< * FLAGS (\Answered \Flagged \Deleted \Draft \Seen) 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Draft \Seen)] Limited 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< A0003 OK [READ-WRITE] SELECT completed 
Dec 16 18:40:03 jdc fetchmail[13415]: 1 message waiting after first poll 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP> A0004 EXPUNGE 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< A0004 OK EXPUNGE completed 
Dec 16 18:40:03 jdc fetchmail[13415]: 1 message waiting after expunge 
Dec 16 18:40:03 jdc fetchmail[13415]: 1 message for jasonjgw at mail.internode.on.net. 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP> A0005 FETCH 1 RFC822.SIZE 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< * 1 FETCH (UID 16 RFC822.SIZE 1447) 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< A0005 OK FETCH completed 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP> A0006 FETCH 1 RFC822.HEADER 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< * 1 FETCH (UID 16 RFC822.HEADER {1360} 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< Received: from beatrice.nipl.net (unverified [62.94.93.142])  
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< ^Iby mail.internode.on.net (SurgeMail 3.2f) with ESMTP id 4426396  
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< ^Ifor <jas...@in...>; Fri, 16 Dec 2005 18:09:13 +1030 (CDT) 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< Return-Path: <ja...@ni...> 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< Received: from localhost (localhost.localdomain [127.0.0.1]) 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< ^Iby beatrice.nipl.net (Postfix) with ESMTP id 0336EE8624 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< ^Ifor <jas...@in...>; Fri, 16 Dec 2005 07:39:04 +0000 (UTC) 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< Received: from beatrice.nipl.net ([127.0.0.1]) 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< ^Iby localhost (beatrice.nipl.net [127.0.0.1]) (amavisd-new, port 10024) 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< ^Iwith ESMTP id 01175-07 for <jas...@in...>; 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< ^IFri, 16 Dec 2005 07:39:01 +0000 (UTC) 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< Received: by beatrice.nipl.net (Postfix, from userid 1010) 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< ^Iid 79CEBE8644; Fri, 16 Dec 2005 07:39:01 +0000 (UTC) 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< Date: Fri, 16 Dec 2005 18:39:00 +1100 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< From: Jason White <ja...@ni...> 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< To: jas...@in... 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< Subject: Fetchmail test for 6.3.1 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< Message-ID: <200...@ni...> 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< Mime-Version: 1.0 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< Content-Type: text/plain; charset=us-ascii 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< Content-Disposition: inline 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< User-Agent: Mutt/1.5.9i 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at nipl.net 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< X-Rcpt-To: <jas...@in...> 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< X-Vpipe: Scanner said clean (/usr/local/clamav/sbin/vscand-clamav) 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< X-NotAscii: charset=us-ascii 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< X-IP-stats: Incoming Last 0, First 0, in=1, out=0, spam=0 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< X-External-IP: 62.94.93.142 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP<  
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< ) 
Dec 16 18:40:03 jdc fetchmail[13415]: IMAP< A0006 OK FETCH completed 
Dec 16 18:40:33 jdc fetchmail[13415]: timeout after 30 seconds waiting for server mail.internode.on.net. 
Dec 16 18:40:33 jdc fetchmail[13415]: socket error while fetching from jas...@ma... 
Dec 16 18:40:33 jdc fetchmail[13415]: 6.3.1-pre1 querying mail.internode.on.net (protocol IMAP) at Fri Dec 16 18:40:33 2005: poll completed 
Dec 16 18:40:33 jdc fetchmail[13415]: Query status=2 (SOCKET) 
Dec 16 18:40:33 jdc fetchmail[13415]: Deleting fetchids file. 
Dec 16 18:40:33 jdc fetchmail[13415]: sleeping at Fri Dec 16 18:40:33 2005 

Configuration:

# global options
set daemon 60
set syslog
# Server options (mail.internode.on.net)
poll mail.internode.on.net
protocol imap
timeout 30
# User options (internode.on.net)
username jasonjgw is jason here
password secret
fetchall

Re: [fetchmail-users] comcast.net pop3 broken after 5.9.12 upgraded

[Rob M. <rob...@gm...>]

On 15/12/05, Denis Hainsworth <de...@al...> wrote:
> hello all,
> fetchmail has been awesome and worked flawlessly for years for me, thats
> why this is so odd.  my system is redhat fedora core 3ish (i upgrade
> packages as I need to) anyway I had an rpm of fetchmail 5.9.12 installed
> which was working fine.  I did something dumb with postfix and stopped
> receiving mail, while troubleshooting it I upgraded fetchmail to
> fetchmail-6.2.5-7.fc4.1.i386.rpm.  Once I found my problem with postfix
> I verified that fetchmail could connect to my univeristy IMAP account
> and it started downloading my mail.  Great.  Several days later I
> realized I was no longer receiving mail from my comcast.new pop3 acocunt
> which is AFTER my IMAP account in my fetchmailrc file.
<---SNIP--->
> -- fetchmailrc file --
>
> poll mail.comcast.net with proto pop3:
>   user xxxxxxxxx is xxxxxxxx here pass xxxxxxxxx options fetchall

That would appear to be less than the entire file, given your initial
statement.  What does the entire .fetchmailrc file look like?

However, once you install 6.3.x we really need to see the output of
that - it's little use providing us with diagnostic output of
everything *but* the version you're having problems with :-)

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-users] comcast.net pop3 broken after 5.9.12 upgraded

[Matthias A. <mat...@gm...>]

Denis Hainsworth <de...@al...> writes:

> fetchmail has been awesome and worked flawlessly for years for me, thats
> why this is so odd.  my system is redhat fedora core 3ish (i upgrade
> packages as I need to) anyway I had an rpm of fetchmail 5.9.12 installed
> which was working fine.  I did something dumb with postfix and stopped
> receiving mail, while troubleshooting it I upgraded fetchmail to 
> fetchmail-6.2.5-7.fc4.1.i386.rpm.  Once I found my problem with postfix
> I verified that fetchmail could connect to my univeristy IMAP account
> and it started downloading my mail.  Great.  Several days later I
> realized I was no longer receiving mail from my comcast.new pop3 acocunt
> which is AFTER my IMAP account in my fetchmailrc file.

Can you try fetchmail-6.3.0 or 6.3.1-pre1 and see if the problem
persists? There have been dozens of bugfixes since 6.2.5.

> [624] christmas /home/denis% fetchmail -v -v
> fetchmail: 5.9.13 querying mail.comcast.net (protocol POP3) at Thu, 15
> Dec 2005 16:56:18 -0500 (EST): poll started
> fetchmail: fetchmail: getaddrinfo(mail.comcast.net.pop3)
> fetchmail: 5.9.13 querying mail.comcast.net (protocol POP3) at Thu, 15
> Dec 2005 16:56:18 -0500 (EST): poll completed
> fetchmail: Query status=2 (SOCKET)
> fetchmail: Deleting fetchids file.
> fetchmail: normal termination, status 2
> fetchmail: Deleting fetchids file.

getaddrinfo is, AFAIR, only used if inet6 is enabled. Does your system
have complete IPv6 implementation, recent (i. e. no older than what was
shipped with Fedora Core 3) glibc, resolver and similar code? Is your
glibc fit for a 2.6 kernel? Pre-2.6 user space may cause trouble on 2.6
kernels, and unfortunately, Linux 2.6.X is not what other operating
systems or software programmers would call "stable".

While Linux 2.6.6 is generally speaking new enough for IPv6, if you
upgrade packages as opportunity arises, this in itself may introduce
inconsistencies that cause all kinds of problems.

This is really hard to debug, if the 6.3.0 (or 6.3.1-pre1) still fail,
perhaps strace can shed some light.

See http://home.pages.de/~mandree/fetchmail/ or
    http://fetchmail.berlios.de/
for downloads of 6.3.1-pre1 and 6.3.0, respectively.

-- 
Matthias Andree

[fetchmail-users] comcast.net pop3 broken after 5.9.12 upgraded

[Denis H. <de...@al...>]

hello all,
fetchmail has been awesome and worked flawlessly for years for me, thats
why this is so odd.  my system is redhat fedora core 3ish (i upgrade
packages as I need to) anyway I had an rpm of fetchmail 5.9.12 installed
which was working fine.  I did something dumb with postfix and stopped
receiving mail, while troubleshooting it I upgraded fetchmail to 
fetchmail-6.2.5-7.fc4.1.i386.rpm.  Once I found my problem with postfix
I verified that fetchmail could connect to my univeristy IMAP account
and it started downloading my mail.  Great.  Several days later I
realized I was no longer receiving mail from my comcast.new pop3 acocunt
which is AFTER my IMAP account in my fetchmailrc file.

After trying a number of different RPMs I grabbed my old working
fetchmail binary from a backup and low and behold everything was fine
again.  As you can see from the enclosed files there is not much of an
error.  More like a lack of anything.  I tried to capture the pop3
packets with ethereal and never saw anything after the host lookup.

So anyway I was curious if anyone had any ideas. Right now I am fine
running 5.9.12. I have tried the earliest RPM I can find which is
fetchmail-5.9.13-1.i386.rpm and it has the same problems.  Output from
both the working and non-working versions is included below.

-- fetchmailrc file --

poll mail.comcast.net with proto pop3:
  user xxxxxxxxx is xxxxxxxx here pass xxxxxxxxx options fetchall


-- output of fetchmail -v -v version 5.9.12

[620] christmas /home/denis% fetchmail -v -v
fetchmail: 5.9.12 querying mail.comcast.net (protocol POP3) at Thu, 15
Dec 2005 16:54:21 -0500 (EST): poll started
fetchmail: POP3< +OK (rwcrpxc60) Maillennium POP3/PROXY server #176
fetchmail: POP3> USER xxxxxxxxxx
fetchmail: POP3< +OK
fetchmail: POP3> PASS 
fetchmail: POP3< +OK ready
fetchmail: selecting or re-polling default folder
fetchmail: POP3> STAT
fetchmail: POP3< +OK 0 0
fetchmail: No mail for xxxxxxxxx at mail.comcast.net
fetchmail: POP3> QUIT
fetchmail: POP3< +OK comcast.net
fetchmail: 5.9.12 querying mail.comcast.net (protocol POP3) at Thu, 15
Dec 2005 16:54:24 -0500 (EST): poll completed
fetchmail: not swapping UID lists, no UIDs seen this query
fetchmail: Query status=1 (NOMAIL)
fetchmail: Deleting fetchids file.
fetchmail: normal termination, status 1
fetchmail: Deleting fetchids file.

-- output of fetchmail -v -v version 5.9.13

[624] christmas /home/denis% fetchmail -v -v
fetchmail: 5.9.13 querying mail.comcast.net (protocol POP3) at Thu, 15
Dec 2005 16:56:18 -0500 (EST): poll started
fetchmail: fetchmail: getaddrinfo(mail.comcast.net.pop3)
fetchmail: 5.9.13 querying mail.comcast.net (protocol POP3) at Thu, 15
Dec 2005 16:56:18 -0500 (EST): poll completed
fetchmail: Query status=2 (SOCKET)
fetchmail: Deleting fetchids file.
fetchmail: normal termination, status 2
fetchmail: Deleting fetchids file.

-- output of fetchmail -V version 5.9.12

[637] christmas /home/denis% fetchmail -V
This is fetchmail release 5.9.12
Linux christmas.auspice.net 2.6.6 #7 Sun Jun 27 15:44:05 EDT 2004 i686
i686 i386 GNU/Linux
Taking options from command line and /home/denis/.fetchmailrc
Idfile is /home/denis/.fetchids
Fetchmail will forward misaddressed multidrop messages to denis.
Options for retrieving from den...@ma...:
  True name of server is mail.comcast.net.
  Protocol is POP3.
  All available authentication methods will be tried.
  Server nonresponse timeout is 300 seconds (default).
  Default mailbox selected.
  All messages will be retrieved (--all on).
  Fetched messages will not be kept on the server (--keep off).
  Old messages will not be flushed before message retrieval (--flush
off).
  Rewrite of server-local addresses is enabled (--norewrite off).
  Carriage-return stripping is disabled (stripcr off).
  Carriage-return forcing is disabled (forcecr off).
  Interpretation of Content-Transfer-Encoding is enabled (pass8bits
off).
  MIME decoding is disabled (mimedecode off).
  Idle after poll is disabled (idle off).
  Nonempty Status lines will be kept (dropstatus off)
  Delivered-To lines will be kept (dropdelivered off)
  Messages will be SMTP-forwarded to: localhost (default)
  Recognized listener spam block responses are: 571 550 501 554
  Single-drop mode: 1 local name(s) recognized.
  No UIDs saved from this host.


-- output of fetchmail -V version 5.9.13

[635] christmas /home/denis% fetchmail -V
This is fetchmail release 5.9.13+INET6
Fallback MDA: (none)
Linux christmas.auspice.net 2.6.6 #7 Sun Jun 27 15:44:05 EDT 2004 i686
i686 i386 GNU/Linux
Taking options from command line and /home/denis/.fetchmailrc
Idfile is /home/denis/.fetchids
Fetchmail will forward misaddressed multidrop messages to denis.
Options for retrieving from den...@ma...:
  True name of server is mail.comcast.net.
  Protocol is POP3.
  All available authentication methods will be tried.
  Server nonresponse timeout is 300 seconds (default).
  Default mailbox selected.
  All messages will be retrieved (--all on).
  Fetched messages will not be kept on the server (--keep off).
  Old messages will not be flushed before message retrieval (--flush
off).
  Rewrite of server-local addresses is enabled (--norewrite off).
  Carriage-return stripping is disabled (stripcr off).
  Carriage-return forcing is disabled (forcecr off).
  Interpretation of Content-Transfer-Encoding is enabled (pass8bits
off).
  MIME decoding is disabled (mimedecode off).
  Idle after poll is disabled (idle off).
  Nonempty Status lines will be kept (dropstatus off)
  Delivered-To lines will be kept (dropdelivered off)
  Messages will be SMTP-forwarded to: localhost (default)
  Recognized listener spam block responses are: 571 550 501 554
  Single-drop mode: 1 local name(s) recognized.
  No UIDs saved from this host.


Final bit of info is that I believe this is the info for the RPM I have
of 5.9.12

[585] christmas /mnt/mac/var/lib/rpm% rpm -qi fetchmail-5.9.12-1
--dbpath /mnt/mac/var/lib/rpm
Name        : fetchmail                    Relocations: (not relocateable)
Version     : 5.9.12                            Vendor: Eric Conspiracy Secret Labs
Release     : 1                             Build Date: Tue Jun  4 15:20:29 2002
Install Date: Sat Jun 15 15:05:26 2002      Build Host: snark.thyrsus.com
Group       : Applications/Mail             Source RPM: fetchmail-5.9.12-1.src.rpm
Size        : 752644                           License: GPL
Signature   : (none)
Packager    : Eric S. Raymond <es...@th...>
URL         : http://www.tuxedo.org/~esr/fetchmail
Summary     : Full-featured POP/IMAP mail retrieval daemon


-denis

-- 
____________________________________________________________
Denis Alan Hainsworth     | http://www.cs.brandeis.edu/~denis/  
de...@al... | "Life is just one big sad christmas."

[fetchmail-users] fetchmail 6.3.1-pre1 snapshot - please test

[Matthias A. <mat...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings,

I have released a first 6.3.1 beta quality release candidate,
dubbed fetchmail-6.3.1-pre1. It is available from
http://home.pages.de/~mandree/fetchmail/

I am calling it "beta quality" because some of the changes at the bottom
of the list require testing in multidrop and/or LMTP configurations
that I do not currently have available.

Feedback is sought, particularly from those who reported bugs against
6.3.0 and haven't yet responded if a patch they had been sent fixed
their bug.

I plan to release this as 6.3.1 next week-end.

These are the changes since 6.3.0:

>-------------------------------------------------------------------------------
* The multidrop To/Cc guessing code along with the fragile duplicate suppressor
  is deprecated and may be removed from a future release. (MA)
>-------------------------------------------------------------------------------
* Fix broken default port in POP2. Patch by Stanislav Brabec, SUSE [CZ]. (MA)
* Fix manual page, some lines starting with ' were escaped by \&.
  Reported by Simon Barner. (MA)
* Ship with gettext-0.14.3 again, as 6.2.9-rc10 did. Found by Sunil Shetye. (MA)
* Actually set default SSL certificate path if --sslcertpath is unset.
  Reported by Heino Tiedemann and Rob MacGregor. (MA)
* Remove bogus Netscape IMAP4rev1 Service >= 3.6 warning about BODY[TEXT]
  that we are not using. Patch by Sunil Shetye. (MA)
* Plug potential memory and socket leak when polling multiple folders or when
  the upstream sends bogus message sizes.  Patch by Sunil Shetye. (MA)
* Fix segfault (null pointer dereference) in multidrop mode with headerless
  email.  Reported by Daniel Drake, patch by Sunil Shetye. (MA)
* Update Catalan translation, by Ernest Adrogué Calveras. (MA)
* Fix segfault (null pointer dereference) on some operating systems with
  fetchmail's obsolete DNS MX/host alias lookups in multidrop mode.
  Patch by Dr.-Ing. Andreas Haakh. (MA)
* Close SMTP sockets early, to reduce resource usage, trigger earlier delivery
  with some MTAs and avoid SIGPIPE (SIG 13) when the SMTP listener gets bored
  and drops the connection after timeout.  Patch by Sunil Shetye. (MA)
* Don't treat hitting a fetch limit as error.  Patch by Sunil Shetye. (MA)
* Fix negative "messages left on server" on idle/repoll with fetchlimit.
  Patch by Sunil Shetye. (MA)
* Properly track logout stage.  Patch by Sunil Shetye. (MA)
* Preserve error conditions across postconnect script.  Sunil Shetye. (MA)
* Do not trash destination domain if multiple messages are forwarded into the
  same SMTP/LMTP connection. Reported by Joachim Feise, Berlios Bug #5849. (MA)
  (This patch had not been committed to SVN at the time of the 6.3.1-pre1
  release on 2005-12-13.)
>-------------------------------------------------------------------------------

- -- 
Matthias Andree
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFDnofhvmGDOQUufZURAv6qAKCPRq70AK3RkGyZRkw69r2HHKnT/gCeLtuu
YVANbwz0kkwHLmM+prwMbsY=
=6zvB
-----END PGP SIGNATURE-----

Fw: [fetchmail-users] segmentation fault using 6.3.0

[Adamsonh <ad...@po...>]

Hi Sunil Shetye,

Thanks for your concern.  Simon has solved my problem.

Adamson

----- Original Message ----- 
From: "Simon Barner" <ba...@gm...>
To: <ad...@po...>
Sent: Sunday, December 11, 2005 10:27 PM
Subject: Re: [fetchmail-users] segmentation fault using 6.3.0


Hi,

I am the maintainer of the FreeBSD port. I've just received a patch that
seems to resolve this issue. I've forwarded it to the authors for review
but you might want to try it.

Just put the attached file into your mail/fetchmail/files directory and
rebuild.

P.S.: I am not subscribed to fetchmail-users, I just had a look at the
      archives because I wanted to see if this was a known issue.

-- 
Best regards / Viele Grüße,                             ba...@Fr...
 Simon Barner                                                ba...@gm...

[fetchmail-users] Re: segmentation fault using 6.3.0

[Sunil S. <sh...@bo...>]

Quoting from Adamsonh's mail on Sun, Dec 11, 2005:
> Hi, I portupgraded my fetchmail to 6.3.0 days ago and found it produced a segmentatioin fault when fetching mails from my ISP.  I have to switch back to 6.2.5 because the old version works fine with my pop3 account.
> 
> Please advice.  Thanks.
> 
> ------------------------
> fetchmail: 6.3.0 querying corppop.netvigator.com (protocol POP3) at Sun Dec 11 18:41:48 2005: poll started

...

> fetchmail: POP3> STAT
> fetchmail: POP3< +OK 1 132323
> 1 message for xxx at corppop.netvigator.com (132323 octets).
> fetchmail: POP3> LIST 1
> fetchmail: POP3< +OK 1 132323
> fetchmail: POP3> RETR 1
> fetchmail: POP3< +OK 132323 octets
> reading message xx...@po...:1 of 1 (132323 octets)
> Segmentation fault (core dumped)

Please send the output (after removing password information) of:

$ fetchmail -V [your fetchmail options here]

Are you using fetchmail in multidrop mode? If so, can you run
fetchmail in single drop mode (with keep option, so that mails can be
distributed later) and report if the segmentation fault still occurs?

To get a better gdb output, rerun configure this way:

$ export CFLAGS="-g -ggdb"
$ ./configure [your configure options here]
$ make
# and as root
$ make install         # not install-strip

-- 
Sunil Shetye.

Re: [fetchmail-users] Fetchmail 6.3.0 released

[Matthias A. <mat...@gm...>]

On Sun, 11 Dec 2005, Simon Barner wrote:

> > The version differences should not matter, as c_rehash only hashes the
> > certificates, i. e. runs openssl against the certificate and creates a
> > symlink from XXXXXXXX.N to the actual certificate file, so that access
> > is fast.
> 
> s/access is fast/the certificates are found/
> 
> From my experience without the c_rehash run openssl will fail to find
> the certificates:

Right, the reason is that hash access is O(1) and scanning the whole
directory would be O(n), so the former is the only implemented access
scheme.

> As previously mentioned, fetchmail-6.3.0_2 and the entry to
> ports/UPDATING should make everybody happy.

-- 
Matthias Andree

Re: [fetchmail-users] Fetchmail 6.3.0 released

[Simon B. <ba...@Fr...>]

Matthias Andree wrote:
> Rob MacGregor <rob...@gm...> writes:
> 
> > So, the only way to get it to work is to point /etc/ssl/certs at
> > /usr/local/openssl/certs and use the c_rehash that comes with the port
> > (but given the version differences, I doubt that's a good idea, even
> > if it's what I've done).
> 
> The version differences should not matter, as c_rehash only hashes the
> certificates, i. e. runs openssl against the certificate and creates a
> symlink from XXXXXXXX.N to the actual certificate file, so that access
> is fast.

s/access is fast/the certificates are found/

From my experience without the c_rehash run openssl will fail to find
the certificates:

% ls .certs 
ca.pem       serverca.pem
% fetchmail
fetchmail: Server certificate verification error: unable to get local
issuer certificate
fetchmail: Server certificate verification error: certificate not
trusted
fetchmail: Server certificate verification error: unable to verify the
first certificate
fetchmail: No mail for user at xxxxxxxxxx
^Cfetchmail: terminated with signal 2
% perl /usr/src/crypto/openssl/tools/c_rehash .certs 
Doing .certs
serverca.pem => 55974652.0
ca.pem => 1356e92d.0
% fetchmail                                         
fetchmail: No mail for user at xxxxxxxxxx
^Cfetchmail: terminated with signal 2
% ls .certs 
1356e92d.0   55974652.0   ca.pem       serverca.pem

Excerpt from .fetchmailrc:

options fetchall ssl sslcertpath /home/simon/.certs sslfingerprint '...'

'
> 
> > I'm happy to raise a PR about this as I'd like to see this easier for
> > others to get working - FreeBSD really shouldn't be this hard, that's
> > what Linux is for :-)
>

As previously mentioned, fetchmail-6.3.0_2 and the entry to
ports/UPDATING should make everybody happy.

-- 
Best regards / Viele Grüße,                             ba...@Fr...
 Simon Barner                                                ba...@gm...

Re: [fetchmail-users] Fetchmail 6.3.0 released

[Matthias A. <mat...@gm...>]

Rob MacGregor <rob...@gm...> writes:

> So, the only way to get it to work is to point /etc/ssl/certs at
> /usr/local/openssl/certs and use the c_rehash that comes with the port
> (but given the version differences, I doubt that's a good idea, even
> if it's what I've done).

The version differences should not matter, as c_rehash only hashes the
certificates, i. e. runs openssl against the certificate and creates a
symlink from XXXXXXXX.N to the actual certificate file, so that access
is fast.

> I'm happy to raise a PR about this as I'd like to see this easier for
> others to get working - FreeBSD really shouldn't be this hard, that's
> what Linux is for :-)

:-)

-- 
Matthias Andree

[fetchmail-users] segmentation fault using 6.3.0

[Adamsonh <ad...@po...>]

Hi, I portupgraded my fetchmail to 6.3.0 days ago and found it produced a segmentatioin fault when fetching mails from my ISP.  I have to switch back to 6.2.5 because the old version works fine with my pop3 account.

Please advice.  Thanks.

------------------------
fetchmail: 6.3.0 querying corppop.netvigator.com (protocol POP3) at Sun Dec 11 18:41:48 2005: poll started
fetchmail: POP3< +OK InterMail POP3 server ready.
fetchmail: POP3> CAPA
fetchmail: POP3< +OK Capability list follows
fetchmail: POP3< TOP
fetchmail: POP3< USER
fetchmail: POP3< RESP_CODES
fetchmail: POP3< PIPELINING
fetchmail: POP3< EXPIRE NEVER
fetchmail: POP3< UIDL
fetchmail: POP3< IMPLEMENTATION Openwave Email vM.6.01.05.02 201-2131-123-102-20
fetchmail: POP3< 050715
fetchmail: POP3< .
fetchmail: POP3> USER xxx
fetchmail: POP3< +OK please send PASS command
fetchmail: POP3> PASS *
fetchmail: POP3< +OK xxx is welcome here
fetchmail: selecting or re-polling default folder
fetchmail: POP3> STAT
fetchmail: POP3< +OK 1 132323
1 message for xxx at corppop.netvigator.com (132323 octets).
fetchmail: POP3> LIST 1
fetchmail: POP3< +OK 1 132323
fetchmail: POP3> RETR 1
fetchmail: POP3< +OK 132323 octets
reading message xx...@po...:1 of 1 (132323 octets)
Segmentation fault (core dumped)

--------------------------
gdb result.
--------------------------
Core was generated by `fetchmail'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/local/lib/libintl.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/local/lib/libintl.so.6
Reading symbols from /usr/lib/libopie.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libopie.so.2
Reading symbols from /lib/libcrypt.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypt.so.2
Reading symbols from /lib/libmd.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libmd.so.2
Reading symbols from /lib/libkvm.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libkvm.so.2
Reading symbols from /usr/local/lib/libcom_err.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/local/lib/libcom_err.so.2
Reading symbols from /usr/lib/libssl.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libssl.so.3
Reading symbols from /lib/libcrypto.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypto.so.3
Reading symbols from /lib/libc.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.5
Reading symbols from /usr/local/lib/libiconv.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/local/lib/libiconv.so.3
Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done.
Loaded symbols for /libexec/ld-elf.so.1
#0  0x282b6e12 in strcasecmp () from /lib/libc.so.5
(gdb) backtrace full
#0  0x282b6e12 in strcasecmp () from /lib/libc.so.5
No symbol table info available.
#1  0x0806551c in sigprocmask ()
No symbol table info available.
#2  0x080590fd in sigprocmask ()
No symbol table info available.
#3  0x0805a9bf in sigprocmask ()
No symbol table info available.
#4  0x08057584 in sigprocmask ()
No symbol table info available.
#5  0x08058963 in sigprocmask ()
No symbol table info available.
#6  0x08058e8c in sigprocmask ()
No symbol table info available.
#7  0x0804e658 in sigprocmask ()
No symbol table info available.
#8  0x0805342b in sigprocmask ()
No symbol table info available.
#9  0x08051709 in sigprocmask ()
No symbol table info available.
#10 0x0804ad82 in sigprocmask ()
No symbol table info available.

[fetchmail-users] segmentation fault using 6.3.0

[test <ad...@po...>]

Hi, I portupgraded my fetchmail to 6.3.0 days ago and found it produced a segmentatioin fault when fetching mails from my ISP.  I have to switch back to 6.2.5 because the old version works smoothly with my pop3 account.

Please help.  Thanks.

------------------------
fetchmail: 6.3.0 querying corppop.netvigator.com (protocol POP3) at Sun Dec 11 18:41:48 2005: poll started
fetchmail: POP3< +OK InterMail POP3 server ready.
fetchmail: POP3> CAPA
fetchmail: POP3< +OK Capability list follows
fetchmail: POP3< TOP
fetchmail: POP3< USER
fetchmail: POP3< RESP_CODES
fetchmail: POP3< PIPELINING
fetchmail: POP3< EXPIRE NEVER
fetchmail: POP3< UIDL
fetchmail: POP3< IMPLEMENTATION Openwave Email vM.6.01.05.02 201-2131-123-102-20
fetchmail: POP3< 050715
fetchmail: POP3< .
fetchmail: POP3> USER xxx
fetchmail: POP3< +OK please send PASS command
fetchmail: POP3> PASS *
fetchmail: POP3< +OK xxx is welcome here
fetchmail: selecting or re-polling default folder
fetchmail: POP3> STAT
fetchmail: POP3< +OK 1 132323
1 message for xxx at corppop.netvigator.com (132323 octets).
fetchmail: POP3> LIST 1
fetchmail: POP3< +OK 1 132323
fetchmail: POP3> RETR 1
fetchmail: POP3< +OK 132323 octets
reading message xx...@po...:1 of 1 (132323 octets)
Segmentation fault (core dumped)

--------------------------
gdb result.
--------------------------
Core was generated by `fetchmail'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/local/lib/libintl.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/local/lib/libintl.so.6
Reading symbols from /usr/lib/libopie.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libopie.so.2
Reading symbols from /lib/libcrypt.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypt.so.2
Reading symbols from /lib/libmd.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libmd.so.2
Reading symbols from /lib/libkvm.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libkvm.so.2
Reading symbols from /usr/local/lib/libcom_err.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/local/lib/libcom_err.so.2
Reading symbols from /usr/lib/libssl.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libssl.so.3
Reading symbols from /lib/libcrypto.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypto.so.3
Reading symbols from /lib/libc.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.5
Reading symbols from /usr/local/lib/libiconv.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/local/lib/libiconv.so.3
Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done.
Loaded symbols for /libexec/ld-elf.so.1
#0  0x282b6e12 in strcasecmp () from /lib/libc.so.5
(gdb) backtrace full
#0  0x282b6e12 in strcasecmp () from /lib/libc.so.5
No symbol table info available.
#1  0x0806551c in sigprocmask ()
No symbol table info available.
#2  0x080590fd in sigprocmask ()
No symbol table info available.
#3  0x0805a9bf in sigprocmask ()
No symbol table info available.
#4  0x08057584 in sigprocmask ()
No symbol table info available.
#5  0x08058963 in sigprocmask ()
No symbol table info available.
#6  0x08058e8c in sigprocmask ()
No symbol table info available.
#7  0x0804e658 in sigprocmask ()
No symbol table info available.
#8  0x0805342b in sigprocmask ()
No symbol table info available.
#9  0x08051709 in sigprocmask ()
No symbol table info available.
#10 0x0804ad82 in sigprocmask ()
No symbol table info available.

Re: [fetchmail-users] Fetchmail 6.3.0 released

[Rob M. <rob...@gm...>]

On 10/12/05, Simon Barner <ba...@fr...> wrote:
>
> The security/ca-roots port creates a link from /etc/ssl/cert.pem ->
> /usr/local/share/certs/ca-root.crt
>
> I've just verified that fetchmail picks up those certificates if they
> are installed onto a pristine /etc/ssl.
>
> I'll think about adding a dependency in fetchmail.

Please do - I can confirm that installing security/ca-roots solves the
problem on my system.

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-users] Fetchmail 6.3.0 released

[Simon B. <ba...@Fr...>]

Rob MacGregor wrote:
> On 10/12/05, Simon Barner <ba...@fr...> wrote:
> >
> > The patch is included in fetchmail-6.3.0_1. Thanks for the quick fix,
> > Matthias!
> 
> Only problem is - it doesn't really fix anything without some other
> finger waving.
> 
> Poking around, using truss etc, I discovered that the base libraries
> look in /etc/ssl/certs, the version of c_rehash (that doesn't get
> installed) that hides in the source tree looks in /usr/local/ssl/certs
> and the ports version of openssl looks in /usr/local/openssl/certs.
> 
> In short, a bit of a mess :-)
> 
> So, the only way to get it to work is to point /etc/ssl/certs at
> /usr/local/openssl/certs and use the c_rehash that comes with the port
> (but given the version differences, I doubt that's a good idea, even
> if it's what I've done).  Or to edit and install the c_rehash that
> comes with the source tree, but doesn't get installed (and create the
> top level directory required).

The security/ca-roots port creates a link from /etc/ssl/cert.pem ->
/usr/local/share/certs/ca-root.crt

I've just verified that fetchmail picks up those certificates if they
are installed onto a pristine /etc/ssl.

I'll think about adding a dependency in fetchmail.

> 
> I'm happy to raise a PR about this as I'd like to see this easier for
> others to get working - FreeBSD really shouldn't be this hard, that's
> what Linux is for :-)
> 
> --
>                  Please keep list traffic on the list.
> Rob MacGregor
>       Whoever fights monsters should see to it that in the process he
>         doesn't become a monster.                  Friedrich Nietzsche

-- 
Best regards / Viele Grüße,                             ba...@Fr...
 Simon Barner                                                ba...@gm...

Re: [fetchmail-users] Fetchmail 6.3.0 released

[Rob M. <rob...@gm...>]

On 10/12/05, Simon Barner <ba...@fr...> wrote:
>
> The patch is included in fetchmail-6.3.0_1. Thanks for the quick fix,
> Matthias!

Only problem is - it doesn't really fix anything without some other
finger waving.

Poking around, using truss etc, I discovered that the base libraries
look in /etc/ssl/certs, the version of c_rehash (that doesn't get
installed) that hides in the source tree looks in /usr/local/ssl/certs
and the ports version of openssl looks in /usr/local/openssl/certs.

In short, a bit of a mess :-)

So, the only way to get it to work is to point /etc/ssl/certs at
/usr/local/openssl/certs and use the c_rehash that comes with the port
(but given the version differences, I doubt that's a good idea, even
if it's what I've done).  Or to edit and install the c_rehash that
comes with the source tree, but doesn't get installed (and create the
top level directory required).

I'm happy to raise a PR about this as I'd like to see this easier for
others to get working - FreeBSD really shouldn't be this hard, that's
what Linux is for :-)

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-users] Fetchmail 6.3.0 released

[Simon B. <ba...@Fr...>]

Matthias Andree wrote:
> > (Keyword: sslcertpath) Sets the directory fetchmail uses to look up
> > local certificates. The default is your OpenSSL default one.
> >
> > Well, actually, it doesn't look like there is a default (at least not
> > on FreeBSD 5.4).  I had to explicitly state the path for each server I
> > was polling.
> 
> Please drop the attached patch-file into your
> ports/mail/fetchmail/files/ directory (as patch-socket.c) and rebuild
> (aka "make clean && make all deinstall install clean") then let me know
> if your problem is fixed.

Content-Description: fix default --sslcertpath
> Index: socket.c
> ===================================================================
> --- socket.c	(Revision 4499)
> +++ socket.c	(Arbeitskopie)
> @@ -841,6 +841,8 @@
>  	}
>  	if (certpath)
>  		SSL_CTX_load_verify_locations(_ctx, NULL, certpath);
> +	else
> +		SSL_CTX_set_default_verify_paths(_ctx);
>  	
>  	_ssl_context[sock] = SSL_new(_ctx);
>  	

The patch is included in fetchmail-6.3.0_1. Thanks for the quick fix,
Matthias!

-- 
Best regards / Viele Grüße,                             ba...@Fr...
 Simon Barner                                                ba...@gm...

Re: [fetchmail-users] Fetchmail 6.3.0 released

[Matthias A. <mat...@gm...>]

Rob MacGregor <rob...@gm...> writes:

> So, finally digging through a truss shows that fetchmail/system
> openssl is looking in /etc/ssl/certs (which doesn't exist), not
> /usr/local/<anything>.  Nice consistency :-)
>
> One sym-link later and all is working as it should.  Not sure how you
> could patch for this though - the whole openssl install is a bit of a
> mess, with the system tools looking in 2 different locations and the
> port in another.

I'm very much inclined to see this as a FreeBSD system or port bug in
OpenSSL - the same patch works for me on Linux, where /etc/ssl/certs is
the right place to look.

Can you file a FreeBSD PR against OpenSSL with your findings? Someone
has to look after this, and c_rehash should also be installed (or
rewritten if it's in Perl - which is no longer part of the base
install).

-- 
Matthias Andree

Re: [fetchmail-users] Fetchmail 6.3.0 released

[Rob M. <rob...@gm...>]

On 07/12/05, Matthias Andree <mat...@gm...> wrote:
>
> Please drop the attached patch-file into your
> ports/mail/fetchmail/files/ directory (as patch-socket.c) and rebuild
> (aka "make clean && make all deinstall install clean") then let me know
> if your problem is fixed.

That didn't (quite) do it I'm afraid:

fetchmail: Server certificate verification error: unable to get local
issuer certificate
fetchmail: Server certificate verification error: certificate not trusted
fetchmail: Server certificate verification error: unable to verify the
first certificate

I still need "ssl sslcertpath /usr/local/openssl/certs" in my
fetchmailrc to get it to work.

Now, checking the libraries fetchmail is linked against the system
openssl, whereas c_rehash is coming from the port install of openssl. 
Having finally found the c_rehash tool in the FreeBSD source tree I
worked out where it expected the certificates (/usr/local/ssl/certs,
when /usr/local/ssl doesn't exist), but that made no difference
either.

So, finally digging through a truss shows that fetchmail/system
openssl is looking in /etc/ssl/certs (which doesn't exist), not
/usr/local/<anything>.  Nice consistency :-)

One sym-link later and all is working as it should.  Not sure how you
could patch for this though - the whole openssl install is a bit of a
mess, with the system tools looking in 2 different locations and the
port in another.

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

[fetchmail-users] Re: more clever resource limitations

[Chris B. <li...@pi...>]

fet...@be... wrote:

>Message: 1
>Date: Thu, 8 Dec 2005 22:04:18 -0800
>From: Greg Lindahl <li...@pb...>
>To: fet...@li...
>Subject: [fetchmail-users] more clever resource limitations
>
>[snip]
>
>Due to the advent of spamassassin, I am no longer able to accept lots
>of messages at once via fetchmail -- it causes my laptop to thrash.  I
>use sendmail -> procmail -> spamassassin, so using sendmail's load
>average limit didn't really help the situation, since sendmail has no
>idea it's kicked off a bunch of spamassassin processes that in a few
>seconds will drive the load way up. (In this analysis I'm assuming that
>it's only cpu time and not disk I/O which is causing the problem.)
>  
>
Can you not limit the no. of messages sendmail accepts per connection 
and a queue limit instead? That's what i did with my similar setup but 
with exim as the MTA. Fetchmail copes with this gracefully. I had 
exactly the same problem every time I went on holiday and shutdown my 
local mailserver: with the huge volume of spam, fetchmai, would go and 
get loads of messages from my POP3 box, deliver them to Exim, which 
would then launch lots of spamd processes. The box would work itself 
into a corner until theer was little choice but to 240-reset it! Once I 
set the limits in exim, it's just a question of waiting a while.

[fetchmail-users] Re: more clever resource limitations

[Sunil S. <sh...@bo...>]

Hi Greg,

Quoting from Greg Lindahl's mail on Thu, Dec 08, 2005:
> Due to the advent of spamassassin, I am no longer able to accept lots
> of messages at once via fetchmail -- it causes my laptop to thrash.  I
> use sendmail -> procmail -> spamassassin, so using sendmail's load
> average limit didn't really help the situation, since sendmail has no
> idea it's kicked off a bunch of spamassassin processes that in a few
> seconds will drive the load way up. (In this analysis I'm assuming that
> it's only cpu time and not disk I/O which is causing the problem.)

Are you invoking spamassassin directly from procmail? Or, are you
using spamc+spamd?

1. If you are invoking spamassassin from procmail, please try starting
spamd and then using spamc from procmail.

2. If you are invoking spamc from procmail, what are the options being
passed to spamd? If the spamd option -m has a high value (more than
10), try reducing it to 5.

3. You can also use locking in procmail. If your recipe looks like
this:

=============================================
:0
| spamc
=============================================

Change it to:

=============================================
# Invoke only one spamc at a time
:0 : $HOME/.spamc.lock
| spamc
=============================================

-- 
Sunil Shetye.

Re: [fetchmail-users] more clever resource limitations

[Matthias A. <mat...@gm...>]

Greg Lindahl <li...@pb...> writes:

> I was pleased to see on LWN that you guys are working on fetchmail,
> I had no idea it had active maintainers.

Good to see that the word is spreading. :-)

> Due to the advent of spamassassin, I am no longer able to accept lots
>of messages at once via fetchmail -- it causes my laptop to thrash.  I

My personal experience is that SpamAssassin's biggest hog is its Bayes
database, disabling it speeds up SA quite a bit. External filters such
as bogofilter, spamprobe or qsf are much faster than SpamAssassin, and
as you are already using procmail, integration is quite easy.

It takes a bit of manual work training the filter, though. Also, if you
use "spamassassin" in your procmail script, consider running a "spamd"
daemon and then use "spamc" in your script instead, this also gives you
some resource control given proper spamd settings.

You can also use a procmail "local lock file" to make sure only one
spamassassin process is running at a time, and you can also precede the
spamassassin line with "nice". (You'd change your ":0fw" rule to
":0fw:.spamassassin.lock" or similar, depending on how _exactly_ you use
spamassassin.)

>use sendmail -> procmail -> spamassassin, so using sendmail's load
>average limit didn't really help the situation, since sendmail has no
>idea it's kicked off a bunch of spamassassin processes that in a few
>seconds will drive the load way up. (In this analysis I'm assuming that
>it's only cpu time and not disk I/O which is causing the problem.)

The actual problem is that sendmail is somewhat limited in terms of
resource control. With Postfix for instance you'd set the
local_concurrency_limit to some low value and that's it, but as shown
above there are ways around your problem.

>  --fetchlimitsize n (kilobytes, default to 100 kilobytes? 1 meg?)
>  --expungesize n (kilobytes, same default as fetchlimitsize)
>
> In addition, we could use a rate limiter sending messages to sendmail
> to dodge the spamassassin problem. My laptop takes 2 cpu seconds to
> spamassassin a small email message (20 seconds elapsed) and so perhaps
> a time-based measure with an optional addition for the message size
> would do it. If you wanted to be clever you could compute this
> dynamically by watching what happens when you throw a bunch of email
> messages at sendmail, but to start:
>
>  --mtacpu n (seconds, default to 0)
>  --mtacpupermeg n (seconds per megabyte, default to 0)
>
> so fetchmail would wait "mtacpu + size / 1meg * mtacpupermeg" seconds
> before sending another message to the MTA. I'd set this to 2. Hm,
> given that this value is small a floating-point value would be a
> good idea instead of an integer.

As you talked about varying network connection quality, a time limit is
probably one easy way to go and it makes good sense for --expunge* like
options, too -- fetchmail might then checkpoint the mailbox every two
minutes or so (or whichever the user set).

-- 
Matthias Andree

Re: [fetchmail-users] more clever resource limitations

[Jakob H. <jh...@pl...>]

Greg Lindahl wrote:

> use sendmail -> procmail -> spamassassin, so using sendmail's load

Why don't you use fetchmail -> procmail -> spamassassin (mda option)? This
way only one message would be delivered at one time and there's no need to
fix other component's issues in fetchmail.
Speaking of, sendmail surely has some mechanisms to queue messages and
deliver only message per time, which would also fix your problem. Or
configure procmail to use some locking to only start one spamassassin
process.

> I worked around this by adding expunge 1 (which since I'm using POP3
> is similar to using a fetchlimit), which slows things down enough by
> re-logging in repeatedly (wastefully) to avoid the problem. But this
> doesn't work well when I'm on the road and have a bad network
> connection; it slows things down quite a lot, especially when I have a

By "bad" you mean, it breaks often? Then expunge 1 is a good idea, else
you could be downloading messages more than once.

[fetchmail-users] more clever resource limitations

[Greg L. <li...@pb...>]

I was pleased to see on LWN that you guys are working on fetchmail,
I had no idea it had active maintainers.

Due to the advent of spamassassin, I am no longer able to accept lots
of messages at once via fetchmail -- it causes my laptop to thrash.  I
use sendmail -> procmail -> spamassassin, so using sendmail's load
average limit didn't really help the situation, since sendmail has no
idea it's kicked off a bunch of spamassassin processes that in a few
seconds will drive the load way up. (In this analysis I'm assuming that
it's only cpu time and not disk I/O which is causing the problem.)

I worked around this by adding expunge 1 (which since I'm using POP3
is similar to using a fetchlimit), which slows things down enough by
re-logging in repeatedly (wastefully) to avoid the problem. But this
doesn't work well when I'm on the road and have a bad network
connection; it slows things down quite a lot, especially when I have a
big inbox. (I see that I'm not using the new fetchsizelimit and
fastuidl options. But, there are more issues:) Also a large expunge
limit is simply a bad idea for slow connections if there are large
messages; the odds of something going wrong is drastically
increased. And if I move my laptop from the office to a hotel I
currently would have to change the expunge/fetchlimit value by hand.

I think what we really want is something which decides when to expunge
or limit based on the *size* of the messages downloaded. This would
address the current issue that we don't expunge very intelligently;
the point is that you don't want hugely long downloads which might be
interrupted and lose some state, so it makes sense to expunge more
frequently when large messages are being downloaded. One size doesn't
fit all.

 --fetchlimitsize n (kilobytes, default to 100 kilobytes? 1 meg?)
 --expungesize n (kilobytes, same default as fetchlimitsize)

In addition, we could use a rate limiter sending messages to sendmail
to dodge the spamassassin problem. My laptop takes 2 cpu seconds to
spamassassin a small email message (20 seconds elapsed) and so perhaps
a time-based measure with an optional addition for the message size
would do it. If you wanted to be clever you could compute this
dynamically by watching what happens when you throw a bunch of email
messages at sendmail, but to start:

 --mtacpu n (seconds, default to 0)
 --mtacpupermeg n (seconds per megabyte, default to 0)

so fetchmail would wait "mtacpu + size / 1meg * mtacpupermeg" seconds
before sending another message to the MTA. I'd set this to 2. Hm,
given that this value is small a floating-point value would be a
good idea instead of an integer.

Just another fetchmail user,

-- greg

Re: [fetchmail-users] Fetchmail 6.3.0 released

[Rob M. <rob...@gm...>]

On 07/12/05, Matthias Andree <mat...@gm...> wrote:
>
> What is your master configuration?

I don't really have one, what I have is a number of files that start like this:

defaults proto pop3                     # Default to POP3 mail servers
        no rewrite                      # Don't rewrite headers
                # NOTE: the TRACEPOLLS line is supressed if the
following is used...
        set invisible                   # Don't appear in headers
        set no bouncemail               # Don't bounce mail in error -
give it to...
        set syslog                      # Log to syslog under the "mail" heading
        set idfile      /etc/fetchids-by        # Where to store the
POP3 UID lists

With the idfile changing for each one.  I've got a total of 4 separate
fetchmailrcs and fetchid files (one for each ISP effectively).  None
of these run as root, so the files in /etc look like:

-rw-------  1 fetchmail-by     wheel     0 Dec  4 22:32 fetchids-by
-rw-------  1 fetchmail-lance  wheel   237 Dec  7 07:19 fetchids-lance
-rw-------  1 fetchmail-one    wheel     0 Jan 23  2004 fetchids-one
-rw-------  1 fetchmail-other  wheel     0 Jan 23  2004 fetchids-other
-rw-------  1 fetchmail-by     wheel  2545 Dec  5 21:18 fetchmailrc-by
-rw-------  1 fetchmail-lance  wheel  1133 Dec  2 14:44 fetchmailrc-lance
-rw-------  1 fetchmail-one    wheel  2381 Oct  3 07:47 fetchmailrc-one
-rw-------  1 fetchmail-other  wheel  1144 Oct 28 05:54 fetchmailrc-other

The solution was easy, create an /etc/fetchmail directory, with
subdirectories for each account.  It's just that the error message
didn't appear under 6.2.x, though it's entirely possible it was
equally broken (on that note, I'm pleased to see a general improvement
of error message quality, including mailbox names in some of the more
common messages was a stroke of genius on somebody's part).

On a vaguely unrelated note - there's a documentation error relating
to ssl support.  Having just this morning noticed that one of my
remote providers supports TLS I set about getting things working, what
caught me was:

(Keyword: sslcertpath) Sets the directory fetchmail uses to look up
local certificates. The default is your OpenSSL default one.

Well, actually, it doesn't look like there is a default (at least not
on FreeBSD 5.4).  I had to explicitly state the path for each server I
was polling.

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche