fetchmail-users — User questions and discussion

Re: [fetchmail-users] Bug in imap_idle() with mail servers not supporting "idle"

[Matthias A. <mat...@gm...>]

Brendan Lynch <bre...@ai...> writes:

> This code works fine until a notification of new mail is received (a "*
> 1 EXISTS" message is received).  At this point normally one is in the
> "imap_ok()" routine called from line 637, and this correctly receives
> the notification message and parses it, updating the "count" variable.
> However it does not the return to imap_idle() routine, but instead
> reissues a recv call having set a much longer (300s) timeout and after
> the 300s have expired it then returns an error causing fetchmail to drop
> the IMAP connection (a main point of this idle code was to keep the
> connection open for subsequent message retrieval).  Net result is that
> delivery of mail is delayed by 5 minutes and the server connection is
> dropped and reestablished each time a wait for mail occurs.
>
> The problem seems to be caused by the loop condition in imap_ok():
>
>    151         }
>    152     } while
>    153         (tag[0] != '\0' && strncmp(buf, tag, strlen(tag)));
>    154
>
> This assumes that tag[0] will be set to '\0' if one is not waiting for a
> tagged response.  In this case the code should not be waiting for a
> tagged response (it is waiting for an unsolicited notification).
> However the 'tag' global character array is set by the gen_transact() at
> line 630 and is not cleared before the call to imap_ok() at line 637.

This is correct, an IMAP client is supposed to parse untagged responses
until a tagged response is received. Trying with Dovecot and hacking
fetchmail a bit so it doesn't recognize RECENT and uses the NOOP
emulation code yields:

...
fetchmail: IMAP> A0010 NOOP
fetchmail: IMAP< A0010 OK NOOP completed.
fetchmail: IMAP> A0011 NOOP
fetchmail: IMAP< * 1 EXISTS
fetchmail: IMAP< * 1 RECENT
fetchmail: IMAP< A0011 OK NOOP completed.
fetchmail: IMAP> A0012 NOOP
...

So it waits for the tagged NOOP response, and this is a requirement so
it actually picks up both the EXISTS and the RECENT responses of working
servers. Servers that do not respond with a tagged response to a NOOP
command are broken.

> The fix is a very simple one-line change to imap_idle (2 lines with
> comments):
>
>    630         ok = gen_transact(sock, "NOOP");
>    631         /* if there's an error (not likely) or we just found mail
> (stage
>    632          * has changed, timeout has also been restored), we're
> done */
>    633         if (ok != 0 || stage != STAGE_IDLE)
>    634             return(ok);
>    635
> +   636         /* clear tag so imap_ok does not expect tagged response */
> +   637         tag[0]='\0';
>    638         /* wait (briefly) for an unsolicited status update */
>    639         ok = imap_ok(sock, NULL);

So this patch would sort of break the IMAP client because it would jump
out of the loop before having read the reply. This requires more
thought. Casper Gripenberg reported a similar problem, so perhaps some
common upstream server software is the culprit (and he suggested a
different fix, I'll have a look at that too).

What software is your upstream server running?

Can I see a "fetchmail -Nvvv --nosyslog" trace of a failing IMAP session
with NOOP emulation?

> A second, more minor problem is that getting a "* RECENT" notification
> does not break a caller out of the imap_idle's imap_ok() call.  This
> causes an extra 28second wait after being notified about a message
> before it is actually received.
>
> Diffs for the complete set of changes against 6.3.2 are attached to this
> email.
>
> I have seen this in fetchmail 6.2.5 and 6.3.2 on linux platforms, but
> this problem should be generic to all platforms.
> diff -Naur fetchmail-6.3.2/imap.c fetchmail-6.3.2a/imap.c
> --- fetchmail-6.3.2/imap.c  2006-01-20 10:38:45.000000000 +0000
> +++ fetchmail-6.3.2a/imap.c 2006-02-23 23:54:52.000000000 +0000
> @@ -116,6 +116,17 @@
>         {
>         recentcount_ok = 1;
>         recentcount = atoi(buf+2);
> +       /*
> +        * Kluge to handle IDLE simulation. If we are in STAGE_IDLE and
> +        * we are simulating (has_idle unset) we need to alert calling
> +        * imap_idle() to the fact that we have received a new "recent"
> +        * alert in order to break the imap_idle() out of its wait.
> +        */
> +       if (!has_idle && stage == STAGE_IDLE)
> +       {
> +           mytimeout = saved_timeout;
> +           stage = STAGE_FETCH;
> +       }
>         }
>         else if (strstr(buf, " EXPUNGE"))
>         {

This looks acceptable.

How can a server mark a new message "RECENT" without also sending an
"EXISTS"? I'm inclined to consider such behavior broken. I'm willing to
apply this nonetheless as I don't think it would hurt anyone.

-- 
Matthias Andree

[fetchmail-users] Re: Fetchmail

[Sunil S. <sh...@bo...>]

Hello Indika,

Quoting from Indika Mendis's mail on Mon, Feb 27, 2006:
>    My  name  is  Indika  and  using  fetchmail in my linux mail server to
>    download  mails.  It  is working properly...  but I have to do a small
>    thing....  Our  staff  members are receiving some unwanted emails with
>    several  subjects.  Is  there  any  configuration  that we can do with
>    fetchmail  or  any  other  thing  in linux servers....  I want to stop
>    these mail by filtering subject wise... pl help me to solve this prob.

This is a fetchmail FAQ entry:

<http://fetchmail.berlios.de/fetchmail-FAQ.html#G4>

For filtering by subject, use procmail or maildrop. For that, you will
have to find out what programs are installed on your system. Also, you
will have to analyse the output of "fetchmail -v".

Here is one sample setup:

Question                                             Possible answer
--------------------------------------------------------------------
What SMTP servers are installed on my system?        qmail, sendmail
What mail filtering programs are installed?          maildrop, procmail
Where is fetchmail forwarding my mails to?           SMTP server
What SMTP server is running on my system?            sendmail
Is my SMTP server also delivering mails directly?    no
Who is delivering my mails?                          procmail
What spam filtering programs are installed?          spamassassin

In this setup, procmail has already been enabled. So, it is the
question of just writing the procmailrc. Details of setting up the
procmailrc (per-user as well as globally) can be found in procmail
documentation.

If you are interesting in spam filtering, you can enable that too in
the rc file of the mail-filtering program.

If you have more queries, please subscribe to the user mailling list.
You can find more details of the lists at
<http://fetchmail.berlios.de/>. After subscribing, send the output of
"fetchmail -V" and "fetchmail -v" to the user mailing list.

-- 
Sunil Shetye.

[fetchmail-users] Bug in imap_idle() with mail servers not supporting "idle"

[Brendan L. <bre...@ai...>]

I've run across what I believe to be a bug in imap_idle() on servers not 
supporting the IDLE extension.  There is code in imap_idle() that tries 
to simulate and idle() call with frequent NOOP calls:

    624     } else {  /* no idle support, fake it */
    625         /* when faking an idle, we can't assume the server will
    626          * send us the new messages out of the blue (RFC2060);
    627          * this timeout is potentially the delay before we notice
    628          * new mail (can be small since NOOP checking is cheap) */
    629         mytimeout = 28;
    630         ok = gen_transact(sock, "NOOP");
    631         /* if there's an error (not likely) or we just found 
mail (stage
    632          * has changed, timeout has also been restored), we're 
done */
    633         if (ok != 0 || stage != STAGE_IDLE)
    634             return(ok);
    635
    636         /* wait (briefly) for an unsolicited status update */
    637         ok = imap_ok(sock, NULL);
    638         /* again, this is new mail or an error */
    639         if (ok != PS_IDLETIMEOUT)
    640             return(ok);
    641     }

This code works fine until a notification of new mail is received (a "* 
1 EXISTS" message is received).  At this point normally one is in the 
"imap_ok()" routine called from line 637, and this correctly receives 
the notification message and parses it, updating the "count" variable.  
However it does not the return to imap_idle() routine, but instead 
reissues a recv call having set a much longer (300s) timeout and after 
the 300s have expired it then returns an error causing fetchmail to drop 
the IMAP connection (a main point of this idle code was to keep the 
connection open for subsequent message retrieval).  Net result is that 
delivery of mail is delayed by 5 minutes and the server connection is 
dropped and reestablished each time a wait for mail occurs.

The problem seems to be caused by the loop condition in imap_ok():

    151         }
    152     } while
    153         (tag[0] != '\0' && strncmp(buf, tag, strlen(tag)));
    154

This assumes that tag[0] will be set to '\0' if one is not waiting for a 
tagged response.  In this case the code should not be waiting for a 
tagged response (it is waiting for an unsolicited notification).  
However the 'tag' global character array is set by the gen_transact() at 
line 630 and is not cleared before the call to imap_ok() at line 637.

The fix is a very simple one-line change to imap_idle (2 lines with 
comments):

    630         ok = gen_transact(sock, "NOOP");
    631         /* if there's an error (not likely) or we just found 
mail (stage
    632          * has changed, timeout has also been restored), we're 
done */
    633         if (ok != 0 || stage != STAGE_IDLE)
    634             return(ok);
    635
+   636         /* clear tag so imap_ok does not expect tagged response */
+   637         tag[0]='\0';
    638         /* wait (briefly) for an unsolicited status update */
    639         ok = imap_ok(sock, NULL);

A second, more minor problem is that getting a "* RECENT" notification 
does not break a caller out of the imap_idle's imap_ok() call.  This 
causes an extra 28second wait after being notified about a message 
before it is actually received.

Diffs for the complete set of changes against 6.3.2 are attached to this 
email.

I have seen this in fetchmail 6.2.5 and 6.3.2 on linux platforms, but 
this problem should be generic to all platforms.

Re: [fetchmail-users] Incorrect header line and lost mails

[Matthias A. <mat...@gm...>]

Frederic Marchal <fre...@wo...> writes:

> Well, I'm a C programmer and I may be able to help but I'm mainly a
> windows programmer (yes, I know, shame on me :-) ) and I'm not yet all
> too familiar with the Linux programming and especially with the patch
> mechanism and none at all with the diff tool.

The workflow is:

1. create a copy of all files you need to edit, for instance, 
   copy foo.c to foo.c.orig

   alternatively, copy the whole directory tree recursively

2. edit until you're happy with the code

3. test the edits (compile, install, run)

4. generate a unified (preferred) or context patch.  Not all diff
   utilities support unified output, but all are to support context
   format, as mandated by the relevant standard, IEEE Std 1003.1-2001.

   Plain ed-style patches (unfortunately still the default in the diff
   utility) are impractical so they are not usually accepted.

The diff tool is quite simple actually, if you created file copies:

diff -u OLDFILE NEWFILE >PATCHFILE (unified diff, preferred)

diff -c OLDFILE NEWFILE >PATCHFILE (context diff, alternative)

If you copied the whole directory before the edits:

diff -u -r BACKUP_DIR EDITED_DIR >PATCHFILE    (or -c instead of -u)

If you added new files to EDITED_DIR, you'd use -N but I don't think
we'll need this here.

diff, used either way, will produce a file (PATCHFILE) that you'd attach
to a message and mail to me (or Rob, or preferably the -devel@ list) or
upload to the BerliOS patch tracker.

If I have the choice, I prefer the -devel@ list. That's public, archived
and easiest for me since I don't need to fetch anything, but get it
pushed into my inbox.


The patch utility does the reverse and will usually just be used like
this, it derives the file names from the PATCHFILE.

patch <PATCHFILE

perhaps with -d (to change directory) or -p (to strip leading path
components from the paths shown in the patch file; used if patch cannot
find the files).

> Nevertheless, I have identified the core of the problem (its the call
> to readheaders in drive.c and the response to PS_REFUSED). I still
> have to check what happens if an invalid FROM or TO header is passed
> after readheaders but I think I can fix the program.

That would be great.

Basically they should not matter that much. fetchmail has some
deprecated code to guess from To:/Cc: or similar, but that is an
unreliable concept and doesn't deserve attention, From: shouldn't matter
as long as Return-Path: is given (and even then, From: shouldn't
matter). Missing or corrupted Return-Path: headers should be treated as
though the message had contained "Return-Path: <>".

> I'll also need some direction about the proper way of doing it for a
> good integration in the current source.

The bigger concern is that fetchmail shouldn't be emitting headers it
knows are broken, so that the next hop (the MTA or MDA) does not get
confused. I wonder if fetchmail should prefix the broken lines with
X-Fetchmail-Escaped-Broken-Header: or something similar.

-- 
Matthias Andree

Re: [fetchmail-users] Incorrect header line and lost mails

[Frederic M. <fre...@wo...>]

Matthias Andree wrote:
> Rob MacGregor <rob...@gm...> writes:
>
>   
>> On 2/13/06, Frederic Marchal <fre...@wo...> wrote:
>>     
>>> Hello,
>>>
>>> I found 15 lines like this in my log this morning and 3 more earlier
>>> last week:
>>>
>>> fetchmail[1226]: incorrect header line found while scanning headers
>>>       
>> You'll find threads about this in the list archive.  I don't remember
>> the outcome, but I've a feeling that 6.3.x handles these (illegal)
>> messages differently.
>>     
>
> I don't recall having a fix for this particular issue, but 6.3.2
> generally has some dozens of bugfixes worth having anyways. I'll put
> this on my 6.3.3 TODO, but don't think it will appear within the next
> three weeks (unless someone else sends a patch). Don't hold your breath,
> I'm busy ATM.
>   

Well, I'm a C programmer and I may be able to help but I'm mainly a 
windows programmer (yes, I know, shame on me :-) ) and I'm not yet all 
too familiar with the Linux programming and especially with the patch 
mechanism and none at all with the diff tool. Nevertheless, I have 
identified the core of the problem (its the call to readheaders in 
drive.c and the response to PS_REFUSED). I still have to check what 
happens if an invalid FROM or TO header is passed after readheaders but 
I think I can fix the program. I'll also need some direction about the 
proper way of doing it for a good integration in the current source.

If you think it is reasonable, we can discuss this further on the 
fetchmail-devel list.

Frederic

Re: [fetchmail-users] Incorrect header line and lost mails

[Frederic M. <fre...@wo...>]

Rob MacGregor wrote:
> On 2/13/06, Frederic Marchal <fre...@wo...> wrote:
>   
>> Hello,
>>
>> I found 15 lines like this in my log this morning and 3 more earlier
>> last week:
>>
>> fetchmail[1226]: incorrect header line found while scanning headers
>>     
>
> You'll find threads about this in the list archive.  I don't remember
> the outcome, but I've a feeling that 6.3.x handles these (illegal)
> messages differently.
>
> It would be worth giving the latest version a try and see if that helps.
>   
I read the source code of 6.3.2 and the handling of those illegal mails 
is still exactly the same. Those e-mails are discarded and the 
"incorrect header line" message is written to the log.

Frederic


WOW Company or its affiliates do not accept legal responsibility 
for the contents of this message. The views or opinions 
presented are solely those of the author and do not necessarily 
represent those of WOW Company or any of its affiliates.

No contract will be established with WOW Company other than 
in writing (letter) and with the explicit authorization of the director 
of WOW Company.

Re: [fetchmail-users] Incorrect header line and lost mails

[Matthias A. <mat...@gm...>]

Rob MacGregor <rob...@gm...> writes:

> On 2/13/06, Frederic Marchal <fre...@wo...> wrote:
>> Hello,
>>
>> I found 15 lines like this in my log this morning and 3 more earlier
>> last week:
>>
>> fetchmail[1226]: incorrect header line found while scanning headers
>
> You'll find threads about this in the list archive.  I don't remember
> the outcome, but I've a feeling that 6.3.x handles these (illegal)
> messages differently.

I don't recall having a fix for this particular issue, but 6.3.2
generally has some dozens of bugfixes worth having anyways. I'll put
this on my 6.3.3 TODO, but don't think it will appear within the next
three weeks (unless someone else sends a patch). Don't hold your breath,
I'm busy ATM.

-- 
Matthias Andree

Re: [fetchmail-users] Incorrect header line and lost mails

[Rob M. <rob...@gm...>]

On 2/13/06, Frederic Marchal <fre...@wo...> wrote:
> Hello,
>
> I found 15 lines like this in my log this morning and 3 more earlier
> last week:
>
> fetchmail[1226]: incorrect header line found while scanning headers

You'll find threads about this in the list archive.  I don't remember
the outcome, but I've a feeling that 6.3.x handles these (illegal)
messages differently.

It would be worth giving the latest version a try and see if that helps.

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

[fetchmail-users] Incorrect header line and lost mails

[Frederic M. <fre...@wo...>]

Hello,

I found 15 lines like this in my log this morning and 3 more earlier 
last week:

fetchmail[1226]: incorrect header line found while scanning headers


I read the source of fetchmail to learn more about it and I found out 
that any header containing at least one line without a colon and not 
starting with a space is considered illegal as per rfc822 and send to 
oblivion without any other consideration (at least none I could find).

Alas, it turns out that one of the three e-mails I could recover was 
definitely not a spam... I agree it was not fully rfc822 compliant but 
it was a good e-mail according to *my* standards and it could have been 
delivered since the invalid line was a very long TO header from hotmail 
(not rfc822 compliant either) wrapped by my ISP  !

I agree that the sin was in the original e-mail, but is it really the 
purpose of fetchmail to enforce rfc822 so drastically ? With such a 
name, I would expect it to fetch the e-mails and leave any 
"counter-measure" to a program designed for such a task (or at least to 
have an option to drop or accept invalid e-mails).

Now, I'm reluctant to patch and recompile the program myself. Is it 
possible to configure fetchmail to save a copy of every mail in a local 
folder before it is processed ? Could that be achieved with the --plugin 
option and how ?

I'm using fetchmail 6.2.5 as provided by the stable branch of debian.

Frederic







WOW Company or its affiliates do not accept legal responsibility 
for the contents of this message. The views or opinions 
presented are solely those of the author and do not necessarily 
represent those of WOW Company or any of its affiliates.

No contract will be established with WOW Company other than 
in writing (letter) and with the explicit authorization of the director 
of WOW Company.

[fetchmail-users] patch 6.3.2.1 for fetchmail 6.3.2 fixes SEGFAULT with .netrc files

[Matthias A. <mat...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings,

Craig Leres identified a new bug in fetchmail 6.3.2 (that now tries to
erase the .netrc passwords before freeing its memory) that causes
fetchmail to crash right after reading the .netrc file.

Craig also provided a patch that I have accepted into the SVN
repository that will appear in fetchmail 6.3.3. I am including the patch
here (if your mailer is not GnuPG enabled, you need to manually edit all
lines that start with "- -" so that they start with "-" - i. e.  remove
the first minus and blank character on those lines).

I recommend to add this patch as an interim fix to all distribution
packages and on all sites that wish to use .netrc files.

The patch can also be downloaded:

http://download.berlios.de/fetchmail/patch-6.3.2.1-fix-netrc-SIGSEGV.diff

My GnuPG signature for the patch is available at:

http://download.berlios.de/fetchmail/patch-6.3.2.1-fix-netrc-SIGSEGV.diff.asc

Here is the patch:

.......................................................................
Craig Leres identified a problem that makes fetchmail 6.3.2 (only this
version) crash if the .netrc file does not contain a password for a
particular account.

This patch is mostly Craig Leres' work has been committed to the
SVN repository and should be applied to fetchmail 6.3.2 on all sites
that plan to use netrc files:

Index: netrc.c
===================================================================
- --- netrc.c	(Revision 4683)
+++ netrc.c	(Revision 4684)
@@ -314,8 +314,10 @@
 free_netrc(netrc_entry *a) {
     while(a) {
 	netrc_entry *n = a->next;
- -	memset(a->password, 0x55, strlen(a->password));
- -	xfree(a->password);
+	if (a->password != NULL) {
+		memset(a->password, 0x55, strlen(a->password));
+		free(a->password);
+	}
 	xfree(a->login);
 	xfree(a->host);
 	xfree(a);

Sorry for the inconvenience.  -- Matthias Andree, 2006-01-30
.......................................................................

Regards,

- -- 
Matthias Andree
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFD3dmkvmGDOQUufZURAv1+AKDYf5zB++Dyj6buzKS0Fz6W9B70bQCglnYI
F7gplc9LV+Ixh88mq0DSFNI=
=4UM8
-----END PGP SIGNATURE-----

[fetchmail-users] Re: [fetchmail-users] Re: [fetchmail-users] Re: [fetchmail-users] fetchmail doesn´t fetching

[Rob M. <rob...@gm...>]

Please:

1) Keep traffic on the list
2) Don't top post


On 26/01/06, Alex Moreno <al3...@gm...> wrote:
> ok, sorry, i thougth it was -V
>
> -------
> trantor:~# fetchmail -v
> fetchmail: no mailservers have been specified.
> trantor:~# fetchmail -v -v -v
> fetchmail: no mailservers have been specified.
> -------
>
>  is that an error?

Yes.  Try specifying the path to your configuration file:

fetchmail -v -v -v -f /path/to/fetchmailrc

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

[fetchmail-users] Re: [fetchmail-users] Re: [fetchmail-users] fetchmail doesn´t fetching

[Rob M. <rob...@gm...>]

On 26/01/06, Alex Moreno <al3...@gm...> wrote:
> smtp works ok. I´m using qmail some time ago and now, after installing mutt
> i can send mails without problems... The problem appears trying to read the
> incoming mails by pop3... fetchmail is not writing my mails on ~/Mail or any
> other known directory...

Once more, before I give up...

Output of "fetchmail -v -v -v" please.

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-users] fetchmail doesn´t fetching

[Matthias A. <mat...@gm...>]

On Thu, 26 Jan 2006, Alex Moreno wrote:

> smtp works ok. I´m using qmail some time ago and now, after installing mutt
> i can send mails without problems... The problem appears trying to read the
> incoming mails by pop3... fetchmail is not writing my mails on ~/Mail or any
> other known directory...

Please do not top-post.  http://www.netmeister.org/news/learn2quote.html
applies to mailing lists, too.

Fetchmail DOES NOT (as of 6.3.X and older versions) deliver to mailboxes
or maildirs.  Fetchmail hands mail off to your SMTP server (qmail in
your case), which is responsible for delivery.

Check your qmail configuration (its mail log, qmail-showctl
output, qmail-qread and similar) to find out where your mail is stuck or
going to. Double-check if qmail knows what your computer is called
(/var/qmail/control/me, .../locals).


Side note: qmail turned out over the years to be a defective and
unmaintained software package, and rather complicated to set up. My
/personal/ favorite alternatives at this time are Postfix + Dovecot.
Others may be happy with Courier-IMAP, popa3d, Exim, Courier as a whole,
or other systems. (<http://www.postfix.org/>, <http://dovecot.org/>)

Details on qmail flaws are given in
<http://home.pages.de/~mandree/qmail-bugs.html>.

-- 
Matthias Andree

[fetchmail-users] Re: [fetchmail-users] fetchmail doesn´t fetching

[Alex M. <al3...@gm...>]

smtp works ok. I´m using qmail some time ago and now, after installing mutt
i can send mails without problems... The problem appears trying to read the
incoming mails by pop3... fetchmail is not writing my mails on ~/Mail or any
other known directory...

On 1/26/06, Matthias Andree <mat...@gm...> wrote:
>
> Alex Moreno <al3...@gm...> writes:
>
> > and it seems that works, it connects and i can see on the shell some
> > messages... until i try to read with mutt. fetchmail is not writing the
> mails
> > or at least i don´t know where is it writing them.
>
> ...
>
> >   Messages will be SMTP-forwarded to: localhost (default)
>
> Check the log of your SMTP listener (Exim or whatever you're using),
> /var/log/maillog and /var/log/mail are common places, I don't know what
> Debian uses though (read /etc/syslog.conf to find out). This seems to be
> a problem outside fetchmail from your description.
>
> --
> Matthias Andree
>

Re: [fetchmail-users] fetchmail doesn´t fetching

[Matthias A. <mat...@gm...>]

Alex Moreno <al3...@gm...> writes:

> and it seems that works, it connects and i can see on the shell some
> messages... until i try to read with mutt. fetchmail is not writing the mails
> or at least i don´t know where is it writing them.

...

>   Messages will be SMTP-forwarded to: localhost (default)

Check the log of your SMTP listener (Exim or whatever you're using),
/var/log/maillog and /var/log/mail are common places, I don't know what
Debian uses though (read /etc/syslog.conf to find out). This seems to be
a problem outside fetchmail from your description.

-- 
Matthias Andree

[fetchmail-users] Re: [fetchmail-users] fetchmail doesn´t fetching

[Rob M. <rob...@gm...>]

On 25/01/06, Alex Moreno <al3...@gm...> wrote:
> Hi all,
>
> First of all, excuse me if i ask some repetitive question. I´ve been loking
> on internet for the solution or a similar problem with no luck. I´m trying
> to use fetchmail with a pop account. I´ve tried this conf:
>
>
> alex@trantor:~$ cat .fetchmailrc
> #set syslog
>
> poll localhost with proto POP3
> user ale...@mo... there to alex here
>
> alex@trantor:~$
>
>
> and it seems that works, it connects and i can see on the shell some
> messages... until i try to read with mutt. fetchmail is not writing the
> mails or at least i don´t know where is it writing them.
>
> I´m running debian stable and fetchmail 6.2.5.4-1. I attach you some more
> information of fetchmail -V

No, it's not the output of "-V" we need, it's the output of "-v -v -v"
(lower case).

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

[fetchmail-users] fetchmail doesn´t fetching

[Alex M. <al3...@gm...>]

Hi all,

First of all, excuse me if i ask some repetitive question. I´ve been loking
on internet for the solution or a similar problem with no luck. I´m trying
to use fetchmail with a pop account. I´ve tried this conf:


alex@trantor:~$ cat .fetchmailrc
#set syslog

poll localhost with proto POP3
user ale...@mo... there to alex here

alex@trantor:~$


and it seems that works, it connects and i can see on the shell some
messages... until i try to read with mutt. fetchmail is not writing the
mails or at least i don´t know where is it writing them.

I´m running debian stable and fetchmail 6.2.5.4-1. I attach you some more
information of fetchmail -V


------------------------------------------------

alex@trantor:~$ fetchmail -V
This is fetchmail release 6.2.5.4+NTLM+SDPS+SSL+NLS
Fallback MDA: (none)
Linux trantor 2.4.27-2-386 #1 Mon May 16 16:47:51 JST 2005 i686 GNU/Linux
Taking options from command line and /home/alex/.fetchmailrc
Idfile is /home/alex/.fetchids
Fetchmail will show progress dots even in logfiles.
Fetchmail will forward misaddressed multidrop messages to alex.
Options for retrieving from ale...@mo...@localhost:
  True name of server is localhost.
  Password will be prompted for.
  Protocol is POP3.
  All available authentication methods will be tried.
  Server nonresponse timeout is 300 seconds (default).
  Default mailbox selected.
  Only new messages will be retrieved (--all off).
  Fetched messages will not be kept on the server (--keep off).
  Old messages will not be flushed before message retrieval (--flush off).
  Rewrite of server-local addresses is enabled (--norewrite off).
  Carriage-return stripping is disabled (stripcr off).
  Carriage-return forcing is disabled (forcecr off).
  Interpretation of Content-Transfer-Encoding is enabled (pass8bits off).
  MIME decoding is disabled (mimedecode off).
  Idle after poll is disabled (idle off).
  Nonempty Status lines will be kept (dropstatus off)
  Delivered-To lines will be kept (dropdelivered off)
  Fetch message size limit is 100 (--fetchsizelimit 100).
  Do binary search of UIDs during 9 out of 10 polls (--fastuidl 10).
  Messages will be SMTP-forwarded to: localhost (default)
  Single-drop mode: 1 local name(s) recognized.
  No UIDs saved from this host.
alex@trantor:~$

------------------------------------------------


Any help or link would be appreciated.

Thanks a lot

[fetchmail-users] fetchmail security announcement 2006-01 (CVE-2006-0321)

[Matthias A. <mat...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

fetchmail-SA-2006-01: crash when bouncing messages.

Topics:		#1 crash when bouncing a message
		#2 fetchmail 6.2.5.X end of life

Author:		Matthias Andree
Version:	1.0
Announced:	2006-01-22
Type:		free() with bogus pointer
Impact:		fetchmail crashes
Danger:		low
Credits:	Nathaniel W. Turner (bug report)
CVE Name:	CVE-2006-0321
URL:		http://fetchmail.berlios.de/fetchmail-SA-2006-01.txt
		http://bugs.debian.org/348747
Project URL:	http://fetchmail.berlios.de/

Affects:	fetchmail release >= 6.3.0
		fetchmail release <  6.3.2
		fetchmail release candidates 6.3.2-rc1, -rc2 and -rc3

Not affected:	fetchmail release candidate 6.3.2-rc4
		other versions not mentioned here or in the previous
		sections have not been checked

Corrected:	2006-01-19 fetchmail 6.3.2-rc4
		2006-01-22 fetchmail 6.3.2


0. Release history
==================

2006-01-19	internal review draft
2006-01-20	add CVE ID
2006-01-22	release 1.0


1. Background
=============

fetchmail is a software package to retrieve mail from remote POP2, POP3,
IMAP, ETRN or ODMR servers and forward it to local SMTP, LMTP servers or
message delivery agents.

fetchmail ships with a graphical, Python/Tkinter based configuration
utility named "fetchmailconf" to help the user create configuration (run
control) files for fetchmail.


2. Problem description and Impact
=================================

Fetchmail contains a bug that causes itself to crash when bouncing a
message to the originator or to the local postmaster. The crash happens
after the bounce message has been sent, when fetchmail tries to free the
dynamic array of failed addresses, and calls the free() function with an
invalid pointer.  This bug was introduced short before fetchmail 6.3.0
and is not present in the now discontinued 6.2.X series (see below).


3. Workaround
=============

None known at this time.


4. Solution
===========

Download and install fetchmail 6.3.2 or a newer stable release from
fetchmail's project site at
<http://developer.berlios.de/project/showfiles.php?group_id=1824>.


5. End of life announcement
===========================

The aged fetchmail 6.2.5.X branch is discontinued effective immediately.
No further releases from the 6.2.5.X branch will be made.

The new 6.3.X stable branch has been available since 2005-11-30
and will not change except for bugfixes, documentation and message
translations.


A. Copyright, License and Warranty
==================================

(C) Copyright 2006 by Matthias Andree, <mat...@gm...>.
Some rights reserved.

This work is licensed under the Creative Commons
Attribution-NonCommercial-NoDerivs German License. To view a copy of
this license, visit http://creativecommons.org/licenses/by-nc-nd/2.0/de/
or send a letter to Creative Commons; 559 Nathan Abbott Way;
Stanford, California 94305; USA.

THIS WORK IS PROVIDED FREE OF CHARGE AND WITHOUT ANY WARRANTIES.
Use the information herein at your own risk.

END OF fetchmail-SA-2006-01.txt
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFD04VrvmGDOQUufZURAuktAKD62llCNM2NlccXzI8NOqU9/sD1hgCeLfw0
fu8vMb8DiolqOUdQU6vVAP0=
=tfMu
-----END PGP SIGNATURE-----

[fetchmail-users] fetchmail 6.3.2 stable release with security relevant bugfixes

[Matthias A. <mat...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings,

I am announcing the release of fetchmail 6.3.2.

This release fixes a denial of service bug/fetchmail crash after sending
a bounce, adds a Maillennium (Comcast) workaround and fixes other bugs.
(The security announcement will be mailed separately.)

This is a recommended upgrade for all users of any previous fetchmail versions.

The software is available from:
<https://developer.berlios.de/project/showfiles.php?group_id=1824&release_id=8784>

These are the relevant changes in 6.3.2 since 6.3.1,
unless otherwise noted, changes to this release were made by Matthias Andree.

# SECURITY FIX IN THIS RELEASE
* CVE-2006-0321: Fix segfault or bus error after bouncing a message.  This bug
  was introduced into 6.3.0 when removing alloca(); it caused fetchmail to free
  random memory.  Reported by Nathaniel W. Turner, Debian Bug#348747.
  See fetchmail-SA-2006-01.txt

# DEPRECATED FEATURES AND MAJOR INCOMPATIBLE CHANGE ADVANCE WARNINGS
* The --enable-fallback (fall back to MDA if MTA unavailable) may be removed
  from a future fetchmail release.

# INCOMPATIBLE CHANGES
* Automatically disable the POP3 TOP command if the greeting string contains
  "Maillennium POP3/PROXY server", which is used by comcast and known to
  truncate messages after 80 kByte. Fall back to RETR, and complain if we had
  used TOP otherwise (the warning is printed only once per server in daemon
  mode).  Suggested by Ed Wilts.
  *Note* that this means messages are marked read on these servers, which is a
  deviation from how 6.3.1 behaved, but we have no alternative, comcast haven't
  fixed this bug in years.  Preventing the loss of the remainder of the message
  justifies this incompatible fix.
* fetchmail, since 6.3.0, requires write permission to the directory holding the
  idfile. See the amendment in the 6.3.0 MAJOR INCOMPATIBLE CHANGES section
  below for details. The manual page was updated.

# CHANGES RELEVANT TO PACKAGERS
* The outdated BUGS document was removed from the distribution.
* Added fetchmail-SA-2006-01.txt to the distribution.

# BUG FIXES
* SMTP/LMTP cleanup to fix these two bugs:
  - switch back to SMTP after having tried LMTP hosts (multiple smtphost hosts)
  - switch back to LMTP after sending a bounce.
  The patch removes the global state variable that was the root of this problem.
  Patch by Sunil Shetye. (MA)
* Don't complain about fetchall keep in --configdump mode. Bug introduced in
  6.3.0.
* fetchmailconf.py: Fix novice help for Poll interval and fetchall.
  Reported by Justin Pryzby, Debian Bug #344978.
* Some verbose output disappeared in debug mode. Adding further -v options would
  alternate between verbose and debug mode. debug mode now comprises all verbose
  output, and adding more -v options does not switch back from debug to verbose
  mode.
* fetchmail.man: Fix accented characters in Héctor García's name. Merged from
  downstream debian/patches/01_man_page.dpatch.
* Add missing --help text for "--sslcertck" option.
* fetchmailconf.py: Accept --help and --version.
* fetchmail --version  now prints the copyright notice.
* don't complain about READ-ONLY IMAP folders in --fetchall --keep mode.
  Reported Alexander Zangerl, Debian Bug#348964.
* the RPM .spec file now generates a -debuginfo package on newer RPM versions.

Regards,

- -- 
Matthias Andree
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFD04UzvmGDOQUufZURApk+AJ46zXfGAd0jHsbxziZ7JQpPugjJKwCeM2xW
DSAxh7uWlnM7Teolv4wF+BE=
=hrZ0
-----END PGP SIGNATURE-----

Re: [fetchmail-users] Exit Codes, a script and daemon

[Matthias A. <mat...@gm...>]

fink <fi...@ra...> writes:

> I would like Fetchmail to check my mailserver periodically for new mail.
>
> I created a small script which just needs one argument (the exit code
> from fetchmail):
>
> if [ $1  = 1]
> then
>   echo 1 > /proc/acpi/asus/mled
> else
>   echo 0 > /proc/acpi/asus/mled
> fi
> exit
>
> The trouble is now how to get the exit code passed to the script: When
> fetchmail is running deamonised it never returns and thus the exit
> code does not exist.

Well, it would trash its value on the next run anyhow, besides that I
think you got the logic in reverse. Exit code 0 = messages
fetched/waiting, Exit code 1 = no mail.

I'd suggest to use a cron job for such a setup and not daemon mode, that
way you can perhaps set a flag file and LED if fetchmail exits 0, ignore
fetchmail exiting 1, and reset the flag file when your mailer has been
started or quit.

-- 
Matthias Andree

[fetchmail-users] Exit Codes, a script and daemon

[fink <fi...@ra...>]

Hi,

I would like Fetchmail to check my mailserver periodically for new mail.

I created a small script which just needs one argument (the exit code
from fetchmail):

if [ $1  = 1]
then
  echo 1 > /proc/acpi/asus/mled
else
  echo 0 > /proc/acpi/asus/mled
fi
exit

The trouble is now how to get the exit code passed to the script:
When fetchmail is running deamonised it never returns and thus the exit
code does not exist.

Is it possible in some odd way to pass the result of the fechmail mail
check to the program called by the "postconnect" command ?

The idea I had was something like this:

[poll SERVERNAME with proto IMAP user 'USER' password 'PASSWD'
postconnect '/usr/bin/servicemled $?']

Cheers
RaceMouse

[fetchmail-users] Exit Codes, a script and daemon

[fink <fi...@ra...>]

Hi,

I would like Fetchmail to check my mailserver periodically for new mail.

I created a small script which just needs one argument (the exit code
from fetchmail):

if [ $1  = 1]
then
  echo 1 > /proc/acpi/asus/mled
else
  echo 0 > /proc/acpi/asus/mled
fi
exit

The trouble is now how to get the exit code passed to the script:
When fetchmail is running deamonised it never returns and thus the exit
code does not exist.

Is it possible in some odd way to pass the result of the fechmail mail
check to the program called by the "postconnect" command ?

The idea I had was something like this:

[poll SERVERNAME with proto IMAP user 'USER' password 'PASSWD'
postconnect '/usr/bin/servicemled $?']

Cheers
RaceMouse

Re: [fetchmail-users] Re: Re: WARNING: Running as root is discouraged.

[Matthias A. <mat...@gm...>]

Michelle Konzack <lin...@fr...> writes:

>> Better pick: Replace procmail by maildrop, compile it so that the
>> fetchmail user is allowed to use -d, and install maildrop setuid-root.
>> It will be a better match for courier-imap anyhow. (Maildir++, quota,
>> same user database and all the nice parts of integration.)
>
> Unfortunatly "courier-maildrop" is no option for me, because I have
> setup a whole "office" web interface with php5 and the filters are
> set directly in procmail.
>
> The syntax of maildrop is driving me crazy and I have not the time
> and LUST to recode all from scratch again!

D'accord.

At least there are ways to do it without making fetchmail UID root.

Personally, I prefer feeding everything through Postfix, because that
gives me Amavisd-new's virus scanning capabilities and decent logging.

-- 
Matthias Andree

[fetchmail-users] Re: message was not the expected length errors

[Frederik <fr...@gm...>]

On 1/19/06, Frederik <fr...@gm...> wrote:
> On 12/24/05, Sunil Shetye <sh...@bo...> wrote:
> > Quoting from Frederik's mail on Fri, Dec 23, 2005:
> > > Since tonight, I'm suffering from "message [...] was not the expected
> > > length" errors. I see it on one mailbox on my ISP (the other mailbox is
> > > unaffected though), and it makes fetchmail completely fail to retrieve
> > > mails from the mailbox concerned.
>
> > Please try adding the "forcecr" option to your fetchmailrc.
> >
> > In case it doesn't work, please send the output of "fetchmail -V" and
> > the details of your smtp server.
> >
> > Is the "message ... was not the expected length" error coming on every
> > mail or on specific mails only?
>
> The problem disappeared as suddenly after it appeared. Well, till
> today at least, because now exactly the same problem started again.

And a bit after I wrote this, all started to work again.

That means: while doing a fetchmail -v -v, it still had the error
messages about inccorret size, but it continued to download
everything. After that, I restarted fetchmail, and mails are now being
downloaded without errors. Weird...


--
Surf veilig en snel met de Mozilla Firefox web browser
http://spreadfirefox.com/community/?q=affiliates&amp;id=617&amp;t=1

[fetchmail-users] Re: message was not the expected length errors

[Frederik <fr...@gm...>]

On 12/24/05, Sunil Shetye <sh...@bo...> wrote:
> Quoting from Frederik's mail on Fri, Dec 23, 2005:
> > Since tonight, I'm suffering from "message [...] was not the expected
> > length" errors. I see it on one mailbox on my ISP (the other mailbox is
> > unaffected though), and it makes fetchmail completely fail to retrieve
> > mails from the mailbox concerned.

> Please try adding the "forcecr" option to your fetchmailrc.
>
> In case it doesn't work, please send the output of "fetchmail -V" and
> the details of your smtp server.
>
> Is the "message ... was not the expected length" error coming on every
> mail or on specific mails only?

The problem disappeared as suddenly after it appeared. Well, till
today at least, because now exactly the same problem started again.

It seems to happen on all messages, deleting the message which causes
the problem, just makes it halt on the next message.

I cannot understand why it works fine so lang, and then it starts to
go mad like this. Another e-mail account I have from the same ISP,
works still fine.

Also something which is not yet clear to me: is it a problem that my
ISP's POP3 server is giving wrong info about the size of the mail to
fetchmail, or is it my own local MTA that complains that the e-mail it
receives is not the size fetchmail said it is? In the first case, it
would not matter much which MTA I am using locally?

Adding forcerc did not help.

My own setup: exim 4.50 from Debian Sarge (using sa-exim and clamav,
but that should not matter much). Problem happens with both Sarge's
fetchmail 6.2.5-12sarge3 and recompiled 6.3.1-4 from unstable.

# fetchmail -V
fetchmail: WARNING: Running as root is discouraged.
This is fetchmail release 6.3.1+NTLM+SDPS+SSL+NLS
Fallback MDA: (none)
Linux Luna 2.6.14-cks4 #1 Wed Nov 9 16:10:51 CET 2005 i686 GNU/Linux
Taking options from command line
No mailservers set up -- perhaps /root/.fetchmailrc is missing?


Is not there simply a way to make it *ignore* this problem? Actually
fetchmail is completely unusable to me now, as I cannot access my
e-mail anymore :-(

--
Surf veilig en snel met de Mozilla Firefox web browser
http://spreadfirefox.com/community/?q=affiliates&amp;id=617&amp;t=1