fetchmail-users — User questions and discussion

[fetchmail-users] fetchmail 6.2.5.4 released (security bugfix) (legacy)

[Matthias A. <mat...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings,

I am announcing the release of fetchmail 6.2.5.4. For the main part, the
updated fetchmailconf that fixed CVE-2005-3088 (the password exposure
problem) is now part of the tarball, many build problems with 6.2.5 and
6.2.5.2 were fixed and the infamous "timeout" bug with IMAP that only
showed with several servers (for instance, older CommuniGate; Debian
Bug#314509) was also fixed. CVE-2005-2335 was already fixed in 6.2.5.2,
the fix is also part of 6.2.5.4. See below for details.

The software is available from:
<https://developer.berlios.de/project/showfiles.php?group_id=1824&release_id=7976>

fetchmail-6.2.5.X is a security fix branch that forked off
fetchmail-6.2.5. It does not change for anything but security and the
most severe bug fixes. Note that no 6.2.5.X security audits are planned
except when a particular bug is reported, and that 6.2.5.X is unsafe to
use on some systems, particularly those that lack a *working and secure*
snprintf implementation.

This 6.2.5.X branch is ONLY intended for packages for systems that
cannot move forward to a newer version for stability policies, such as
Debian stable.  Note that this branch may be discontinued alongside the
official 6.3.0 release without further notice.

End users and all other systems should therefore use a current
fetchmail-6.2.9-rc* release candidate or, if available at that time,
6.3.X or newer release.

These are the relevant changes since (and excluding) 6.2.5.2:

* SECURITY FIX CVE-2005-3088: fetchmailconf: fix password exposure: use
  umask 077 before opening output file and restore umask later.
* Critical fix: fix IMAP timeouts, counting message count down on
  servers that do not send EXISTS counts after EXPUNGE. Debian Bug#314509.
* On FreeBSD, add /usr/local/include to CPPFLAGS so that libintl.h is found.
* Avoid automatically picking up HESIOD implementations that lack
  hesiod_getmailhost, such as the one in FreeBSD's base system.
* Fix makedepend for separated build (where the build is not run from
  the source directory), but prevent packaging from separated build, it
  yields bogus results.
* Fix resolv.h autodetection.
* Add +HESIOD to version printout if appropriate.
* Ship pre-built rcfile_l.c for systems that don't have flex.
* Also ship pre-built rcfile_y.[ch] for systems that don't have flex,
  yacc or bison.
* Build environment: Update included gettext. Fix
  --with-included-gettext. Fix parallel build (make -j). Fix "always
  rebuild fetchmail" syndrome.
* Do not link against -ll or -lfl (not needed).

Regards,

- -- 
Matthias Andree
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFDfTM3vmGDOQUufZURAtu2AKCyPEETBn+q1vTQBMF3eHCR0UlEhQCdE7Os
i1DBvMPM0ry0ufynC+0QjKE=
=fUW6
-----END PGP SIGNATURE-----

[fetchmail-users] fetchmail 6.2.9-rc8 available (bug fixes)

[Matthias A. <mat...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings,

I have just released fetchmail 6.2.9-rc9, the hopefully last release
candidate before 6.3.0, with minor bug fixes.

For distributors, please upload this to every unstable/testing/current
distribution that you have (do not upload to "stable", it changes
behavior in a few places) to expose this to wider testing.

Changes since -rc8 are:
* The default for --smtphost is now always "localhost" regardless of
  authentication types and protocols, so as to simplify configurations for
  workstations where the SMTP daemon only listens on the loopback interface.
  Sunil Shetye & Matthias Andree
* Man page: update --smtphost documentation. Sunil Shetye, Matthias Andree.
* Man page: update --smtpaddress documentation. Sunil Shetye.
* Fix several memory leaks and bugs in the SMTP/LMTP retry logic where
  fetchmail confused UNIX and Internet domain sockets. Sunil Shetye.
* Man page (BUGS): document that passwords are length limited. Matthias Andree
* Man page: Document that quoted strings that run across line boundaries
  contain the control characters (CR or LF). Document explicitly the backslash
  escape sequences and their differences from the escape sequences used in the
  C programming language.  Matthias Andree
* Fix segfault when run control file ends with a backslash inside an
  unterminated quoted string. Matthias Andree.
* In quoted strings, support backslash as last character on a line to join the
  following line to the current. Matthias Andree.
* Parsing untagged IMAP responses is more robust now. Matthias Andree.
* Man page: Remove some procmail praises in --mda documentation, suggest
  maildrop instead, warn of procmail fallthrough behavior. Matthias Andree.
* Man page: Revise AUTHORS and SEE ALSO sections. Matthias Andree.
* Updated translations: Albanian [sq] (Besnik Bleta), Catalan [ca] (Ernest
  Adrogué Calveras), Czech [cs] (Miloslav Trmac), German [de] (MA),
  Spanish (Castilian) [es] (Javier Kohen), French [fr] (MA),
  Polish [pl] (Jakub Bogusz), Russian [ru] (Pavel Maryanov).
* In oversized warning messages, print the account name, too. Fixes Debian
  Bug#213299. Sunil Shetye (MA).
* Fix '!: command not found' in configure script on nonconformant shells
  such as Solaris' /bin/sh.
* The IMAP trail-eating fix in -rc8 was broken and fixed by Sunil
  Shetye.

I seek everyone to test it out, report remaining bugs, update outdated
translations (send your translated .po files to the translation
project's robot, it'll forward your translation to the -devel list) or
send patches for documentation or report inconsistencies (including
formatting!) in the documentation.

The software is available from:
<https://developer.berlios.de/project/showfiles.php?group_id=1824&release_id=8049>
<http://home.pages.de/~mandree/fetchmail/>

Regards,

- -- 
Matthias Andree
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFDfTEivmGDOQUufZURAhvXAKD6MF7T8MhWvpjlJ/11PsRjaVjmRgCgnry2
oGSAFM4uTU3QdG6IZB1FgcA=
=82bb
-----END PGP SIGNATURE-----

[fetchmail-users] Re: [PATCH] Re: oversized messages notification provides server info but lacks user info

[Matthias A. <mat...@gm...>]

Sunil Shetye schrieb am 2005-11-17:

> Quoting from Matthias Andree's mail on Thu, Nov 17, 2005:

> > Unless someone posts a patch do this, suitably to apply against the SVN
> > repository at <https://decoy.wox.org/svn/fetchmail/trunk/>, this is not
> > going to make fetchmail 6.3.0.
> 
> ===================================================================================
> Index: fetchmail/driver.c
> ===================================================================
> --- fetchmail/driver.c	(revision 4449)
> +++ fetchmail/driver.c	(working copy)

Wow, that was fast. Thanks a lot, committed for 6.2.9-rc9.

-- 
Matthias Andree

[fetchmail-users] [PATCH] Re: oversized messages notification provides server info but lacks user info

[Sunil S. <sh...@bo...>]

Quoting from Matthias Andree's mail on Thu, Nov 17, 2005:
> Gregor Zattler <te...@un...> writes:
> 
> > [please Cc: me, i'm not subscribed to this list]
> >
> > Hi,
> >
> > this is a feature wish i posted as a debian bug report but
> > nothing happend.  Now I read (http://lwn.net/Articles/89403/)
> > there are plans to revive the development of fetchmail and so I
> > post this here...
> 
> Unless someone posts a patch do this, suitably to apply against the SVN
> repository at <https://decoy.wox.org/svn/fetchmail/trunk/>, this is not
> going to make fetchmail 6.3.0.

===================================================================================
Index: fetchmail/driver.c
===================================================================
--- fetchmail/driver.c	(revision 4449)
+++ fetchmail/driver.c	(working copy)
@@ -338,12 +338,12 @@
     stuff_warning(NULL, ctl, "%s", "");
     if (ctl->limitflush)
 	stuff_warning(NULL, ctl,
-		GT_("The following oversized messages were deleted on the mail server %s:"),
-		ctl->server.pollname);
+		GT_("The following oversized messages were deleted on server %s account %s:"),
+		ctl->server.pollname, ctl->remotename);
     else
 	stuff_warning(NULL, ctl,
-		GT_("The following oversized messages remain on the mail server %s:"),
-		ctl->server.pollname);
+		GT_("The following oversized messages remain on server %s account %s:"),
+		ctl->server.pollname, ctl->remotename);
 
     stuff_warning(NULL, ctl, "%s", "");
 
===================================================================================

-- 
Sunil Shetye.

[fetchmail-users] Re: unsafe MDA procmail recommended by fetchmail

[Michelle K. <lin...@fr...>]

Hello Sunil,

Am 2005-11-14 18:23:58, schrieb Sunil Shetye:

> procmail has a way of returning with an error. You may append the
> lines below to your procmailrc for error handling.
> 
> ==========================================================================
> 
> # if procmail has reached here, delivery has failed. return with a
> # temporary failure code from <sysexits.h>.
> # 75 = EX_TEMPFAIL
> EXITCODE=75
> :0
> /dev/null
> ==========================================================================

This is, what I was searching for.

> WARNING: the above recipe works only when the program invoking
> procmail handles the exit code gracefully. Otherwise, all your mails
> will be trashed.
> 
> fetchmail will not delete these mails. Even, most SMTP servers should

I use only fetchmail + courier-mta while the MTA receives
regulary messages and fetchmail collect (at $USER request)
messages from other ISP's

> queue up such mails for a few days before bouncing them back. But, if
> you have other programs invoking procmail, you will need to check if
> they handle such exit codes gracefully.

So I think, courier-mta will work to.

Greetings
Michelle

-- 
Linux-User #280138 with the Linux Counter, http://counter.li.org/
Michelle Konzack   Apt. 917                  ICQ #328449886
                   50, rue de Soultz         MSM LinuxMichi
0033/3/88452356    67100 Strasbourg/France   IRC #Debian (irc.icq.com)

Re: [fetchmail-users] [texmex@uni.de: fetchmail: oversized messages notification provides server info but lacks user info]

[Matthias A. <mat...@gm...>]

Gregor Zattler <te...@un...> writes:

> [please Cc: me, i'm not subscribed to this list]
>
> Hi,
>
> this is a feature wish i posted as a debian bug report but
> nothing happend.  Now I read (http://lwn.net/Articles/89403/)
> there are plans to revive the development of fetchmail and so I
> post this here...

Unless someone posts a patch do this, suitably to apply against the SVN
repository at <https://decoy.wox.org/svn/fetchmail/trunk/>, this is not
going to make fetchmail 6.3.0.

-- 
Matthias Andree

Re: [fetchmail-users] feedback on failed delivery

[Thomas W. <to...@to...>]

I wrote:
> > 1 message for deblnss01a/demsn702 at blnss35a.
> > reading message deblnss01a/demsn702@blnss35a:1 of 1 (902 header octets)  (6 body octets) Segmentation Fault
> Crash.

Matthias Andree wrote:
> Can you redo the situation and obtain a backtrace for me, ...
Unfortunately, the crash is not reproducible anymore today, sorry.

Now it's:
> 1 message for deblnss01a/demsn702 at blnss35a.
> reading message deblnss01a/demsn702@blnss35a:1 of 1 (3257 header octets) ... (2542 body octets) ..fetchmail: MDA returned nonzero status 255
>  not flushed
which is fine. Just make it better visible by putting the error notice 
on a new line.

Actually, running remotely (rsh ... fetchmail), the output turns into:
> fetchmail: MDA returned nonzero status 255
> 1 message for deblnss01a/demsn702 at blnss35a.
> reading message deblnss01a/demsn702@blnss35a:1 of 1 (3257 header octets) ... (2542 body octets) .. not flushed
which indicates that stdout and stderr gets merged in an unsynchronised 
way. Probably a flush at some point (flush stdout before any error output) 
would fix this.

Thanks and best regards,
Thomas Wolff

Re: [fetchmail-users] feedback on failed delivery

[Matthias A. <mat...@gm...>]

Thomas Wolff schrieb am 2005-11-16:

> > 1 message for deblnss01a/demsn702 at blnss35a.
> > reading message deblnss01a/demsn702@blnss35a:1 of 1 (902 header octets)  (6 body octets) Segmentation Fault
> Crash.

Can you redo the situation and obtain a backtrace for me, from your core
file? See <http://leafnode.sourceforge.net/doc_en/FAQ.html#backtrace>
for instructions. The document is for leafnode, but the procedure is the
same for fetchmail.

NOTE: If you had installed fetchmail with "make install-strip", please
reinstall with "make install" (do not strip), and you may need to enable
core dumps. In bash, use ulimit (see help ulimit) to let the system
allow core files, and you need to run fetchmail with "-v -d0" so
fetchmail itself does not disable core files again.

You can also run fetchmail from gdb, i. e.

(in the shell)
gdb fetchmail
(inside gdb)
run -v -N -d0 --nosyslog

(you can pass more options to run, but do leave -v -N and -d0 in).

Thank you.

-- 
Matthias Andree

Re: [fetchmail-users] WARNING: Running as root is discouraged.

[Matthias A. <mat...@gm...>]

Thomas Wolff schrieb am 2005-11-11:

> There is this new warning "WARNING: Running as root is discouraged." 
> which is somehow disturbing.

Yes, and that is deliberate.

Networking clients should never run with root privileges, and fetchmail
is no exception. Fetchmail does not need root privileges to forward
mail, the only conceivable scenario where it needs root privileges is
one where it calls an MDA for a different user.

In this scenario, fetchmail should use IPC (Unix domain socket, named
pipe or similar) to talk from one unprivileged process to another.  This
won't happen in the near future though, unless someone else finds the
time to do that.

> I thought fetchmail is also intended as a system installation 
> for multi-user mail retrieval so it's actually a designed mode to 
> run fetchmail as root (and the message seems to be unconditional, 
> looking at the code).

That was how it originally worked, but the fetchmail 6.2.5 code Rob,
Graham and I took over was utterly unsuitable to run as root, as you can
read in <http://fetchmail.berlios.de/fetchmail-SA-2005-01.txt>
(CVE-2005-2335).

The 6.2.5 code was so bold as to fall back to sprintf on systems where
snprintf was unavailable, which opened up even more buffer overrun
possibilities. 6.3.0 uses trio_snprintf instead on such systems.

There is a reason why I recommend 6.2.9-rc* to everyone, and that is
many more bugfixes.

> Also I don't need this message when I retrieve my personal mailbox while 
> regularly working as root on my home machine; this may not be 

I don't care if you work as root on your home machine and fetchmail
pesters you with this warning.

On some day in the future, fetchmail might have a model where it can
give up privileges for good in the process that talk to the network and
only retain privileges for MDA based setups where someone needs to run
the MDA of somebody else's. This model will certainly not be the
seteuid() swapping crap as that's not secure. Look at sendmail to learn
some lessons about this.

> recommended in general but it's not fetchmail to tell me that every 
> time I retrieve my mail.

Evidently, it is. Sorry for my being so boldfaced.

> Please remove the message.

Refused. Please fix the way you work. This is not Windows, and sudo(1)
works well for the few moments when you need more power than the regular
user.  <http://www.courtesan.com/sudo/>

fetchmail is a tool for the end user who is usually untrained about
systems administrations, putting up obstacles and annoyances while he
works as root is the right thing to do.

-- 
Matthias Andree

Re: [fetchmail-users] Re: unsafe MDA procmail recommended by fetchmail

[Matthias A. <mat...@gm...>]

Thomas Wolff schrieb am 2005-11-10:

> >        ORGMAIL     Usually  the  system  mailbox  (ORiGinal MAILbox).  If, for
> >                    some obscure reason (like `filesystem full') the mail could
> >                    not  be  delivered,  then  this  mailbox  will  be the last
> >                    resort.  If procmail fails to save the mail in here  (deep,
> >                    deep  trouble  :-),  then  the mail will bounce back to the
> >                    sender.
> What would "bounce back" actually mean in an environment with no working 
> or properly configured sendmail interface?

Do not worry, fetchmail will not delete the message if you use a
reliable retrieval protocol such as POP3 + UIDL and retry on the next
run.

> Actually, I think there is also an option missing to prevent procmail 
> from attempts to bounce a mail after failed delivery. As I indicated 
> above, not all system configuration provide working outgoing mail 
> (at least not through the standard interfaces). I, for instance, use 
> ssmtp with a script wrapper; /usr/sbin/sendmail is not working here.
> In this case (not bouncing failed mail), procmail should of course 
> return an error code so e.g. fetchmail notices and doesn't delete 
> the mail from the server.

In that particular case, ! redirections will not work either (unless you
override SENDMAIL).

-- 
Matthias Andree

Re: [fetchmail-users] Re: unsafe MDA procmail recommended by fetchmail

[Matthias A. <mat...@gm...>]

It is a well-known fact that procmail is too demanding to configure
properly for the average user, and the documentation is insufficient in
this respect.

What have the procmail maintainers done in the past four years to remedy
this? Nothing. No documentation, no code changes, no new release.

With my system administrator hat on, I have taken procmail away from
my users and given them maildrop instead, years ago, and I think
fetchmail 6.3.0 should go this route, as suggested by Thomas Wolff.

To use procmail safely, you have to add something after every delivering
recipe, that either you or Philip posted to the postfix-users list ages
ago when I complained:

:0e
{
  EXITCODE=75
  HOST
}

This is cumbersome, unintuitive, in none of the prominent tutorials, and
ugly as sin. The ugliness at that time caused embarrassed silence...

To the user: use maildrop, it does The Right Thing[TM] out of the box,
namely exiting with EX_TEMPFAIL as per /usr/include/sysexits.h

What's worse, procmail shows many incomplete examples that confuse the
user.

This procmail fallthrough behavior that makes the many popular and
shipped (see procmailex!) examples so unusable that users have to start
chasing their mail in bad weather is what makes me support Thomas's
original suggestion in spite of the problem being his own fault.

The problem gets worse when multiple rules may match a message. In that
case, the order of matches makes a difference, and mail may end up in
the wrong box, when, for instance by a POP3 or IMAP server shuffling its
mailbox around, /var/mail is TEMPORARILY out of space - the first rule
matches, attempt FOR DELIVERY fails, the mail server lets go of the
temporary file of the other user, meanwhile procmail falls through, the
second rule matches and delivery succeeds -> boom, the mail is in the
wrong destination.

Just unsetting DEFAULT/ORGMAIL as you suggest below is not sufficient to
fix this. It's also a long shot from being anywhere close to a solution.

> > User feedback and documentation should be improved,
> 
> Please provide suggestions as to what wording in the logfile would have made
> it more clear?

The logfile should be enabled by default, and right in the face of the
user: $HOME/procmail.log

> Same question for the manpage; in what way should we rephrase it?

The code must change, not the manual page. In order not to break
existing setups, you should make new "EX_TEMPFAIL right away" behavior a
compile-time switch that defaults to off, or you should make it a
feature that is so prominently documented it warrants a version bump to
procmail 4.

> > though, and maybe
> > this case of redirection should not be applied at all
> 
> Mail wants to arrive.  That means, if we can salvage the mail, instead of
> bouncing it, then that is the preferred solution.  Rescueing instead of
> giving up is the proper action.  It basically guards the user from his
> own lack of knowledge on the situation (and since the user can't
> be asked at that point, procmail has to make an intelligent decision).

procmail's undeterministic behavior is worse than a bounce.

A message that is delivered to the wrong place is a message lost for the
user, and at best one that causes additional work. Why use a filter at
all if it doesn't work?  procmail's "intelligence" is a disservice to
users. We've been through this before, and you fail to see the point.

Imagine someone filtering his important messages (from VIP, with special
subject) to a particular "important" mailbox, to see the mail early, and
only checks his default mailbox once in a while. He'll "lose" that
important mail and only find it later. Is *that* smart? No, it isn't.

maildrop takes care to exit with EX_TEMPFAIL unless there is a really
clear permanent problem (such as user not found in virtual setups). That
makes the message stay in the queue, and fetchmail will thus leave it on
the server and retry later, and that is what users expect.

> > (or be configurable)
> 
> It is, just put the following lines anywhere in your .procmailrc file:
>   DEFAULT
>   ORGMAIL
> 
> Which will unset the DEFAULT and ORGMAIL variables (and prevent
> evasive action in case of system or user error).

As shown above, this does not work for any setups except perhaps (full
example) this trivial one:

DEFAULT
ORGMAIL

:0:
everything

-- 
Matthias Andree

[fetchmail-users] feedback on failed delivery

[Thomas W. <to...@to...>]

Hello,
I'm bothering you again with the issue of error behaviour, this time 
it is indeed about fetchmail.
As I'm still trying to achieve a reliable overall mail configuration, 
I just tried the following in .fetchmailrc:
mda "false"

As expected, fetchmail leaves the mail on the server but user feedback 
is not good, and there is even a crash:

With fetchmail release 6.2.9-rc5+SSL:

> 1 message for deblnss01a/demsn702 at blnss35a.
> reading message deblnss01a/demsn702@blnss35a:1 of 1 (902 header octets)  (6 body octets) fetchmail: SIGPIPE thrown from an MDA or a stream socket error
> fetchmail: socket error while fetching from blnss35a
> fetchmail: Query status=2 (SOCKET)
If you look exactly, you find MDA mentioned and can draw your conclusion, 
but it's not really while "reading" the message or while "fetching" it 
(at least it's not in the process of fetching, so this is confusing), 
and it's not a socket error either, I think.

With fetchmail release 6.2.9-rc8+SSL

> 1 message for deblnss01a/demsn702 at blnss35a.
> reading message deblnss01a/demsn702@blnss35a:1 of 1 (902 header octets)  (6 body octets) Segmentation Fault
Crash.


Kind regards,
Thomas Wolff

Re: [fetchmail-users] Re: unsafe MDA procmail recommended by fetchmail

[Thomas W. <to...@to...>]

I had confused:
>> # if procmail has reached here, delivery has failed. return with a
>> ...
> Fetchmail needs ...

Jakob Hirsch wrote:
> This is not a fetchmail issue and this whole discussion is unrelated to
> fetchmail. If you are really interested in improving procmail's error
> behaviour, you should move this to the procmail-list.
Sure, sorry for my confusion.

> As a sidenote: I doubt procmail is so bad, but you never know. Anyway, I used
> maildrop for quite a while. Main reason was its non-obfuscating filter
> language, but it also has a proper error handling. And I never needed anything
> like formail to use it as MDA.
Thanks for the hint, it looks promising.

Kind regards,
Thomas Wolff

Re: [fetchmail-users] Re: unsafe MDA procmail recommended by fetchmail

[Jakob H. <jh...@pl...>]

Thomas Wolff wrote:

>> # if procmail has reached here, delivery has failed. return with a
>> # temporary failure code from <sysexits.h>.
>> # 75 = EX_TEMPFAIL
>> EXITCODE=75
>> :0
>> /dev/null
> So that may work, but it is really an obscure trick.
> Fetchmail needs to have an easy-to-use, well-documented facility to
> configure its error behaviour in an obvious way.

This is not a fetchmail issue and this whole discussion is unrelated to
fetchmail. If you are really interested in improving procmail's error
behaviour, you should move this to the procmail-list.

As a sidenote: I doubt procmail is so bad, but you never know. Anyway, I used
maildrop for quite a while. Main reason was its non-obfuscating filter
language, but it also has a proper error handling. And I never needed anything
like formail to use it as MDA.

> That other program may often be formail. While formail -s does
> propagate an error return code here, this is not reliable because it's
> not documented. Should be fixed.

By whom? Certainly not by any fetchmail maintainer.

[fetchmail-users] Re: unsafe MDA procmail recommended by fetchmail

[Thomas W. <to...@to...>]

Sunil Shetye wrote:
> Quoting from Michelle Konzack's mail on Mon, Nov 07, 2005 at 07:29:19PM +0100:
> > ...
> > I think, procmail should have an config option to stop
> > this behaviour and return an error instead.
> > ...
> 
> procmail has a way of returning with an error. You may append the
> lines below to your procmailrc for error handling.
> ...
> ==========================================================================
> 
> # if procmail has reached here, delivery has failed. return with a
> # temporary failure code from <sysexits.h>.
> # 75 = EX_TEMPFAIL
> EXITCODE=75
> :0
> /dev/null
> ==========================================================================

So that may work, but it is really an obscure trick.
Fetchmail needs to have an easy-to-use, well-documented facility to 
configure its error behaviour in an obvious way.


> WARNING: the above recipe works only when the program invoking
> procmail handles the exit code gracefully. Otherwise, all your mails
> will be trashed.
> 
> fetchmail will not delete these mails. Even, most SMTP servers should
> queue up such mails for a few days before bouncing them back. But, if
> you have other programs invoking procmail, you will need to check if
> they handle such exit codes gracefully.

That other program may often be formail. While formail -s does 
propagate an error return code here, this is not reliable because it's 
not documented. Should be fixed.


Kind regards,
Thomas Wolff

Re: [fetchmail-users] RE: Fetchmail error!

[Rob M. <rob...@gm...>]

On 13/11/05, deven barhate <dev...@gm...> wrote:
> fetchmail: POP3> USER us...@so...
> fetchmail: POP3< +OK User name accepted, password please
> fetchmail: POP3> PASS
> fetchmail: POP3< -ERR Bad login
> fetchmail: Bad login
> fetchmail: Authorization failure on us...@so...
> fetchmail: POP3> QUIT
> fetchmail: POP3< +OK Sayonara
> fetchmail: 6.2.0 querying somemailserver.com (protocol POP3) at Sun 13 Nov
> 2005 06:36:18 PM IST: poll completed
>
> Now the thing is I can access the mails through webmail over the internet
> with same USERNAME and PASSWD.
>
> then why fetchmail is not able to fetch the mails?

Have you tried using just "user" (dropping "@somemailserver.com")? 
Are there any special (ie not A-Z, a-z or 0-9) characters in your
password?

Finally, try the latest version of fetchmail, 6.2.0 is old and has a
number of known security issues.  You should either be on 6.2.5.2 or
the latest 6.3.0 release candidate.

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

[fetchmail-users] Re: unsafe MDA procmail recommended by fetchmail

[Sunil S. <sh...@bo...>]

Quoting from Michelle Konzack's mail on Mon, Nov 07, 2005 at 07:29:19PM +0100:
> So right, I am using fetchmail + procmail + courier-imap
> and sometimes They are errors and messages are going into
> /var/mail/<$USER> which are inaccessible.  Specialy if I
> have more then 17.000 $USER.
> 
> I think, procmail should have an config option to stop
> this behaviour and return an error instead.
> 
> It is not funny to walk through 17.000 Mailboxes to get
> the "lost" messages back into the $USER mailboxes.

procmail has a way of returning with an error. You may append the
lines below to your procmailrc for error handling.

==========================================================================

# if procmail has reached here, delivery has failed. return with a
# temporary failure code from <sysexits.h>.
# 75 = EX_TEMPFAIL
EXITCODE=75
:0
/dev/null
==========================================================================

WARNING: the above recipe works only when the program invoking
procmail handles the exit code gracefully. Otherwise, all your mails
will be trashed.

fetchmail will not delete these mails. Even, most SMTP servers should
queue up such mails for a few days before bouncing them back. But, if
you have other programs invoking procmail, you will need to check if
they handle such exit codes gracefully.

-- 
Sunil Shetye.

[fetchmail-users] RE: Fetchmail error!

[deven b. <dev...@gm...>]

 Devendra Barhate.
Pune.
 Sub: I'm getting error "Query status=3 (AUTHFAIL)" while using
fetchmail-6.2.0-3
 Dear Friends,
 I have installed redhat 9.0 with fetchmail-6.2.0-3 on P4 system.
 While fetchmail starts it gives me an error no 3 i.e.Query status=3
(AUTHFAIL)
or bad login .
 Have look at follwing logs messages..
  fetchmail: 6.2.0 querying somemailserver.com
<http://somemailserver.com/>(protocol POP3) at Sun 13 Nov 2005
06:33:52 PM IST: poll started
fetchmail: POP3< +OK POP3 accord1.ads4globe.net<http://accord1.ads4globe.net/>
v2003.83rh server ready
fetchmail: POP3> CAPA
fetchmail: POP3< +OK Capability list follows:
fetchmail: POP3< TOP
fetchmail: POP3< LOGIN-DELAY 180
fetchmail: POP3< UIDL
fetchmail: POP3< USER
fetchmail: POP3< SASL PLAIN LOGIN
fetchmail: POP3< .
fetchmail: POP3> USER fetchmail: POP3> CAPA
fetchmail: POP3< +OK Capability list follows:
fetchmail: POP3< TOP
fetchmail: POP3< LOGIN-DELAY 180
fetchmail: POP3< UIDL
fetchmail: POP3< USER
fetchmail: POP3< SASL PLAIN LOGIN
fetchmail: POP3< .
fetchmail: POP3> USER us...@so...
fetchmail: POP3< +OK User name accepted, password please
fetchmail: POP3> PASS
fetchmail: POP3< -ERR Bad login
fetchmail: Bad login
fetchmail: Authorization failure on us...@so...
fetchmail: POP3> QUIT
fetchmail: POP3< +OK Sayonara
fetchmail: 6.2.0 querying somemailserver.com
<http://somemailserver.com/>(protocol POP3) at Sun 13 Nov 2005
06:36:18 PM IST: poll completed

Now the thing is I can access the mails through webmail over the internet
with same USERNAME and PASSWD.
 then why fetchmail is not able to fetch the mails?
  Awaiting for reply.
 Thanking you,
 Deven.

Re: [fetchmail-users] WARNING: Running as root is discouraged.

[Rob M. <rob...@gm...>]

On 11/11/05, Thomas Wolff <to...@to...> wrote:
> There is this new warning "WARNING: Running as root is discouraged."
> which is somehow disturbing.
> I thought fetchmail is also intended as a system installation
> for multi-user mail retrieval so it's actually a designed mode to
> run fetchmail as root (and the message seems to be unconditional,
> looking at the code).

Why?  There is no need to run fetchmail as root.  You can, and should,
run it as an unpriviledged user.

Look on the bright side, it had been proposed to refuse (as some
software does) to run as root.  It was agreed that at least for 6.3
this wouldn't happen, giving people like you time to change their
configuration.

> Also I don't need this message when I retrieve my personal mailbox while
> regularly working as root on my home machine; this may not be
> recommended in general but it's not fetchmail to tell me that every
> time I retrieve my mail.

I dunno, it does encourage you to use it in a more secure manner :-)

Seriously, running anything as root that doesn't have to be run as
root is a security risk.  Simply run fetchmail using a normal user
account and the message will go away.

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

[fetchmail-users] WARNING: Running as root is discouraged.

[Thomas W. <to...@to...>]

There is this new warning "WARNING: Running as root is discouraged." 
which is somehow disturbing.
I thought fetchmail is also intended as a system installation 
for multi-user mail retrieval so it's actually a designed mode to 
run fetchmail as root (and the message seems to be unconditional, 
looking at the code).
Also I don't need this message when I retrieve my personal mailbox while 
regularly working as root on my home machine; this may not be 
recommended in general but it's not fetchmail to tell me that every 
time I retrieve my mail.

Please remove the message.

Kind regards,
Thomas Wolff

Re: [fetchmail-users] fetchmail gets stuck because of malformed spam

[Rob M. <rob...@gm...>]

On 10/11/05, Thomas Wolff <to...@to...> wrote:
>
> Then I switched to fetchmail 6.2.9rc7 and it fetched all mails as it
> should do, except the affected message, on which it still reported:
> fetchmail: message delimiter found while scanning headers
>
> What still concerns me, however, is that it discarded that message
> completely and wrote no trace of it to the mailbox, so messages with
> that kind of problem (whatever it may be) can not be checked or
> analysed anymore with fetchmail.

The message is illegally formatted (in this case, has no blank line
between headers and body) and should never have been accepted by 1&1's
mail server (though from personal experience I'm not surprised). 
Fetchmail is doing the correct thing and discarding this email.

Previously it has been suggested that one option might be to wrap the
entire of the illegal mail and forward it to the postmaster.  Nobody
however was willing to put the effort into a patch to do this, and
frankly given that every occurence to date has been identified as spam
I'm not surprised.

I am however happy to see that the forthcoming 6.3.0 release will
handle this more gracefully than previous versions :)

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-users] Re: Re: unsafe MDA procmail recommended by fetchmail

[Rob F. <rf...@fu...>]

Michelle Konzack wrote:
> Am 2005-11-07 13:46:15, schrieb Rob Funk:
> > As Stephen van den Berg posted:
> > | It is, just put the following lines anywhere in your .procmailrc
> > | file: DEFAULT
> > |   ORGMAIL
> > |
> > | Which will unset the DEFAULT and ORGMAIL variables (and prevent
> > | evasive action in case of system or user error).
>
> Hmmm, I have tried this, but $DEFAULT is set in the $USER configs to
>
>     DEFAULT=$MAILDIR/.maybe_spam/
>
> and ORGMAIL is unset.

Do you have a line that's just:
ORGMAIL
in the file?  If you don't, then ORGMAIL is not unset.

-- 
==============================|   "A microscope locked in on one point
 Rob Funk <rf...@fu...> |Never sees what kind of room that it's in"
 http://www.funknet.net/rfunk |    -- Chris Mars, "Stuck in Rewind"

Re: [fetchmail-users] Re: unsafe MDA procmail recommended by fetchmail

[Thomas W. <to...@to...>]

I had written:
> > > I had a "disk quota exceeded" condition here which procmail failed
> > > to handle properly. It accepted its input, threw it away and just
> > > returned success.

Stephen van den Berg wrote:
> > > Procmail handles quota-exceeded problems
> > > with flying colours since about 1993.
> >
> > > There is a 99.997% probability that the serious
> > > issue is a user error on your part.
> ...


> > bash> cat /tmp/procmail.log.
> > procmail: Quota exceeded while writing "testbox"
> > procmail: Truncated file to former size
> >  Subject: test mail
> >   Folder: /var/mail/....                                                 1266
> 
> > At this point, I noticed that procmail has a fallback behaviour
> > that when delivery to a local folder fails, it delivers into
> > /var/mail instead. So actually the mail was not lost.
> > This is, however, not documented anywhere.
> [...]
> > So, quoting you again
> > > There is a 99.997% probability that the serious
> > > issue is a user error on your part.
> > I would not say this issue was a user error on my part (because it's
> > not properly documented),

> $ man procmailrc
> [...]
Well, this is not very obvious, because I have to check the manual of 
procmailrc to find out about some basic workflow behaviour of procmail.
And I even have to study the descriptions of the environment variables 
(total of > 250 lines) before I get a clue on this handling mode.

>        ORGMAIL     Usually  the  system  mailbox  (ORiGinal MAILbox).  If, for
>                    some obscure reason (like `filesystem full') the mail could
>                    not  be  delivered,  then  this  mailbox  will  be the last
>                    resort.  If procmail fails to save the mail in here  (deep,
>                    deep  trouble  :-),  then  the mail will bounce back to the
>                    sender.
What would "bounce back" actually mean in an environment with no working 
or properly configured sendmail interface?


> > User feedback and documentation should be improved,
> 
> Please provide suggestions as to what wording in the logfile would have made
> it more clear?
Instead of just reporting the actual output file:
  Folder: /var/mail/....                                                 1266
the logfile should clearly indicate that this was a fallback situation, 
like:
procmail: Truncated file to former size
procmail: Redirecting to system folder as a replacement


> Even without asking on the mailinglist, the logfile
> should have made clear what happened.
Maybe, once you look at it. You don't normally do that, especially as 
the manual is very terse about the logfile and doesn't mention that 
it will be in /tmp by default. In any case, there should be a clear 
error message and indication what procmail did on stderr, too.


> Same question for the manpage; in what way should we rephrase it?
The mail delivery strategy, as well as (at least the names of all) the 
options and settings that can modify it, should be clearly described 
in a paragraph of the initial DESCRIPTION section already.
Currently, it's:
>       ... $HOME/.procmailrc.   According  to  the  processing
>       recipes in this file, 
The recipes do not include any fallback behaviour which is rather implicit.
>       the mail message that  just  arrived
>       gets  distributed into the right folder (and more).
So which are the right and, especially, which are the "more" folders?
>       If no
>       rcfile is found, or processing of the rcfile falls off the
>       end,  procmail  will  store the mail in the default system
>       mailbox.
OK, the system mailbox is mentioned, but not for the case in question.
Same in man procmailrc. All description of error handling is hidden 
deep in the description of settings that one could well assume to be 
optional.


> > ... and maybe this case of redirection should not be applied at all
> 
> Mail wants to arrive.  That means, if we can salvage the mail, instead of
> bouncing it, then that is the preferred solution.  Rescueing instead of
> giving up is the proper action.
I agree this is basically the right strategy.

> It basically guards the user from his
> own lack of knowledge on the situation (and since the user can't
> be asked at that point, procmail has to make an intelligent decision).
The problem here is that the user also has a lack of knowledge on 
procmail's handling of the situation. Once the manual is improved, 
and also the options to tweak the behaviour, it will be fine.


> > (or be configurable)
> 
> It is, just put the following lines anywhere in your .procmailrc file:
>   DEFAULT
>   ORGMAIL
> Which will unset the DEFAULT and ORGMAIL variables (and prevent
> evasive action in case of system or user error).
These variables only briefly occur in man procmail and are not described 
there. There is no hint either that man procmailrc will have to be 
studied to configure behaviour in error situations, and even there it's 
(as said above) hidden somewhere in the long list of settings.
The DESCRIPTION should clearly refer to the relevance of DEFAULT and 
ORGMAIL for proper adaptation to your specific environment, otherwise 
it's next to useless for the non-Guru, and I'm glad to be confirmed 
that I'm not the only one who couldn't easily assemble a proper 
configuration as Michelle Konzack wrote:
> So right, I am using fetchmail + procmail + courier-imap
> and sometimes They are errors and messages are going into
> /var/mail/<$USER> which are inaccessible.  Specialy if I
> have more then 17.000 $USER.
> 
> I think, procmail should have an config option to stop
> this behaviour and return an error instead.
> 
> It is not funny to walk through 17.000 Mailboxes to get
> the "lost" messages back into the $USER mailboxes.


Actually, I think there is also an option missing to prevent procmail 
from attempts to bounce a mail after failed delivery. As I indicated 
above, not all system configuration provide working outgoing mail 
(at least not through the standard interfaces). I, for instance, use 
ssmtp with a script wrapper; /usr/sbin/sendmail is not working here.
In this case (not bouncing failed mail), procmail should of course 
return an error code so e.g. fetchmail notices and doesn't delete 
the mail from the server.


Thanks for your consideration and your request for suggestions to 
improve documentation, which I hope to have done.

Best regards,
Thomas Wolff

[fetchmail-users] Re: Re: unsafe MDA procmail recommended by fetchmail

[Michelle K. <lin...@fr...>]

Hi Rob,

Am 2005-11-07 13:46:15, schrieb Rob Funk:

> As Stephen van den Berg posted:
> 
> | It is, just put the following lines anywhere in your .procmailrc file:
> |   DEFAULT
> |   ORGMAIL
> |
> | Which will unset the DEFAULT and ORGMAIL variables (and prevent
> | evasive action in case of system or user error).

Hmmm, I have tried this, but $DEFAULT is set in the $USER configs to

    DEFAULT=$MAILDIR/.maybe_spam/

and ORGMAIL is unset.

Greetings
Michelle

-- 
Linux-User #280138 with the Linux Counter, http://counter.li.org/
Michelle Konzack   Apt. 917                  ICQ #328449886
                   50, rue de Soultz         MSM LinuxMichi
0033/3/88452356    67100 Strasbourg/France   IRC #Debian (irc.icq.com)

Re: [fetchmail-users] fetchmail gets stuck because of malformed spam

[Thomas W. <to...@to...>]

I wrote:
> But I've actually had similar problems a while ago; fetchmail couldn't 
> get past certain spam mails. I had to use a webmail interface to 
> delete them to get any further. Unfortunately, as I needed quick 
> access to my other mails, I did not care to retrieve any of those 
> (using the webmail interface) which I should have sent here for bug 
> analysis, sorry.

mbarsalou (having a similar problem) wrote:
> Subject: [fetchmail-users] fetchmail stopping on spam
> ...
> This is purely anecdotal, however.


This problem occurred again yesterday and I have additional evidence now:

reading message pt8...@po...:18 of 43 (1873 octets) . flushed
reading message pt8...@po...:19 of 43 (1293 octets) fetchmail: incorrect header line found while scanning headers
fetchmail: message delimiter found while scanning headers
 flushed
fetchmail: client/server protocol error while fetching from pop.kundenserver.de
fetchmail: Query status=4 (PROTOCOL)

This was with fetchmail 6.2.3 and it stopped fetching mails at this point. 
There are two additional observations to note:
* Although fetchmail said "flushed" for all the previous mails (and 
  even the affected one, actually), it did not delete them from the 
  server.
* It left the affected message in the mailbox though (or at least 
  some initial part of the headers) (domain name changed manually):

Return-Path: <OCH...@no...>
Delivery-Date: Tue, 08 Nov 2005 17:25:36 +0100
Received: from pop.kundenserver.de [212.227.15.181]
	by localhost with POP3 (fetchmail-6.2.3)
	for root@localhost (single-drop); Thu, 10 Nov 2005 00:13:00 +0100 (CET)
Received: from [83.242.73.87] (helo=host-n2-73-87.telpol.net.pl)
	by mx.kundenserver.de (node=mxeu3) with ESMTP (Nemesis),
	id 0MKqIe-1EZWHr1aO4-0001NV for th...@xx...; Tue, 08 Nov 2005 17:25:31 +0100
Received: from peternixon.net [215.199.192.160


Then I switched to fetchmail 6.2.9rc7 and it fetched all mails as it 
should do, except the affected message, on which it still reported:
fetchmail: message delimiter found while scanning headers

What still concerns me, however, is that it discarded that message 
completely and wrote no trace of it to the mailbox, so messages with 
that kind of problem (whatever it may be) can not be checked or 
analysed anymore with fetchmail.


Kind regards,
Thomas Wolff