fetchmail-users — User questions and discussion

Re: [fetchmail-users] SSL authentication problems with Gmail

[Matthias A. <mat...@gm...>]

Jakob Hirsch schrieb am 2006-01-05:

> Matthias Andree wrote:
> 
> > 2. Debian's ca-certificates package has the Thawte root certificate in
> >    the default place, this proved sufficient to verify Google's
> >    certificate (which is signed by Thawte) in fetchmail 6.3.1 even with
> 
> just a little unimportant addition: pop.googlemail.com has a certificate
> signed by Thawte, while pop.gmail.com's is signed by Equifax.

Right, I know I confused something WRT the signing authorities from
memory.  Thank you!

-- 
Matthias Andree

Re: [fetchmail-users] SSL authentication problems with Gmail

[Jakob H. <jh...@pl...>]

Matthias Andree wrote:

> 2. Debian's ca-certificates package has the Thawte root certificate in
>    the default place, this proved sufficient to verify Google's
>    certificate (which is signed by Thawte) in fetchmail 6.3.1 even with

just a little unimportant addition: pop.googlemail.com has a certificate
signed by Thawte, while pop.gmail.com's is signed by Equifax.

[fetchmail-users] Re: [fetchmail-announce] fetchmail 6.3.0 contained incompatible idfile change

[Simon B. <ba...@gm...>]

Matthias Andree wrote:
> > What about using one of the system-wide temporary directories instead?
> 

[ Nice explanation skipped -- thanks! ]

> 
> Debian for instance uses a fetchmail user for the system wide fetchmail
> daemon, home directory is /var/run/fetchmail, yet they use
> /var/mail/.i-forgot-the-UIDL-name for their idfile. If they used
> /var/run/fetchmail/.i-forgot-the-UIDL-name, everything would be fine. No
> need for ${TMPDIR-/tmp}.

I thought about adding something similar to the FreeBSD port, but I thought
of having one fetchmail instance per user (if that is possible). OTOH, this
will not work if there are many users.

Do you think, a system-wide fetchmail is worth the trouble? (I know, you are
biased since you are the current maintainer ;-)

-- 
Best regards / Viele Grüße,                             ba...@Fr...
 Simon Barner                                                ba...@gm...

[fetchmail-users] Re: [fetchmail-announce] fetchmail 6.3.0 contained incompatible idfile change

[Matthias A. <mat...@gm...>]

Simon Barner schrieb am 2006-01-05:

> Matthias Andree wrote:
> > Beginning with release 6.3.0, fetchmail needs not only need write access
> > to the idfile ($HOME/.fetchids), but also to the directory holding this
> > file. fetchmail 6.2.X and older versions got away if just the idfile had
> > write permission.
> > 
> > The reason is that fetchmail 6.3.0 and newer now writes the ids to a
> > temporary file, to avoid truncation of the idfile if running short of
> > diskspace; a truncated idfile might lead to many duplicate messages,
> > which might, in turn, aggravate the "low space" situation.
> 
> What about using one of the system-wide temporary directories instead?

The basic idea is: the UIDs are somewhat precious (and we have had
issues to that extent in the bug trackers). That's why I chose to write
a temporary file and atomically (i. e. rename(2)) replace the original
file. This guarantees that we'll always have the old UID file available
even if writing the new fails at some point. In unlucky situations,
6.2.5.X and older might end up with an empty ID file, would start
refetching and flooding /var/mail with dupes.

Rename(2) only works within the same device or perhaps mountpoint (some
file systems such as AFS and Coda even turn every directory into a
filesystem of its own). Besides that, I will not use directories with
public write access (/tmp, /var/tmp), these are minefields, and handling
subdirectories within them still doesn't guarantee we can rename without
truncating the output file.

Plus, creating backup files still requires write permission on the
directory...

Debian for instance uses a fetchmail user for the system wide fetchmail
daemon, home directory is /var/run/fetchmail, yet they use
/var/mail/.i-forgot-the-UIDL-name for their idfile. If they used
/var/run/fetchmail/.i-forgot-the-UIDL-name, everything would be fine. No
need for ${TMPDIR-/tmp}.

-- 
Matthias Andree

[fetchmail-users] Re: [fetchmail-announce] fetchmail 6.3.0 contained incompatible idfile change

[Simon B. <ba...@gm...>]

Matthias Andree wrote:
> Beginning with release 6.3.0, fetchmail needs not only need write access
> to the idfile ($HOME/.fetchids), but also to the directory holding this
> file. fetchmail 6.2.X and older versions got away if just the idfile had
> write permission.
> 
> The reason is that fetchmail 6.3.0 and newer now writes the ids to a
> temporary file, to avoid truncation of the idfile if running short of
> diskspace; a truncated idfile might lead to many duplicate messages,
> which might, in turn, aggravate the "low space" situation.

What about using one of the system-wide temporary directories instead?

-- 
Best regards / Viele Grüße,                             ba...@Fr...
 Simon Barner                                                ba...@gm...

Re: [fetchmail-users] SSL authentication problems with Gmail

[Matthias A. <mat...@gm...>]

Sebastian Tennant <se...@sm...> writes:

> Doh!  Just when you think you've wrapped something up...
>
> I didn't attach the init script did I?  I attached my fetchmailrc,
> including my password!
>
> I've changed the password, and there were no other account details
> included, so no harm done... luckily!
>
> Take two.  Init script attached.

OK, that, and the relevant syslog except allow me to write a concluding
report, Sebastian's problems are completely solved.

1. grabbing the certificate from the server dialogue failed; although
   c_rehash had worked properly, it was the wrong certificate
   apparently. ("unable to get local issuer certificate")

   There are certainly people with a deeper understanding of the SSL
   certification process that can explain this better than I can.

2. Debian's ca-certificates package has the Thawte root certificate in
   the default place, this proved sufficient to verify Google's
   certificate (which is signed by Thawte) in fetchmail 6.3.1 even with
   --sslcertck (which I recommend to use, as it's safer).

   NOTE: older fetchmail versions fail to set the SSL default
   certificate path, you must set "--sslcertpath /etc/ssl/certs"
   manually (or whichever the path is; you can also specify this in the
   fetchmailrc file.).

3. Debian's init script diverts logging to syslog by default, and the
   reporter's syslog.conf split error messages out to a separate file,
   where they went unnoticed.
   
   I therefore take the right to advise against using the "=" and "!"
   operators in syslog.conf. "mail.info" is the correct left-hand-side
   to use in syslog.conf for fetchmail 6.2.5.X and 6.3.X.

4. Debian's init script supports an operation "debug-run", which avoids
   syslog, and logs everything on the console in verbose mode. This
   appears to be a simple way to procure all necessary debug information
   on Debian systems.

Happy fetchmailing,

-- 
Matthias Andree

[fetchmail-users] fetchmail 6.3.0 contained incompatible idfile change

[Matthias A. <mat...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings,

in the original 6.3.0 announcement, I mentioned that we're more careful
about writing the .fetchids file. While this is true, the change
introduced a minor incompatibility.

This announcement attempts to avoid astonishment throught that change.

Beginning with release 6.3.0, fetchmail needs not only need write access
to the idfile ($HOME/.fetchids), but also to the directory holding this
file. fetchmail 6.2.X and older versions got away if just the idfile had
write permission.

The reason is that fetchmail 6.3.0 and newer now writes the ids to a
temporary file, to avoid truncation of the idfile if running short of
diskspace; a truncated idfile might lead to many duplicate messages,
which might, in turn, aggravate the "low space" situation.

For end users running fetchmail with default idfile on their own
account, nothing changes: they have write access to their home
directory.

For system-wide execution of the script as used by some distributions
(for instance Debian), it is now important that the idfile is in a
directory writable by fetchmail, else creation of the temporary file and
renaming into place will fail. Debian for instance places the idfile
under /var/mail, this will no longer work with 6.3.0.

I still think the new behavior is a step ahead,
towards a more reliable fetchmail.

Please accept my apologies for missing this in the 6.3.0/6.3.1 materials
I provided to Rob Funk and that we shipped with fetchmail 6.3.0.

Happy fetchmailing,

- -- 
Matthias Andree
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFDvF7dvmGDOQUufZURAiknAKCFQxw6GWYmrvh2TW/qASnSA2n66QCfVrR/
IY+E3PqfFeWJW560g5IPDmQ=
=uILr
-----END PGP SIGNATURE-----

[fetchmail-users] [JFTR] (HEADS UP Rob & Rob!) debugging on Debian

[Matthias A. <mat...@gm...>]

Greetings,

if fetchmail is started on Debian, the default init script appears to
support a "debug-run", which starts fetchmail in foreground mode,
logging to console, with verbose info -- this can help debug fetchmail
in some cases.

I hope we all remember this option next time a Debian user has trouble.

Happy mail fetching,

-- 
Matthias Andree

[fetchmail-users] Updated Cygwin Package: fetchmail-6.3.1-1

[Jason T. <ja...@ti...>]

New News:
=== ====
I have updated the version of fetchmail to 6.3.1-1.  The tarballs should
be available on a Cygwin mirror near you shortly.

The only change between this version and the previous one is the
following:

    o update to version 6.3.1

Old News:
=== ====
Fetchmail is a remote mail retrieval and forwarding utility intended
for use over on-demand TCP/IP links, like SLIP or PPP connections.
Fetchmail supports every remote-mail protocol currently in use on the
Internet (POP2, POP3, RPOP, APOP, KPOP, all IMAPs, ESMTP ETRN, IPv6,
and IPSEC) for retrieval. Then Fetchmail forwards the mail through SMTP
so you can read it through your favorite mail client.

See the fetchmail home page for more details:

    http://fetchmail.berlios.de/

Please read the README file:

    /usr/share/doc/Cygwin/fetchmail-6.3.1.README

since it covers requirements, installation, known issues, etc.

Standard News:
======== ====
To update your installation, click on the "Install Cygwin now" link on
the http://cygwin.com/ web page.  This downloads setup.exe to your
system.  Then, run setup and answer all of the questions.

If you have questions or comments, please send them to the Cygwin
mailing list at: cy...@cy... .

              *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO ***

If you want to unsubscribe from the cygwin-announce mailing list, look
at the "List-Unsubscribe: " tag in the email header of this message.
Send email to the address specified there.  It will be in the format:

cyg...@cy...

If you need more information on unsubscribing, start reading here:

http://sources.redhat.com/lists.html#unsubscribe-simple

Please read *all* of the information on unsubscribing that is available
starting at this URL.

Jason

--
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6

Re: [fetchmail-users] Fetchmail is running slow on Mac OS X

[Matthias A. <mat...@gm...>]

Stéphane Lux <ste...@lu...> writes:

> Am 02.01.2006 um 16:38 schrieb Rob Funk:
>
>> Stéphane Lux wrote:
>>> fetchmail: starting fetchmail 6.2.5 daemon
>>> fetchmail: 6.2.5 querying ... (protocol IMAP) at Sat, 31 Dec 2005
>>> 03:03:13 +0100 (CET): poll started
>>>
>>> Then nothing happens for about one minute  (timout?), then fetchmail
>>> ist running normaly:
>>
>> Normally delays of a minute or two indicate a DNS problem somewhere.
>
> What kind of DNS problem? Server or client? I can telnet to the port  
> 143 without a delay.

Probably the same as <https://bugzilla.mozilla.org/show_bug.cgi?id=231607>.

Try this on your fetchmail 6.3.1 source directory (requires Perl installed):

perl -ple 's/([AP])F_UNSPEC/\1F_INET/g' -i.bak \
     checkalias.c driver.c env.c servport.c

Then rebuild and reinstall fetchmail 6.3.1 and let us know if the
problem is gone.

Merci.

-- 
Matthias Andree

Re: [fetchmail-users] SSL authentication problems with Gmail

[Matthias A. <mat...@gm...>]

Sebastian Tennant <se...@sm...> writes:

>>> OK, added another `-v' and it just lists the Thawte server as well...
>>>
>>>   fetchmail[4584]: starting fetchmail 6.3.1 daemon
>>>   fetchmail[4584]: 6.3.1 querying pop.googlemail.com (protocol POP3) at Wed Jan  4 11:47:17 2006: poll started
>>>   fetchmail[4584]: Issuer Organization: Thawte Consulting cc
>>>   fetchmail[4584]: Issuer CommonName: Thawte Premium Server CA
>>>   fetchmail[4584]: Server CommonName: pop.googlemail.com
>>>   fetchmail[4584]: pop.googlemail.com key fingerprint: 46:8B:6C:F4:3E:4C:56:29:83:54:2C:37:42:F1:67:80
>>>   fetchmail[4584]: 6.3.1 querying pop.googlemail.com (protocol POP3) at Wed Jan  4 11:47:18 2006: poll completed
>>>   fetchmail[4584]: Query status=2 (SOCKET)
>>>   fetchmail[4584]: sleeping at Wed Jan  4 11:47:18 2006
>>
>>Looks like it never talks to the POP server.  Can you drop the "port
>>995" and "sslcertck" options from your fetchmailrc and see what you
>>get.
>
> Removed these lines and it works.  Thanks to everyone who helped.

Well, I checked the source code and found no code path where SSL
certificate verification would fail without leaving log messages, such
as 1. the actual error and 2. "SSL connection failed".

POP3 was configured explicitly, so "port 995" forth or back doesn't make
a difference either -- removing this option can only make things worse,
not better.

Remains the question after sslcertck -- it will log trouble, too, EXCEPT
if a certificate at greater depth causes a preverification failure
without setting the error code in the X.509 context variables (and we'd
still get "SSL connection failed" in this case).

It appears as though the server dropped the connection after the SSL
negotiation and before the greeting, or that your log information is
incomplete.  Your logging appears to be from syslog, so could you post
your syslog.conf or syslog-ng.conf (whichever you're *actually* using)?

Do you get more detailed logging with "fetchmail --nosyslog -vv -N -d0
--sslcertck --port 995"? Can you try running this and see if you still
get socket errors and if so, which errors they print?

Thanks in advance,

-- 
Matthias Andree

[fetchmail-users] SSL authentication problems with Gmail

[Sebastian T. <se...@sm...>]

>> OK, added another `-v' and it just lists the Thawte server as well...
>>
>>   fetchmail[4584]: starting fetchmail 6.3.1 daemon
>>   fetchmail[4584]: 6.3.1 querying pop.googlemail.com (protocol POP3) at Wed Jan  4 11:47:17 2006: poll started
>>   fetchmail[4584]: Issuer Organization: Thawte Consulting cc
>>   fetchmail[4584]: Issuer CommonName: Thawte Premium Server CA
>>   fetchmail[4584]: Server CommonName: pop.googlemail.com
>>   fetchmail[4584]: pop.googlemail.com key fingerprint: 46:8B:6C:F4:3E:4C:56:29:83:54:2C:37:42:F1:67:80
>>   fetchmail[4584]: 6.3.1 querying pop.googlemail.com (protocol POP3) at Wed Jan  4 11:47:18 2006: poll completed
>>   fetchmail[4584]: Query status=2 (SOCKET)
>>   fetchmail[4584]: sleeping at Wed Jan  4 11:47:18 2006
>
>Looks like it never talks to the POP server.  Can you drop the "port
>995" and "sslcertck" options from your fetchmailrc and see what you
>get.

Removed these lines and it works.  Thanks to everyone who helped.

>I'll stick my neck out and say that it's likely a local config or
>compilation error.  I had zero trouble getting fetchmail to pull my
>mail from GMail with the following fetchmailrc:
>
>poll pop.gmail.com
>        proto pop3
>        username "ME<AT>gmail.com"
>        password "PASS"
>        ssl

sdt

[fetchmail-users] Re: broken POP3 servers - future fetchmail policy (IMPORTANT) (was: [fetchmail]comcast.net and attachments - workaround)

[Matthias A. <mat...@gm...>]

Jakob Hirsch schrieb am 2006-01-04:

> Well, RFC 1939 says "Such message deletions are outside the scope of the
> POP3 protocol and are not considered a protocol violation." (thanks for
> pointing me to this chapter, btw)

Right, but I took your report as though your server doesn't wait until
QUIT, in which case it *is* a violation.

> Site policies always override RFC-specified behaviour, there's not much
> you can do about that.

If these site policies aren't explicitly allowed by the RFC, they site
is not in compliance, and all bets WRT client function are off.

> > If talking to the admins doesn't resolve their bad behavior, find a
> > different server.
> 
> It's my university's server...

So unless it's in the AUP ("Nutzungsbedingungen") or other relevant
documents, they're trashing your mail without informed consent, and thus
punishable according to German law.

Check the paperworks and unless you (perhaps implicitly) agreed to such
a policy, talk to them.

Most sites refuse new messages if the mailbox size exceeds a quota, or
delete old messages after a month or so.  That would solve the problem
on either side: you can use several clients, they can limit the disk
space consumption.

> > Loss scenario: What happens if the client crashes between the server's
> > shipping CRLF.CRLF and storing the message? Whoops, the server deleted
> > the message, and the client retry.
> 
> Not necessarily. Consider a normal POP3 server, only that he does an
> implicit DELE after each RETR. Of course, a client could still fail to
> store the message (or to forward it somewhere, like fetchmail) without the
> server noticing. But only if the client tries to behave nicely and sends
> QUIT even after something went wrong. In this case, nice behaviour does
> not pay off.

Will check.

> >> What changes do you plan for the seen tracking? To do it on the client
> >> side is already possible with the UIDL option.
> > UIDL is not an option, but a necessity.
> 
> For simply retrieving mail, delivering locally, retry in case of failure?
> I don't see why.

1. Because it is the only reliable alternative to "fetchall nokeep",
   independent of the current TOP-vs-RETR discussion.

2. Because the assumption fetchmail were the only client doesn't hold in
   many cases (such as yours) and isn't reasonable either.

3. Because servers are often subjected to what I see as "empirical
   programming" - write server code that appears to work with Outlook
   Express at first glance, rather than writing standards compliant code
   that is in compliance with RFC-1939.

-- 
Matthias Andree

[fetchmail-users] Re: broken POP3 servers - future fetchmail policy (IMPORTANT) (was: [fetchmail]comcast.net and attachments - workaround)

[Jakob H. <jh...@pl...>]

Matthias Andree wrote:

> [Cc:ing BerliOS users

Right, we better continue on the new list.

>> Dropping it completely would be a pity. I have one upstream server
>> (Mercury on a Netware server...) that deletes messages after RETR
>> automatically. That may be a nice feature for the admin, but it's
>> annoying if you like to read mail from more than one place.
> It is not only "annoying", but up front a critical data loss defect of
> the server, needless to say it is a violation of the protocol, too.

Well, RFC 1939 says "Such message deletions are outside the scope of the
POP3 protocol and are not considered a protocol violation." (thanks for
pointing me to this chapter, btw)

Site policies always override RFC-specified behaviour, there's not much
you can do about that.

> If talking to the admins doesn't resolve their bad behavior, find a
> different server.

It's my university's server...

> Loss scenario: What happens if the client crashes between the server's
> shipping CRLF.CRLF and storing the message? Whoops, the server deleted
> the message, and the client retry.

Not necessarily. Consider a normal POP3 server, only that he does an
implicit DELE after each RETR. Of course, a client could still fail to
store the message (or to forward it somewhere, like fetchmail) without the
server noticing. But only if the client tries to behave nicely and sends
QUIT even after something went wrong. In this case, nice behaviour does
not pay off.

>> What changes do you plan for the seen tracking? To do it on the client
>> side is already possible with the UIDL option.
> UIDL is not an option, but a necessity.

For simply retrieving mail, delivering locally, retry in case of failure?
I don't see why.

[fetchmail-users] Re: [fetchmail]comcast.net and attachments - workaround

[Matthias A. <mat...@gm...>]

On Tue, 03 Jan 2006, Ed Wilts wrote:

> On Tue, Jan 03, 2006 at 04:43:28PM +0100, Matthias Andree wrote:
> > In the meanwhile, please complain to comcast.net for violating the POP3
> > standard, which states expressis verbis that the full message is to be
> > fetched if the number passed to TOP exceeds the actual number of lines
> > in the message.
> 
> That would be a waste of time.  After all, it works fine with all the
> normal Windows clients.

The bug doesn't depend on the operating system.

After all, comcast need not offer TOP at all if they don't plan to do it
right. TOP is optional.

Now, if Novell or Red Hat could phone comcast (rather than individual
users) and demand the fix, that might make a difference...

> In my opinion, we (the Linux community) need to spend more time working
> around broken implementations and less time standing on our podiums
> claiming we're right.

I don't plan to spend much time working around broken implementations,
there are more important things to do in fetchmail.

Standards are there for a reason, and not only Microsoft will notice
that soon enough.

In this particular example, you're lucky as the test is trivial and
"more RETR, less TOP" coincides with my goals for fetchmail.

Please test the attached patch (save it to a file, type "patch
<workaround-Maillennium.patch"), then recompile ("make") and reinstall
("make install") fetchmail and see if it solves your problem. It will
however mark messages as downloaded on the server, so I'm not sure if
or how I'll merge it into 6.3.2 or if it needs to wait until 6.4.0.

It is untested (I have no comcast account) and should apply against 6.3.1.
As usual, no warranties, use at your own risk.

If someone else wishes to test, go ahead and let me know what you find.
The warning message only appears if fetchmail had used TOP otherwise.

> I know it's disgusting, but unfortunately we're
> propogating the attitudes that Windows works better than Linux, because
> in this case, it does.  My Windows clients work fine (which is how I
> narrowed it down to fetchmail), my Linux one doesn't (until I
> implemented my workaround), yet the server is unlikely to change.

They, too, must learn that standards conformance is a way to keep
customers happy and money flowing in. If they truncate your messages
because their server is buggy, why give them your best^W money?

> I do see that fetchmail will be fixed to work better in cases like this
> so I'm not putting you in the same camp as some other Linux developers
> I've run into.

My motivation to add _complex_ workaround code as little as that of many
fellow developers. You may get the workaround if it's simple and the
problem hurts really bad.

Software should work efficiently on conformant servers, and our
installed base is large enough we can take some risks of not pleasing
everyone.  Fetchmail is default install in many systems.

I won't be adding workarounds that are disadvantages to users of intact
servers.

Kind regards,

-- 
Matthias Andree

Re: [fetchmail-users] broken POP3 servers - future fetchmail policy (IMPORTANT)

[Rob M. <rob...@gm...>]

On 03/01/06, Matthias Andree <mat...@gm...> wrote:
>
> That's what I was trying to say: download completely, hand on
> completely, obtain status of destination SMTP/LMTP/BSMTP/MDA, and only
> assume "completed" if every step succeeded. If any intermediate step
> fails, we'll retry the message next time. Although it's bad luck we've
> missed the goal of one attempt, and we'll have 1.7 downloads or so.

That's what I thought.  Thanks for confirming that :)

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-users] broken POP3 servers - future fetchmail policy (IMPORTANT)

[Matthias A. <mat...@gm...>]

Rob MacGregor <rob...@gm...> writes:

> On 03/01/06, Matthias Andree <mat...@gm...> wrote:
>>
>> fetchmail's task is to fetch all messages exactly once, nothing more,
>> nothing less. If we must fetch a message twice because the transfer was
>> interrupted, well, bad luck.
>
> You sure you're not ESR in disguise :-)

100% sure I'm not ESR. 

> I'm happy with that statement, if by fetch you mean "download and hand
> it on to the specified MTA".

That's what I was trying to say: download completely, hand on
completely, obtain status of destination SMTP/LMTP/BSMTP/MDA, and only
assume "completed" if every step succeeded. If any intermediate step
fails, we'll retry the message next time. Although it's bad luck we've
missed the goal of one attempt, and we'll have 1.7 downloads or so.

> Otherwise the statement sounds like you don't care of there's a
> problem with the local MTA, fetchmail will have done it's bit by
> downloading the email, chucking it at /dev/null and deleting it from
> the remote server...

Fetchmail will continue to make as many attempts as necessary to
retrieve the message for reliable protocols such as POP3+UIDL.

This requires co-operation of the server though, like not doing stupid
things such as deleteing the message without knowing if the client was
able to store the message. Looking at code and FAQ, many TOP
implementations leave things to be desired.

Some minor polishing has yet to happen though, for instance, after a
crash, fetchmail may retrieve messages it had successfully retrieved and
delivered before the crash, causing duplicates, but better get a message
again that you have already read than miss out on one you hadn't
read. We could reduce the number of duplicates by checkpointing the
.fetchids more often.

-- 
Matthias Andree

Re: [fetchmail-users] broken POP3 servers - future fetchmail policy (IMPORTANT) (was: [fetchmail]comcast.net and attachments - workaround)

[Rob M. <rob...@gm...>]

On 03/01/06, Matthias Andree <mat...@gm...> wrote:
>
> fetchmail's task is to fetch all messages exactly once, nothing more,
> nothing less. If we must fetch a message twice because the transfer was
> interrupted, well, bad luck.

You sure you're not ESR in disguise :-)

I'm happy with that statement, if by fetch you mean "download and hand
it on to the specified MTA".  Otherwise the statement sounds like you
don't care of there's a problem with the local MTA, fetchmail will
have done it's bit by downloading the email, chucking it at /dev/null
and deleting it from the remote server...

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

[fetchmail-users] broken POP3 servers - future fetchmail policy (IMPORTANT) (was: [fetchmail]comcast.net and attachments - workaround)

[Matthias A. <mat...@gm...>]

[Cc:ing BerliOS users 

Jakob Hirsch schrieb am 2006-01-03:

> Matthias Andree wrote:
> 
> > fetchmail will gradually move to client-side seen tracking since that's
> > the only reliable way to do it, and we'll probably get rid of TOP
> > because it's just too painful with the many b0rked upstreams.
> 
> Dropping it completely would be a pity. I have one upstream server
> (Mercury on a Netware server...) that deletes messages after RETR
> automatically. That may be a nice feature for the admin, but it's annoying
> if you like to read mail from more than one place.

It is not only "annoying", but up front a critical data loss defect of
the server, needless to say it is a violation of the protocol, too.

And it has legal implications at least in Germany for most practical
purposes, unless you signed a clause to the extent that "$ISP is allowed
to delete messages even after unsuccessful retrieval attempts, in
violation of pertinent standards" or it's a company machine with no
permission for you to use the mailbox for private purposes, too.

If talking to the admins doesn't resolve their bad behavior, find a
different server.

Loss scenario: What happens if the client crashes between the server's
shipping CRLF.CRLF and storing the message? Whoops, the server deleted
the message, and the client retry.

There's a reason why messages MUST NOT be deleted unless the client
sends DELE 1234 for message 1234 and then commits the changes with QUIT
("UPDATE" state), and that reason is called reliability.

Working around such massively broken server behavior will just carry on
the users' suffering perpetually because server admins will think that
clients have alternatives.

> What changes do you plan for the seen tracking? To do it on the client
> side is already possible with the UIDL option.

UIDL is not an option, but a necessity.

The only reason why I didn't make it default in 6.3.X is that I wanted
6.3.X to be as compatible with 6.2.X as needed, so that most people
could just drop 6.3.X in to have their 6.2.X bugs fixed, without further
thinking. We'll need to enable UID/UIDVALIDITY for IMAP, too.

fetchmail's task is to fetch all messages exactly once, nothing more,
nothing less. If we must fetch a message twice because the transfer was
interrupted, well, bad luck.

-- 
Matthias Andree

Re: [fetchmail-users] Fetchmail is running slow on Mac OS X

[Stéphane L. <ste...@lu...>]

Am 03.01.2006 um 00:50 schrieb Rob MacGregor:

> [ Please don't CC me - I get the mail on the list ]
>
> On 02/01/06, Stéphane Lux <ste...@lu...> wrote:
>>
>> I have tried fetchmail 6.3.1. I still have the one minute delays for
>> setting up a connection:
>
> Next thing to do is run fetchmail under truss to get a dump of what's
> going on.  Watching that it should be fairly obvious where the problem
> lies.  The output of that, with the area the pause occurs marked,
> would help in tracking down the problem.


Here is the output of kdump:

...
2710 fetchmail RET   sigaction 0
12710 fetchmail CALL  sigaction(0xd,0xbfffd3d8,0xbfffd450)
12710 fetchmail RET   sigaction 0
12710 fetchmail CALL  sigprocmask(0x1,0,0x32c28)
12710 fetchmail RET   sigprocmask 0
12710 fetchmail CALL  setitimer(0,0xbfffd460,0)
12710 fetchmail RET   setitimer 0

--- here is the one minute delay ---

12710 fetchmail CALL  socket(0x2,0x1,0)
12710 fetchmail RET   socket 4
12710 fetchmail CALL  connect(0x4,0x3018b0,0x10)
12710 fetchmail RET   connect 0
12710 fetchmail CALL  setitimer(0,0xbfffd460,0)
12710 fetchmail RET   setitimer 0
12710 fetchmail CALL  setitimer(0,0xbfffd460,0)
12710 fetchmail RET   setitimer 0
12710 fetchmail CALL  setitimer(0,0xbfffb3a0,0)
12710 fetchmail RET   setitimer 0
12710 fetchmail CALL  recvfrom(0x4,0xbfffb450,0x2000,0x2,0,0)
12710 fetchmail GIO   fd 4 wrote 147 bytes
...

Does this help in tracking down the problem?

[fetchmail-users] Re: [fetchmail]Re: SSL authentication problems with Gmail

[Matthias A. <mat...@gm...>]

[Cc:ing fet...@li..., JFTR]

Sebastian Tennant <se...@sm...> writes:

> I unpacked the 6.3.1 tarball in my workspace directory, downloaded
> the openssl source from the Debian apt repositories (testing) and
> copied it across to my workspace too.  Then I ran:
>
>   $ cd fetchmail-6.3.1
>   $ ./configure --with-ssl=/home/sebyte/workspace/openssl-0.9.8a
>   [...]
>   configure: error: cannot link with SSL - check config.log
>
> I read the INSTALL file for building with ssl and the directory I
> specified on the comand line definitely includes the include/
> directory...

Actually, the system SSL should have been fine after installation of the
-dev package. I know not a single system with OpenSSL in the base system
where --with-ssl=/usr hasn't worked for me.

> I hope I've attached the relevant section from the config.log.

Yup.

> configure:12595: gcc -o conftest -g -O2   conftest.c  -L/home/sebyte/workspace/openssl-0.9.8a/lib -lcrypt  -lresolv -lssl -lcrypto  >&5
> /usr/bin/ld: cannot find -lssl

Try running "ldconfig -n /home/sebyte/workspace/openssl-0.9.8a/lib" and
re-run ./configure.

-- 
Matthias Andree

Re: [fetchmail-users] Fetchmail is running slow on Mac OS X

[Rob M. <rob...@gm...>]

[ Please don't CC me - I get the mail on the list ]

On 02/01/06, Stéphane Lux <ste...@lu...> wrote:
>
> I have tried fetchmail 6.3.1. I still have the one minute delays for
> setting up a connection:

Next thing to do is run fetchmail under truss to get a dump of what's
going on.  Watching that it should be fairly obvious where the problem
lies.  The output of that, with the area the pause occurs marked,
would help in tracking down the problem.

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-users] Fetchmail is running slow on Mac OS X

[Stéphane L. <ste...@lu...>]

Am 02.01.2006 um 00:32 schrieb Rob MacGregor:

> On 01/01/06, Stéphane Lux <ste...@lu...> wrote:
>>
>> When I start fetchmail, I get this message:
>>
>> fetchmail: starting fetchmail 6.2.5 daemon
>
> As per my message on the other list - please try 6.3.1.

;)
I still have the same delays wit 6.3.1.

Re: [fetchmail-users] Fetchmail is running slow on Mac OS X

[Stéphane L. <ste...@lu...>]

Am 02.01.2006 um 16:38 schrieb Rob Funk:

> Stéphane Lux wrote:
>> fetchmail: starting fetchmail 6.2.5 daemon
>> fetchmail: 6.2.5 querying ... (protocol IMAP) at Sat, 31 Dec 2005
>> 03:03:13 +0100 (CET): poll started
>>
>> Then nothing happens for about one minute  (timout?), then fetchmail
>> ist running normaly:
>
> Normally delays of a minute or two indicate a DNS problem somewhere.

What kind of DNS problem? Server or client? I can telnet to the port  
143 without a delay.


>> On my linux server it takes only a few seconds for fetchmail to check
>> the mail accounts? What can I do? Is it a DNS or IDENT problem? I'm
>> using a NAT router.
>
> Ah, ident could be a possibility too, depending on the server  
> software, but
> DNS is more likely.
>
> As the other Rob recommended, first try fetchmail 6.3.1 from
> http://fetchmail.berlios.de/.  If that doesn't help let us know.

I have tried fetchmail 6.3.1. I still have the one minute delays for  
setting up a connection:

mac-mini:~/Desktop root# ./fetchmail -Nv -f /var/root/.fetchmailrc
fetchmail: WARNING: Running as root is discouraged.
fetchmail: starting fetchmail 6.3.1 daemon
fetchmail: 6.3.1 querying *** (protocol IMAP) at Mon, 02 Jan 2006  
22:56:38 +0100 (CET): poll started
fetchmail: IMAP< * OK [CAPABILITY IMAP4REV1 LOGIN-REFERRALS STARTTLS  
AUTH=LOGIN] *** IMAP4rev1 2003.339 at Mon, 2 Jan 2006 22:57:37 +0100  
(CET)
...

Re: [fetchmail-users] Fetchmail is running slow on Mac OS X

[Rob F. <rf...@fu...>]

Stéphane Lux wrote:
> fetchmail: starting fetchmail 6.2.5 daemon
> fetchmail: 6.2.5 querying ... (protocol IMAP) at Sat, 31 Dec 2005
> 03:03:13 +0100 (CET): poll started
>
> Then nothing happens for about one minute  (timout?), then fetchmail
> ist running normaly:

Normally delays of a minute or two indicate a DNS problem somewhere.  

> On my linux server it takes only a few seconds for fetchmail to check
> the mail accounts? What can I do? Is it a DNS or IDENT problem? I'm
> using a NAT router.

Ah, ident could be a possibility too, depending on the server software, but 
DNS is more likely.

As the other Rob recommended, first try fetchmail 6.3.1 from 
http://fetchmail.berlios.de/.  If that doesn't help let us know.

-- 
==============================|   "A microscope locked in on one point
 Rob Funk <rf...@fu...> |Never sees what kind of room that it's in"
 http://www.funknet.net/rfunk |    -- Chris Mars, "Stuck in Rewind"