Menu
▾
▴
fetchmail-users
|
From: Matthias A. <mat...@gm...> - 2006-01-03 18:24:31
|
[Cc:ing BerliOS users
Jakob Hirsch schrieb am 2006-01-03:
> Matthias Andree wrote:
>
> > fetchmail will gradually move to client-side seen tracking since that's
> > the only reliable way to do it, and we'll probably get rid of TOP
> > because it's just too painful with the many b0rked upstreams.
>
> Dropping it completely would be a pity. I have one upstream server
> (Mercury on a Netware server...) that deletes messages after RETR
> automatically. That may be a nice feature for the admin, but it's annoying
> if you like to read mail from more than one place.
It is not only "annoying", but up front a critical data loss defect of
the server, needless to say it is a violation of the protocol, too.
And it has legal implications at least in Germany for most practical
purposes, unless you signed a clause to the extent that "$ISP is allowed
to delete messages even after unsuccessful retrieval attempts, in
violation of pertinent standards" or it's a company machine with no
permission for you to use the mailbox for private purposes, too.
If talking to the admins doesn't resolve their bad behavior, find a
different server.
Loss scenario: What happens if the client crashes between the server's
shipping CRLF.CRLF and storing the message? Whoops, the server deleted
the message, and the client retry.
There's a reason why messages MUST NOT be deleted unless the client
sends DELE 1234 for message 1234 and then commits the changes with QUIT
("UPDATE" state), and that reason is called reliability.
Working around such massively broken server behavior will just carry on
the users' suffering perpetually because server admins will think that
clients have alternatives.
> What changes do you plan for the seen tracking? To do it on the client
> side is already possible with the UIDL option.
UIDL is not an option, but a necessity.
The only reason why I didn't make it default in 6.3.X is that I wanted
6.3.X to be as compatible with 6.2.X as needed, so that most people
could just drop 6.3.X in to have their 6.2.X bugs fixed, without further
thinking. We'll need to enable UID/UIDVALIDITY for IMAP, too.
fetchmail's task is to fetch all messages exactly once, nothing more,
nothing less. If we must fetch a message twice because the transfer was
interrupted, well, bad luck.
--
Matthias Andree
|
|
From: Rob M. <rob...@gm...> - 2006-01-03 18:57:39
|
On 03/01/06, Matthias Andree <mat...@gm...> wrote:
>
> fetchmail's task is to fetch all messages exactly once, nothing more,
> nothing less. If we must fetch a message twice because the transfer was
> interrupted, well, bad luck.
You sure you're not ESR in disguise :-)
I'm happy with that statement, if by fetch you mean "download and hand
it on to the specified MTA". Otherwise the statement sounds like you
don't care of there's a problem with the local MTA, fetchmail will
have done it's bit by downloading the email, chucking it at /dev/null
and deleting it from the remote server...
--
Please keep list traffic on the list.
Rob MacGregor
Whoever fights monsters should see to it that in the process he
doesn't become a monster. Friedrich Nietzsche
|
|
From: Matthias A. <mat...@gm...> - 2006-01-03 21:19:18
|
Rob MacGregor <rob...@gm...> writes: > On 03/01/06, Matthias Andree <mat...@gm...> wrote: >> >> fetchmail's task is to fetch all messages exactly once, nothing more, >> nothing less. If we must fetch a message twice because the transfer was >> interrupted, well, bad luck. > > You sure you're not ESR in disguise :-) 100% sure I'm not ESR. > I'm happy with that statement, if by fetch you mean "download and hand > it on to the specified MTA". That's what I was trying to say: download completely, hand on completely, obtain status of destination SMTP/LMTP/BSMTP/MDA, and only assume "completed" if every step succeeded. If any intermediate step fails, we'll retry the message next time. Although it's bad luck we've missed the goal of one attempt, and we'll have 1.7 downloads or so. > Otherwise the statement sounds like you don't care of there's a > problem with the local MTA, fetchmail will have done it's bit by > downloading the email, chucking it at /dev/null and deleting it from > the remote server... Fetchmail will continue to make as many attempts as necessary to retrieve the message for reliable protocols such as POP3+UIDL. This requires co-operation of the server though, like not doing stupid things such as deleteing the message without knowing if the client was able to store the message. Looking at code and FAQ, many TOP implementations leave things to be desired. Some minor polishing has yet to happen though, for instance, after a crash, fetchmail may retrieve messages it had successfully retrieved and delivered before the crash, causing duplicates, but better get a message again that you have already read than miss out on one you hadn't read. We could reduce the number of duplicates by checkpointing the .fetchids more often. -- Matthias Andree |
|
From: Rob M. <rob...@gm...> - 2006-01-03 23:33:45
|
On 03/01/06, Matthias Andree <mat...@gm...> wrote:
>
> That's what I was trying to say: download completely, hand on
> completely, obtain status of destination SMTP/LMTP/BSMTP/MDA, and only
> assume "completed" if every step succeeded. If any intermediate step
> fails, we'll retry the message next time. Although it's bad luck we've
> missed the goal of one attempt, and we'll have 1.7 downloads or so.
That's what I thought. Thanks for confirming that :)
--
Please keep list traffic on the list.
Rob MacGregor
Whoever fights monsters should see to it that in the process he
doesn't become a monster. Friedrich Nietzsche
|
|
From: Matthias A. <mat...@gm...> - 2006-01-04 02:55:59
Attachments:
workaround-Maillennium.patch
|
On Tue, 03 Jan 2006, Ed Wilts wrote: > On Tue, Jan 03, 2006 at 04:43:28PM +0100, Matthias Andree wrote: > > In the meanwhile, please complain to comcast.net for violating the POP3 > > standard, which states expressis verbis that the full message is to be > > fetched if the number passed to TOP exceeds the actual number of lines > > in the message. > > That would be a waste of time. After all, it works fine with all the > normal Windows clients. The bug doesn't depend on the operating system. After all, comcast need not offer TOP at all if they don't plan to do it right. TOP is optional. Now, if Novell or Red Hat could phone comcast (rather than individual users) and demand the fix, that might make a difference... > In my opinion, we (the Linux community) need to spend more time working > around broken implementations and less time standing on our podiums > claiming we're right. I don't plan to spend much time working around broken implementations, there are more important things to do in fetchmail. Standards are there for a reason, and not only Microsoft will notice that soon enough. In this particular example, you're lucky as the test is trivial and "more RETR, less TOP" coincides with my goals for fetchmail. Please test the attached patch (save it to a file, type "patch <workaround-Maillennium.patch"), then recompile ("make") and reinstall ("make install") fetchmail and see if it solves your problem. It will however mark messages as downloaded on the server, so I'm not sure if or how I'll merge it into 6.3.2 or if it needs to wait until 6.4.0. It is untested (I have no comcast account) and should apply against 6.3.1. As usual, no warranties, use at your own risk. If someone else wishes to test, go ahead and let me know what you find. The warning message only appears if fetchmail had used TOP otherwise. > I know it's disgusting, but unfortunately we're > propogating the attitudes that Windows works better than Linux, because > in this case, it does. My Windows clients work fine (which is how I > narrowed it down to fetchmail), my Linux one doesn't (until I > implemented my workaround), yet the server is unlikely to change. They, too, must learn that standards conformance is a way to keep customers happy and money flowing in. If they truncate your messages because their server is buggy, why give them your best^W money? > I do see that fetchmail will be fixed to work better in cases like this > so I'm not putting you in the same camp as some other Linux developers > I've run into. My motivation to add _complex_ workaround code as little as that of many fellow developers. You may get the workaround if it's simple and the problem hurts really bad. Software should work efficiently on conformant servers, and our installed base is large enough we can take some risks of not pleasing everyone. Fetchmail is default install in many systems. I won't be adding workarounds that are disadvantages to users of intact servers. Kind regards, -- Matthias Andree |
|
From: Jakob H. <jh...@pl...> - 2006-01-04 11:48:06
|
Matthias Andree wrote: > [Cc:ing BerliOS users Right, we better continue on the new list. >> Dropping it completely would be a pity. I have one upstream server >> (Mercury on a Netware server...) that deletes messages after RETR >> automatically. That may be a nice feature for the admin, but it's >> annoying if you like to read mail from more than one place. > It is not only "annoying", but up front a critical data loss defect of > the server, needless to say it is a violation of the protocol, too. Well, RFC 1939 says "Such message deletions are outside the scope of the POP3 protocol and are not considered a protocol violation." (thanks for pointing me to this chapter, btw) Site policies always override RFC-specified behaviour, there's not much you can do about that. > If talking to the admins doesn't resolve their bad behavior, find a > different server. It's my university's server... > Loss scenario: What happens if the client crashes between the server's > shipping CRLF.CRLF and storing the message? Whoops, the server deleted > the message, and the client retry. Not necessarily. Consider a normal POP3 server, only that he does an implicit DELE after each RETR. Of course, a client could still fail to store the message (or to forward it somewhere, like fetchmail) without the server noticing. But only if the client tries to behave nicely and sends QUIT even after something went wrong. In this case, nice behaviour does not pay off. >> What changes do you plan for the seen tracking? To do it on the client >> side is already possible with the UIDL option. > UIDL is not an option, but a necessity. For simply retrieving mail, delivering locally, retry in case of failure? I don't see why. |
|
From: Matthias A. <mat...@gm...> - 2006-01-04 13:21:07
|
Jakob Hirsch schrieb am 2006-01-04:
> Well, RFC 1939 says "Such message deletions are outside the scope of the
> POP3 protocol and are not considered a protocol violation." (thanks for
> pointing me to this chapter, btw)
Right, but I took your report as though your server doesn't wait until
QUIT, in which case it *is* a violation.
> Site policies always override RFC-specified behaviour, there's not much
> you can do about that.
If these site policies aren't explicitly allowed by the RFC, they site
is not in compliance, and all bets WRT client function are off.
> > If talking to the admins doesn't resolve their bad behavior, find a
> > different server.
>
> It's my university's server...
So unless it's in the AUP ("Nutzungsbedingungen") or other relevant
documents, they're trashing your mail without informed consent, and thus
punishable according to German law.
Check the paperworks and unless you (perhaps implicitly) agreed to such
a policy, talk to them.
Most sites refuse new messages if the mailbox size exceeds a quota, or
delete old messages after a month or so. That would solve the problem
on either side: you can use several clients, they can limit the disk
space consumption.
> > Loss scenario: What happens if the client crashes between the server's
> > shipping CRLF.CRLF and storing the message? Whoops, the server deleted
> > the message, and the client retry.
>
> Not necessarily. Consider a normal POP3 server, only that he does an
> implicit DELE after each RETR. Of course, a client could still fail to
> store the message (or to forward it somewhere, like fetchmail) without the
> server noticing. But only if the client tries to behave nicely and sends
> QUIT even after something went wrong. In this case, nice behaviour does
> not pay off.
Will check.
> >> What changes do you plan for the seen tracking? To do it on the client
> >> side is already possible with the UIDL option.
> > UIDL is not an option, but a necessity.
>
> For simply retrieving mail, delivering locally, retry in case of failure?
> I don't see why.
1. Because it is the only reliable alternative to "fetchall nokeep",
independent of the current TOP-vs-RETR discussion.
2. Because the assumption fetchmail were the only client doesn't hold in
many cases (such as yours) and isn't reasonable either.
3. Because servers are often subjected to what I see as "empirical
programming" - write server code that appears to work with Outlook
Express at first glance, rather than writing standards compliant code
that is in compliance with RFC-1939.
--
Matthias Andree
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X