Re: [fetchmail-users] Re: RETR with UIDL

[Matthias A. <mat...@gm...>]

Sunil Shetye <sh...@bo...> writes:

> Quoting from Matthias Andree's mail on Fri, Mar 03, 2006:
>> > Please note that the change I am recommending is that fetchmail
>> > *should* use RETR with pop3+uidl servers. Other pop3 servers will not
>> > be affected by this change.
>> 
>> Yes, it's still 6.4.X stuff since it deliberately changes behavior for
>> no good reason.
>
> There is a good reason.
>
> The advice being given to force fetchmail to avoid using "TOP" with
> these buggy servers is to enable the "fetchall" option. The FAQ is
> littered with this advice for various servers with variety of
> problems. The problems with "fetchall" option are:
>
> - "fetchall" cannot be used with "uidl"

Well, we can fix eventual incompatibilities between these two options
any time. 6.3.X is open for bug fixes.

> - "fetchall" causes mails to be redownloaded if there is a line-hit
> while fetchmail is downloading mails. If there are consecutive
> line-hits, the same set of mails could be downloaded any number of
> times.

Yes, true. So use just uidl without fetchall on flakey lines. There is
little reason to add fetchall to uidl. We can decouple TOP/RETR from
fetchall in 6.4.X and default to RETR if we want. Not in 6.3.X.

Let's do 6.3.3 (thanks for your efforts in getting the IMAP NOOP bugs
nailed BTW), and then fork 6.4.X and make all interesting changes and
cleanups in particular in 6.4.X.

> To avoid the redownloading of mails, one has to now probably add the
> "expunge" option also. Finding the best value for "expunge" is a
> problem itself:
>
> - "expunge 1" is expensive. It will reconnect for every mail. Many
>   mailservers may not like this too. However, each mail will be
>   downloaded only once.
> - "expunge 2" is less expensive. Mailservers may still object to this.
>   Very few mails will be redownloaded.
> - "expunge 20" is relatively cheap. Mailservers may not object to
>   this. Some mails will get redownloaded. However, on a bad day, if
>   you do get consecutive line-hits, you will still end up with
>   multiple copies of mails.

The best option would be to use UIDL. Still we'll not be forcing UIDL
downloads to use RETR in 6.3.X. We can safely do that in 6.4.X: the
6.3.X manpage already has warnings in place.

>> fetchmail 6.3.2 already introduced the (incompatible) "use RETR with
>> Maillennium server" change because it was considered we'd rather screw
>> some message flags and a LAST pointer rather than truncate messages.
>
> I don't get this part. If the pop3 server is supporting UIDL and
> fetchmail is running with the "uidl" option, why bother about the LAST
> pointer and message flags?

Why bother? Because I promised I'd stop the ESR madness of gold versions
and promised 6.3.X would be compatible with 6.3.0 when I released 6.3.0.

> And, as has already been said, most standard interactive clients use
> "uidl" anyway. So, they also will not get affected.

This is not a valid reason to change behavior in incompatible ways in
the midst of a stable version. I'd wondered if I'd wanted to add the
Maillennium workaround in 6.3.X at all or delay it until 6.4.X. There'd
been loads of reasons to defer it, not the least being a documented
workaround.

> Jakob, please give more details about the Mercury server:
>
> - its signature line
> - its CAPA output
> - whether it supports UIDL
> - whether you are using it with "uidl"
> and finally,
> - your opinion on whether fetchmail should use "RETR" with "uidl".

I'm not accepting changes into 6.3.X that switch TOP to RETR except if
they fix security bugs, data corruption or other critical issues, so
please don't spend your time before we've opened 6.4.X for hacking.

Priorities for incompatible changes are:

1. Security
2. Critical bugs (data loss, crash, similar)
3. Compatibility within stable releases
4. Protocol compliance
5. Consistency
6. Anything else

-- 
Matthias Andree