fetchmail-devel

[fetchmail-devel] RFC: Workaround for comcast bug - 6.3.X or 6.4.X stuff?

[Matthias A. <mat...@gm...>]

Greetings,

I prepared this patch in response to the many comcast ("Maillennium")
problem reports all across the net, which blacklists the POP3 TOP
command on servers that greet us with "Maillennium POP3/PROXY server".

I'm asking for comments: do others think avoiding the truncation of
messages after approx. 82 kB justifies this incompatible bug fix (it is
incompatible because it marks messages read on the server)?

Should we apply this patch on the 6.3.X branch or should it wait for
6.4.X?

I also found out that many people try to ask on random forums,
vendor-forums such as SUSE, independent forums, rather than asking on
the official fetchmail support lists, and apparently without checking
the FAQ first.  How do users procure information where to post their 
question?

(I'd like to post a prominent notice referring them to our FAQ and lists.)


This is the patch:

--------------------------------------------------------------------------------
--- pop3.c	(Revision 4585)
+++ pop3.c	(Arbeitskopie)
@@ -611,6 +611,12 @@
 #endif
 
     set_peek_capable(ctl);
+    /* comcast's Maillennium POP3/PROXY is full of bugs and truncates
+     * TOP replies after c. 80 kByte, so disable TOP. */
+    if (peek_capable && strstr(greeting, "Maillennium POP3/PROXY server")) {
+	report(stdout, GT_("Warning: Maillennium POP3/PROXY server found, using RETR command.\n"));
+	peek_capable = 0;
+    }
 
     /* we're approved */
     return(PS_SUCCESS);
--- NEWS	(Revision 4597)
+++ NEWS	(Arbeitskopie)
@@ -24,6 +24,17 @@
 
 fetchmail 6.3.2 (to be released):
 
+# INCOMPATIBLE CHANGE:
+* Automatically disable the POP3 TOP command if the greeting string contains
+  "Maillennium POP3/PROXY server", which is used by comcast and known to
+  truncate messages after 80 kByte. Fall back to RETR, and complain if we had
+  used TOP otherwise.  Suggested by Ed Wilts.
+  *Note* that this means messages are marked read on these servers, which is a
+  deviation from how 6.3.1 behaved, but we have no alternative, comcast haven't
+  fixed this bug in years.  Preventing the loss of the remainder of the message
+  justifies this incompatible fix.  Matthias Andree
+
+# BUG FIXES:
 * SMTP/LMTP cleanup to fix these two bugs:
   - switch back to SMTP after having tried LMTP hosts (multiple smtphost hosts)
   - switch back to LMTP after sending a bounce.
--- fetchmail-FAQ.html	(Revision 4585)
+++ fetchmail-FAQ.html	(Arbeitskopie)
@@ -2004,13 +2004,19 @@
 <h2><a id="I8" name="I8">I8. How can I use fetchmail with comcast.net?</a></h2>
 
 <p>Stock fetchmail will work with a comcast.net server...<em>but</em>
-the Maillennium POP3 server comcat uses seems to have an 80K limit on
+the Maillennium POP3 server comcast use seems to have an 80&nbsp;kB limit on
 the length of downloaded messages if you use POP3 TOP to retrieve.
 Anything larger is silently truncated.  Don't mistake this for a
 fetchmail bug. (Reported July 2003.)</p>
 
-<p>Workaround: use the <tt>fetchall</tt> option.</p>
+<p>Beginning with version 6.3.2,
+fetchmail will fall back to the RETR command if the greeting string
+contains "Maillennium POP3/PROXY server". This means however that
+fetchmail has no means to prevent the "seen" flag from being set on the
+server.</p>
 
+<p>Workaround for older versions: use the <tt>fetchall</tt> option.</p>
+
 <hr/>
 <h1>How to set up well-known security and authentication
 methods</h1>
--------------------------------------------------------------------------------

Thanks,

-- 
Matthias Andree

Re: [fetchmail-devel] RFC: Workaround for comcast bug - 6.3.X or 6.4.X stuff?

[Rob M. <rob...@gm...>]

On 04/01/06, Matthias Andree <mat...@gm...> wrote:
> Greetings,
>
> I prepared this patch in response to the many comcast ("Maillennium")
> problem reports all across the net, which blacklists the POP3 TOP
> command on servers that greet us with "Maillennium POP3/PROXY server".
>
> I'm asking for comments: do others think avoiding the truncation of
> messages after approx. 82 kB justifies this incompatible bug fix (it is
> incompatible because it marks messages read on the server)?

I think that, assuming the warning is logged every time fetchmail
starts up (ie once in daemon mode, every time when run by hand), then
this isn't a bad thing.

I'm in favour of fetchmail being as flexible as it can when dealing
with remote POP/IMAP/whatever servers.  If it becomes necessary to
build a table of cludges to cope with badly broken servers, just so
that fetchmail isn't labelled as being incompatible (and the usual
"well it works under Windows, so it must be a Linux thing"), I think
it's a small price to pay.

Of course, I'm not the one doing the coding, so I don't have to suffer
the mess that'll result :)

> Should we apply this patch on the 6.3.X branch or should it wait for
> 6.4.X?

I'd probably go with 6.3.X - it could be argued that this is a bug
fix.  I suppose it depends on when 6.4.0 is likely.  If it's going to
be a long time off then it's probably worth adding the patch now.  If
however 6.4.0 is likely in the next 6 months or so, then I'd say
waiting to 6.4.0 shouldn't be too much of a pain.

> I also found out that many people try to ask on random forums,
> vendor-forums such as SUSE, independent forums, rather than asking on
> the official fetchmail support lists, and apparently without checking
> the FAQ first.  How do users procure information where to post their
> question?

I think they pull them from their <censored>.  Sadly fetchmail is far
from unique here, I've seen behaviour like this many times.

> (I'd like to post a prominent notice referring them to our FAQ and lists.)

How about including the URLs of the FAQ and the lists in the output of
the help output, and the FAQ URL above the output of "fetchmail -v -v"
and above?

Probably wouldn't hurt to include the URL for the FAQ in the man page,
at the top (yes, the fetchmail page URL is at the bottom, but who
reads that far?).  Also at the bottom of the list footer?  Ram it in
their faces :-)

Finally, I think that as long as ESRs page exists and makes no
reference to the new project, there are going to be problems.

--
                 Please keep list traffic on the list.
Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-devel] RFC: Workaround for comcast bug - 6.3.X or 6.4.X stuff?

[Matthias A. <mat...@gm...>]

Rob MacGregor <rob...@gm...> writes:

> On 04/01/06, Matthias Andree <mat...@gm...> wrote:
>> Greetings,
>>
>> I prepared this patch in response to the many comcast ("Maillennium")
>> problem reports all across the net, which blacklists the POP3 TOP
>> command on servers that greet us with "Maillennium POP3/PROXY server".
>>
>> I'm asking for comments: do others think avoiding the truncation of
>> messages after approx. 82 kB justifies this incompatible bug fix (it is
>> incompatible because it marks messages read on the server)?
>
> I think that, assuming the warning is logged every time fetchmail
> starts up (ie once in daemon mode, every time when run by hand), then
> this isn't a bad thing.

Well, the warning is logged every time. After all, we want to annoy
users a bit and have them pester Comcast.

> Of course, I'm not the one doing the coding, so I don't have to suffer
> the mess that'll result :)

No, but you keep pointing people to the FAQ and --fetchall otherwise :-)

>> (I'd like to post a prominent notice referring them to our FAQ and lists.)
>
> How about including the URLs of the FAQ and the lists in the output of
> the help output, and the FAQ URL above the output of "fetchmail -v -v"
> and above?

Sounds reasonable.

> Probably wouldn't hurt to include the URL for the FAQ in the man page,
> at the top (yes, the fetchmail page URL is at the bottom, but who
> reads that far?).  Also at the bottom of the list footer?  Ram it in
> their faces :-)

My plan.

> Finally, I think that as long as ESRs page exists and makes no
> reference to the new project, there are going to be problems.

Yes, but we cannot do much about that except grab a phonebook and call
him up. At least this procedure worked with bogofilter at some point in
time...

-- 
Matthias Andree