Re: [Zymail-devel] line wrap

[Tim J. <tj...@to...>]

On Jan 2, 2008, at 2:22 PM, Joe Strout wrote:

> On Jan 2, 2008, at 1:58 PM, Tim Jones wrote:
>
>> When Jason changed the mail list manager in 2006, we all started
>> seeing our mails trnucated.  It raised quite the dust storm and
>> resulted in a few feature requests to return things to the pre-change
>> settings where paragraphs/long test lines were wrapped by the mail
>> client.  The biggest issue was the wrapping of links.  We all
>> (apparently incorrectly) assumed this was something Jason/Mailman
>> did.
>
> I still believe it was.
>
>>   However, since your email from Zymail got through "unwrapped" -
>> at least on my end - it appears that the change may have actually
>> been in a Mac Mail update that occurred at the same time.
>
> I don't see how that follows at all.  We have observed:
>
> 1. Mail sent to the RS list with many clients was unwrapped before  
> 2006.
> 2. Mail sent to the RS list with many clients was overly wrapped
> after 2006, and as I recall, Jason said that this was caused by a
> change in the list manager.
> 3. My unwrapped Zymail message, sent to a SourceForge list and NOT to
> any RS list, came through unwrapped.

Both RS and SourceForge use Mailman.  RS has placed a wrapper on it,  
but I believe that Jason indicated it was still Mailman underneath.

> I don't see how observation 3 has anything do do with observations 1
> and 2, and would allow us to conclude anything at all.  Again, please
> spell out how you reach this conclusion in simple steps.

Since both use the same list manager, I would expect both to wrap the  
text.  However, I've now done more tests and I retract my statement  
as mail sent to my ./Mac account from this account arrive unmolested  
and unwrapped.  Therefore, Apple Mail client must not be the guilty  
party in the artificial wrap scandal.

> Incidentally, your message (to which I am now replying) appears to
> have hard line breaks in it, at least as viewed by Apple Mail.  Mine
> from Zymail, sent to the same list, does not.  From THESE two
> observations we can conclude that your client differs from Zymail's
> current wrapping behavior, and that the SourceForge list doesn't hard-
> wrap lines (which is as it should be; a well-behaved mailing list
> will leave messages intact as much as possible, just as a well-
> behaved text/plain email client will send its lines already
> wrapped).  But it still doesn't say anything about RS's mailing lists
> or Apple Mail in particular.

I've sent you an email from this same account off-list where I've  
unwrapped most of these paragraphs.  is it wrapped when you receive it?

>
> It seems to me that to verify what you're claiming, we'd need to send
> a clearly unwrapped message to one of RS's mailing lists and see how
> it comes back.
>
>>> For styled text, sure, but not for plain text.  Some (admittedly
>>> poorly-written but nonetheless common) MTAs will truncate or
>>> otherwise abuse lines longer than 76 characters.  Staying under 72
>>> characters is good practice for a MUA.  See RFC 1521 for example:
>>> <http://www.math-inf.uni-greifswald.de/~teumer/mime/1521/
>>> Appendix_B.html>
>>>
>>> (Mind the line wrap!)
>>
>> The point exactly...
>
> Yes, but that is an issue only if a URL gets broken, and the
> receiving client fails to put it back together.  A sensible line-wrap
> algorithm will not break a URL (note to self: check whether
> StringUtils.WrapLines handles this), and a sensible client will
> repair any URL surrounded in angle brackets anyway.  My "Mind the
> line wrap!" comment was meant to be ironic.
>
>> However, the line length definition as presented in 1522 states:
>>
>> " (4) Lines longer than 76 characters may be wrapped or truncated in
>> some environments. Line wrapping and line truncation are STRONGLY
>> DISCOURAGED, but unavoidable in some cases. Applications which
>> require long lines must somehow differentiate between soft and hard
>> line breaks.  (A simple way to do this is to use the quoted-printable
>> encoding.)"
>
> That's a description of how MTAs (Mail Transfer Agents, e.g. email
> servers) should behave.  We're writing a MUA (Mail User Agent), not
> an MTA.  Again, this just reemphasizes the general principle: every
> agent should both follow standards as much as possible, and be
> prepared to deal with other agents that fail to follow the standards.
>
> The standard for text/plain data is to wrap at 80 characters or less
> (though 72 is safer, allowing several levels of quoted before you
> have to get fancy).  Though as noted earlier, we still need to look
> at the later RFCs discussing format=flowed.
>
>> But, according to 1522, the creation should not artificially wrap
>> text.
>
> No.  1521 states that mail *servers* should not artificially wrap
> text they receive from mail clients.  (And it points out that many
> servers do so anyway, so to be safe, mail clients should anticipate
> that).

Re-read that Appendix B section:

" There exist many widely-deployed non-conformant MTAs in the Internet.
    These MTAs, speaking the SMTP protocol, alter messages on the fly to
    take advantage of the internal data structure of the hosts they are
    implemented on, or are just plain broken.

    The following guidelines may be useful to anyone devising a data
    format (Content-Type) that will survive the widest range of
    networking technologies and known broken MTAs unscathed."

That sounds to me like it is specifying the manner in which a MUA  
should prepare the content for transmission.  And, from the last  
paragraph in Appendix B:

"  Please note that the above list is NOT a list of recommended
    practices for MTAs.  RFC 821 MTAs are prohibited from altering the
    character of white space or wrapping long lines.  These BAD and
    illegal practices are known to occur on established networks, and
    implementations should be robust in dealing with the bad effects  
they
    can cause."

So getting back to the issue at REAL and their mail list  
implementation, I revert to the feature request submitted when the  
change occurred and retract my placing the blame on Apple Mail :-).   
The REAL server should not be modifying the mail as transmitted so  
long as it meets plain/text definitions.

>
>>> I didn't see that.  Both your response and my original looked the
>>> same to me.
>>
>> Did your original expand past the 72 character width for you?  For
>> me, it flowed to the width of my page (around 158 characters) while
>> my reply wrapped everything - including your original.
>
> Yes, I see that too.
>
>> As for the other RFC's, I believe that they refer to handling
>> received text, not sent text.  I believe that we should not modify
>> the sender's text outbound and allow teh recipients' mail client
>> (Zymail or otherwise) decide what to do with the incoming data.
>
> "Not modify the sender's text outbound" here is deceptive.  WE are
> the sender.

No we're not.  The author of the email is the sender I was referring  
to.  If I want hard returns, I'll place them.  Otherwise, if I type  
my paragraph continuously (as in this one), I want it sent as I type  
it.  Again, it's the receiver app's job to manage the display of the  
incoming data.

> It's up to us -- the Zymail developers -- to send text
> according to the standards.  For regular old text/plain content, that
> clearly means line breaks.

It actually doesn't.  "text/plain" only implies non-formatted text in  
concert with RFC 822, not that it must contain line breaks.

>     But I think the other RFCs need further
> study.  In a nutshell, I believe that if you set the Content-Type
> correctly and use " \n" instead of just "\n" for your inserted
> breaks, then savvy receiving clients will interpret those as soft
> breaks and reflow them with the window width, while older clients
> will still display the text properly wrapped.

But, if we send the text content unmolested, the result is any MUA  
recipient will properly display the received data as determined by  
the local settings.  The only thing that plain text should change is  
the removal of any type of non-ASCII formatting in compliance with  
RFC 822.

Tim