VMime

Hi,

I hope somebody can help me understand why the space seems to trigger QP encoding in the "from:" and "to:" header, but not in the "subject:".

Code sample:

vmime::messageBuilder mb;
mb.setSubject(vmime::text("Message Subject"));
mb.setExpeditor(vmime::mailbox(vmime::text("Message Expeditor"), "mabrand@mabrand.nl"));

Headers:

Subject: Message Subject From: =?us-ascii?Q?Message_Expeditor?= mabrand@mabrand.nl

The encoding seems unnecessary in both cases. Thunderbird does not encode these.

This QP encoding raises the spam hits in spamassassin for "FROMEXCESSQP".

X-Spam-Status: No, hits=-4.307 required=2.4 tests=ALLTRUSTED, AWL, BAYES00, FROMEXCESSQP, HTML90100, HTML_MESSAGE

Hello!

Actually, using spaces for a mailbox name in "From" is forbidden unless it is either encoded (eg. QP) or... quoted. I admit in this case, the name should be quoted, encoding is useless.

I will have a look at this, thanks!

Vincent

I'm a bit over my head here, but I think I agree that rfc822 forbids spaces in the name phrase without encoding or quoting. But, if I understand rfc2822 section 3.4, it seems to allow spaces without encoding or quoting. Such spaces would be folding white space. The rule I am relying on is:

angle-addr = [CFWS] "<" addr-spec ">" [CFWS] / obs-angle-addr

What do you think about this?

Hi,

Here is a more complete excerpt from section 3.4 of RFC-2822:

mailbox         =       name-addr / addr-spec
name-addr       =       [display-name] angle-addr
angle-addr      =       [CFWS] "<" addr-spec ">" [CFWS] / obs-angle-addr
display-name    =       phrase
word            =       atom / quoted-string
phrase          =       1*word / obs-phrase

CFWS is a place-holder for "Comment or Folding Whitespace". Here, the mailbox name ("Message Expeditor" in your case) corresponds to the "display-name" part, which is actually either an "atom" or a "quoted-string" (an atom must not contain whitespace).

The excerpt you wrote just means there may be whitespaces between the "display-name" part and the "" one.

Regards,

Vincent

You've convinced me. I was also relying on rfc2822 3.2.3 for the expansion of CFWS into spaces and comments. But now I see that comments seem to need to be in parentheses ().

Anyway, is there anything that can be said in defense of a popular email client that produces "display-names" containing spaces without encoding or quoting? If it's a bug it seems pretty egregious.

Maybe RFC-822 (previous version of RFC-2822) allowed spaces in display names, but I'm not sure, as I didn't have a look at it since a while now.

If this is the case, mail parsers should continue to understand such obsolete constructions, but an RFC-2822-compliant generator (like VMime) should not generate these anymore.

As read it, RFC-822 does not allow spaces in unquoted/unencoded display names either. I'm tempted to go check whether there is a Thunderbird or old Mozilla bug about this.

By the way, there might be something to be said for quoting subjects containing spaces. I have noticed recently that some mail lists seem to fold long subject lines at spaces. Not sure if that is correct behavior, but it causes Thunderbird to display ugly artefacts. Is Thunderbird to blame for not unfolding correctly? Would it be a good workaround to quote the subject to prevent this? If so, should VMime do this by default or provide a way for the user to select this workaround?

I'd appreciate reading your comments about this.

This link seems to be about the subject issue. It seems to involve possible problems in both mailman and certain MUAs. There's a link there to mailman bug.

http://mail.python.org/pipermail/mailman-users/2004-December/041549.html

Hello!

Here is a patch for quoting display names in mailboxes:

http://www.vmime.org/download/patches/vmime-20091111-quotemailboxname.patch

Please give it a try and tell me if it fixes the issue.

Thank you,

Vincent

Hi, I applied the patch to libvmime-0.9.0, but there seems to be a problem with extra digits and QP is still applied. The test case can be derived from example1.cpp with this patch:

diff -urN example1.cpp  example1a.cpp
--- a/example1a.cpp        2008-10-12 10:54:34.000000000 +0200
+++ b/example1a.cpp       2009-11-12 10:47:01.000000000 +0100
@@ -49,7 +49,7 @@
                vmime::messageBuilder mb;

                // Fill in the basic fields
-               mb.setExpeditor(vmime::mailbox("me@somewhere.com"));
+               mb.setExpeditor(vmime::mailbox(vmime::text("Message Expeditor"), "me@somewhere.com"));

                vmime::addressList to;
                to.appendAddress(vmime::create <vmime::mailbox>("you@elsewhere.com"));

Steps:

tar -jxf ../mingw-cross-env/pkg/libvmime-0.9.0.tar.bz2
cd libvmime-0.9.0
patch -p0  < ../vmime-20091111-quotemailboxname.patch
./configure --enable-debug
make
cd examples
cp example1.cpp example1a.cpp
patch  < a.patch
g++ -o example1a example1a.cpp -I.. -L../src/.libs/ -lvmime
LD_LIBRARY_PATH=../src/.libs/ ./example1a

Output:

Generated message:
==================
Subject: My first message generated with vmime::messageBuilder
From: "17"=?us-ascii?Q?Message_Expeditor?= <me@somewhere.com>
To: you@elsewhere.com
Bcc: you-bcc@nowhere.com

By the way, I think the README in examples should be updated. In step 2 the command should be:

g++ -o exampleX exampleX.cpp `pkg-config --libs vmime`

Hi!

I fixed the patch, same URL.

Vincent

Looks good. Thanks. I didn't notice the .length() either!

Generated message:
==================
Subject: My first message generated with vmime::messageBuilder
From: "Message Expeditor" <me@somewhere.com>
To: you@elsewhere.com

I would like to reopen the question of whether display names containing spaces require quoting or QP-encoding.

In my earlier post, I proposed an incorrect analysis of a display name containing spaces. You convinced I was wrong and also that the RFCs require quoting or QP-encoding for these cases. I posted a comment about this at https://bugzilla.mozilla.org/show_bug.cgi?id=273884

The reply to my comment points out the examples in Appendix A .1.1 of RFC 5322 (which obsoletes RFC 2822) which clearly show unquoted display names containing spaces. RFC 2822 has such examples too.

To make a long story short, I was indeed wrong in my analysis, but it now appears that the RFCs do not require quoting or QP-encoding of display names containing spaces.

Going back to the definitions in the RFCs, I now think the parse tree for the display name "Mary Smith" (without the quotes) looks like this:

[display-name [phrase [word [atom [atext "Mary"] [CFWS [FWS " "]] ] ] [word [atom [atext "Smith"] ] ] ] ]

Do you agree with this?

Hello!

Yes, it seems right in regard to the RFC.

I will have a look again at the code to see whether this can be changed.

For now, it is not a big issue, as quoting is still valid...

Vincent