[maildropl] Maildrop handling subject containing unicode characters

[Milan O. <cou...@di...>]

Hi,

recently I found a problem in maildrop when handling subject containing
unicode characters. Occasionally, I am experiencing mail not being
delivered to some remote server with error message being (last line)

553 Unable to deliver Unicode E-mail to a non-Unicode mail server.

After some investigation I found it comes from my maildrop script doing
some kind of mailing list, and now I have a simple test case.

I added to my maildrop script (another mailbox, so I can test things
without disturbing my application) following:

if (/^Subject: *(Test .*)/)
 {Subject=$MATCH1
  echo "subject $MATCH1"
  xfilter 'reformail -i"Subject: $Subject"'
 }

This should just check for subject starting with Test to invoke some
action, replacing the subject with new subject, constructed, but for
this example it should be the same.

Things get broken when original subject contains unicode characters.
For test, I sent to myself a mail with subject 'Test ♪♫'. From my mail
client, this is encoded as

Subject: Test =?UTF-8?B?4pmq4pmr?=

My maildrop script snippet logs then following in maillog:

subject Test <E2>M-^Y<AA><E2>M-^Y<AB>

and mail being delivered to my mailbox has subject rewritten as

Subject: Test <E2><99><AA><E2><99><AB>

or, as a hexdump, modified for clarity,

53 75 62 6a 65 63 74 3a 20 54 65 73 74 20 e2 99 aa e2 99 ab 0a
S  u  b  j  e  c  t  :     T  e  s  t

It is now understandable that mail with subject mangled this way could
be rejected by some external mail server. Mostly it works, but this is
just kind of happy accident, given the spec.

My question is, how should this be handled correctly. If I understand
what happens, $MATCH variable contains matched text in internal Unicode
binary encoding, and it should be encoded back using either base64 or
quoted printable character encoding, if some non ASCII characters are
in string given.

Any hints appreciated...

Regards,
Milan