Menu

#30 Stripping trailing whitespace from headers breaks messages

v1.0 (example)
closed-works-for-me
nobody
None
5
2018-06-20
2015-05-22
No

I configured emailrelay to forward mail using a verifier and exit status 1. I
found that the forwarded mail was getting garbled and some headers were getting put in the body.
I did some debugging and found that it seems to be because emailrelay strips
leading tabs from the headers (e.g. the Received header), and then the remote server (Postfix in my case)
sees the header lines that should be indented and thinks that they are the
start of the message body.

Here is an example of what happens with the messages in this situation (with
hostnames changed), showing a message sent from cel@example.com (Postfix)
to root@example.org (emailrelay).

Here is a message delivered locally by emailrelay:

Received: from example.com ([removed]) by ExampleCom with ESMTP ; Fri, 22 May 2015 01:39:22 -0400
Received: by example.com (Postfix, from userid 1000)
        id 662C640F69; Fri, 22 May 2015 01:36:42 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com;
        s=20150507; t=1432273002;
        bh=nypZpg5l+81aPhtySK35KJDOOjKxnkP7R1HCZXGW3hM=;
        h=Date:To:Subject:From:From;
        b=RgI0E4TLQITvof+Y5QkNFngLrSgHRSbstL/cqyN8Zl/yAbYJJ4+ldNqQIDxfUlGGz
         57LSs6kp+25/WcdSpDDcVjOWRU/ehYLXY3RobN383PbxFTNd4l3FAAzDt4xm6bhdCH
         mnF8CtYn0xfXmvUbY/rh2YFzaC8NA7kpIN6HtoMNFkm7ko33f+7Ruw5SHlozE1fgdO
         N19MI3XGFZ7YBVuDDCcNtceVimJqlTyPhPqvFx8msGeHGmQQ6NQzzXHoyeZhgQer4M
         ahpfZd0gjhrANwEQsEBSojcQrRtqhXR96Nqe9D2GVu1BfeLSKvdraDyccxF7SLzRaO
         y0KtW5cZjldlQ==
Date: Fri, 22 May 2015 01:36:42 -0400
To: root@example.org
Subject: hi23
User-Agent: Heirloom mailx 12.5 6/20/10
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <20150522053642.662C640F69@example.com>
From: cel@example.com

ok

Here is the data that emailrelay sends to the remote server when I configure it
to forward using the verifier back to example.com:

Received: from example.com ([removed]) by ExampleCom with ESMTP ; Fri, 22 May 2015 01:55:20 -0400
Received: by example.com (Postfix, from userid 1000)
id F1F4C40F69; Fri, 22 May 2015 01:55:17 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com;
s=20150507; t=1432274117;
bh=nypZpg5l+81aPhtySK35KJDOOjKxnkP7R1HCZXGW3hM=;
h=Date:To:Subject:From:From;
b=fPQGi2K/1AvZKwLNCePsiKJMONghhVT0XZh/x+tueJ1Ucdr31I0xU5OlOYL3eGnfU
V6QUjiO7bHRX29Wihz9uh/3STlEMhpfe4xRqajiFBo1Hfo+SiKg4VnWhhx3C5RD9QV
afcofreYHEREGuqPZBeSnhCmpYYukzMFTFDlufFadCEF3+Q8pZvGViWZMVoNOWSdAE
nkppLEMOkimEi+XAwO9GICZgWr66UH6XneukIO/tkGCFQPJw93ulhEC6fl2nOMnGJA
Q8m9bM9I9DyyfW3qTAfRletq5DN9+7OKoW3mSdaFnVDr3IhIzJmnH+Jjqxc13T0+u9
raZrndiMKTs8A==
Date: Fri, 22 May 2015 01:55:17 -0400
To: root@example.org
Subject: hi27
User-Agent: Heirloom mailx 12.5 6/20/10
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <20150522055517.F1F4C40F69@example.com>
From: cel@example.com
ok

And here is what the remote server delivers:

Return-Path: <cel@example.com>
Delivered-To: <emailrelay.cel@example.com>
Received: from example.com
    by example.com (Dovecot) with LMTP id rG7hBNO7XlWMMgAAfa0aaw
    for <emailrelay.cel@example.com>; Fri, 22 May 2015 01:17:07 -0400
Received: from ExampleCom ([removed])
    (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits))
    (No client certificate requested)
    by example.com (Postfix) with ESMTPS id D032340881
    for <emailrelay.cel@example.com>; Fri, 22 May 2015 01:17:06 -0400 (EDT)
Authentication-Results: example.com; dmarc=fail header.from=example.com
Received: from example.com ([removed]) by ExampleCom with ESMTP ; Fri, 22 May 2015 01:17:06 -0400
Received: by example.com (Postfix, from userid 1000)
Message-Id: <20150522051706.D032340881@example.com>
Date: Fri, 22 May 2015 01:17:06 -0400 (EDT)
From: cel@example.com

id C194140F69; Fri, 22 May 2015 01:16:35 -0400 (EDT)
Date: Fri, 22 May 2015 01:16:35 -0400
To: root@example.org
Subject: hi22
User-Agent: Heirloom mailx 12.5 6/20/10
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <20150522051635.C194140F69@example.com>
From: cel@example.com
ok

Notice the Received header that originally looked like this:

Received: by example.com (Postfix, from userid 1000)
        id 662C640F69; Fri, 22 May 2015 01:36:42 -0400 (EDT)

gets split, with the first line staying in the headers and the second getting
put into the body. I think Postfix sees that the line starting with "id"
doesn't have a colon and therefore isn't a header and so decides it is that
start of the message body.

I looked in the emailrelay source and found in
GSmtp::ServerProtocol::commandLine() at src/gsmtp/gserverprotocol.cpp:611:

G::Str::trimLeft( line , " \t" ) ;

I haven't yet set up my build environment properly to test if removing this fixes
my issue, but I think that line is what is causing it. Are there cases in which
it is beneficial to remove leading tabs and spaces from header lines?

Discussion

  • Charles Lehner

    Charles Lehner - 2015-05-22

    I found another issue. In the example above, in the middle message where I am using emailrelay as a proxy to example.com (it's a little confusing because I use example.com as both the sender host and the --as-proxy host), the blank line between the headers and body is missing. In the .content file (or .local as in the first message above), the blank line is correctly present, but when the message is transmitted to the remote server, blank lines after the headers are removed. Postfix is able to figure out where the body begins without the blank lines, but my ISP's mail server is more strict and rejects the message with "5.6.0 Invalid header found (see RFC2822 section 3.6)", probably because it reads the first line of the body and tries to interpret it as a header.

    So after some more testing it seems that any blank lines as well as leading whitespace in the message content (headers + body) are getting removed by emailrelay's SMTP client. This can break delivery and garble messages as I've experienced, and could interfere with other things where whitespace is significant, such as PGP-signed messages. Have other people encountered these issues?

    P.S.
    I made a mistake with the title of this bug since I meant to refer to leading whitespace, not trailing. But since it's also blank lines being stripped, and not just in the headers, the title should rather just be "Stripping whitespace breaks messages"

     
  • Graeme Walker

    Graeme Walker - 2018-05-13

    That is all very odd, because by design emailrelay does not mess with message content except to add a Received line at the top. Your "trimLeft" snippet is surely not relevant as it only applies to SMTP commands (like AUTH and DATA), not message content. I can only assume (at this distance) that there was confusion about line endings; emailrelay adds a Received line with CR-LF, but if the rest of the content was just LF then postfix might have been too liberal in its parsing and made a bad situation worse.

     
  • Graeme Walker

    Graeme Walker - 2018-06-20
    • status: open --> closed-works-for-me
     

Log in to post a comment.