[Mingw-notify] [ mingw-Bugs-1437313 ] MSYS shell introduces extraneous SOH bytes in expressions

[SourceForge.net <no...@so...>]

Bugs item #1437313, was opened at 2006-02-23 10:08
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=102435&aid=1437313&group_id=2435

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: msys
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Keith Marshall (keithmarshall)
Assigned to: Earnie Boyd (earnie)
Summary: MSYS shell introduces extraneous SOH bytes in expressions

Initial Comment:
Command substitutions, delimited by either back-quotes
or by $(...), introduce an extraneous SOH byte
following the substituted text, when the substitution
itself is enclosed within a double quoted expression
which extends beyond the end of the command substitution.

This appears to affect only substitutions involving
CRLF delimited text output, as illustrated by:

  $ echo "`printf 'foo\n'` bar" | od -cb
  0000000   f   o   o       b   a   r  \n
          146 157 157 040 142 141 162 012
  0000010

  $ echo "`printf 'foo\r\nbar\r\n'`" | od -cb
  0000000   f   o   o  \r  \n   b   a   r  \n
          146 157 157 015 012 142 141 162 012
  0000011

  $ echo "`printf 'foo\r\nbar\r\n'` baz" | od -cb
  0000000   f   o   o  \r  \n   b   a   r 001       b 
 a   z  \n
          146 157 157 015 012 142 141 162 001 040 142
141 172 012
  0000016

  $ echo "`printf 'foo\r\nbar\n'`"" baz" | od -cb
  0000000   f   o   o  \r  \n   b   a   r       b   a 
 z  \n
          146 157 157 015 012 142 141 162 040 142 141
172 012
  0000015

  $ echo "`printf 'foo\r\nbar\n'` baz" | od -cb
  0000000   f   o   o  \r  \n   b   a   r       b   a 
 z  \n
          146 157 157 015 012 142 141 162 040 142 141
172 012
  0000015

Observe that:

1) The extraneous byte seems always to be SOH.

2) The extraneous byte appears *only* when the output from
   the substituted command is CRLF delimited text,
*and* the
   double-quoted expression extends *beyond* the end of the
   command substitution.

3) It is only the terminal CRLF, on the final line of
output
   from the substituted command, that participates in the
   generation of the extraneous byte; any other CRLF line
   terminators, appearing earlier in the output, remain
   unchanged in the substitution.

4) If the end of the command substitution is coincident
with
   the end of the double-quoted expression, then *both* the
   terminal LF *and* its associated CR are dropped from the
   substituted command output;  OTOH, if the double-quoted
   expression extends beyond the end of the substitution,
   then only the terminal LF is dropped, and its associated
   CR is *replaced* by SOH.

Repeat these examples on a GNU/Linux box produces
identically
the same results, *except* that CRs are preserved
unchanged, in *all*
cases, e.g.

  $ echo "`printf 'foo\r\nbar\r\n'` baz" | od -cb
  0000000   f   o   o  \r  \n   b   a   r  \r       b 
 a   z  \n
          146 157 157 015 012 142 141 162 015 040 142
141 172 012
  0000016

From this, it appears that the problem lies
specifically in the
mishandling of line terminators, when the output from
the substituted
command is CRLF delimited text.  Even more
specifically, it lies in
the inconsistent handling of the terminal CRLF,
depending on whether
the end of the command substitution is coincident with
the end of the
enclosing double-quoted expression, or that expression
extends beyond
the end of the command substitution, i.e.

a) in the case of coincident ends of substitution and
expression,
   the terminal CRLF is discarded, in its entirety.

b) in the case of the double-quoted expression
extending beyond
   the end of the command substitution, only the
terminal LF is
   discarded, and its associated CR is erroneously
replaced by
   a single SOH.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=102435&aid=1437313&group_id=2435