[Mingw-notify] [ mingw-Bugs-1959974 ] msys/bash command pipes not 8 bit clean

[SourceForge.net <no...@so...>]

Bugs item #1959974, was opened at 2008-05-08 12:55
Message generated for change (Comment added) made by anthonywesley
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=102435&aid=1959974&group_id=2435

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: MSYS
Group: Known bugs
>Status: Open
Resolution: Invalid
Priority: 5
Private: No
Submitted By: Anthony Wesley (anthonywesley)
Assigned to: Nobody/Anonymous (nobody)
Summary: msys/bash command pipes not 8 bit clean

Initial Comment:
Using I/O redirection through pipes from rxvt/bash is not 8 bit clean, one particular character - ascii(26) - causes the pipe to break.

This same behaviour is not seen under Linux, hence I think it's a bug not a feature :-)

OS Version: WinXP SP2

MSYS Runtime 1.0.10

How to produce:

- write a simple C program that write all 256 bytes to standard output with write()

- write a simple C program that read()'s and prints bytes from standard input.

run ./writer | ./reader from an rxvt/bash terminal.

the reader will stop prematurely when it reaches ascii(26).

I believe this may also be at the bottom of other bug reports on strange pipe behaviour in mingw.

regards, Anthony



----------------------------------------------------------------------

>Comment By: Anthony Wesley (anthonywesley)
Date: 2008-05-09 09:46

Message:
Logged In: YES 
user_id=2081890
Originator: YES

Thanks Keith, I added _setmode(fileno(stdin),O_BINARY) and fixed my
problem.

regards, Anthony

----------------------------------------------------------------------

Comment By: Earnie Boyd (earnie)
Date: 2008-05-09 09:24

Message:
Logged In: YES 
user_id=15438
Originator: NO

You need to try the 1.0.11 version.  I think this issue is resolved
already.  IIRC some instances of the pipe were O_TEXT instead of O_BINARY.

----------------------------------------------------------------------

Comment By: Anthony Wesley (anthonywesley)
Date: 2008-05-09 09:18

Message:
Logged In: YES 
user_id=2081890
Originator: YES

Keith,

is this the expected behaviour when connecting standard output of one
application to standard input of another using a pipe from the shell? I had
thought that should be 8 bit clean, else you can't do things like
decompress binary data and pipe it somewhere.

fyi I came across this behaviour when I was doing exactly that, "tar xzOf
file.tgz | /some/app/that/reads/binary/data" and it kept terminating
prematurely.

Shouldn't bash be creating O_BINARY pipes and not O_TEXT if that's what is
going on? It still seems like a bug, I don't see how this behaviour can be
considered correct?

cheers, Anthony

----------------------------------------------------------------------

Comment By: Keith Marshall (keithmarshall)
Date: 2008-05-09 08:15

Message:
Logged In: YES 
user_id=823908
Originator: NO

I think it is invalid, irrespective of the version.

Anthony has stated that his pipes block on ascii(26); this is *not* a bug,
as he suggests, nor is it anything to do with being 8-bit clean.  It is
simply normal behaviour for any stream opened in O_TEXT mode, (the
default), for ascii(26) is an explicit EOF mark on this type of stream.

Yes, this *is* a feature of MS-Windows, which will not be seen on Linux,
(or any other variety of Unix), for these make no distinction between
O_BINARY and O_TEXT, (and indeed, do not define those symbols).  To get
behaviour similar to that seen on Linux, the pipe must be opened
*explicitly* in O_BINARY mode.

This behaviour is as expected, (and as documented, IIRC).

----------------------------------------------------------------------

Comment By: Earnie Boyd (earnie)
Date: 2008-05-08 22:22

Message:
Logged In: YES 
user_id=15438
Originator: NO

Anthony, I've marked Invalid because of the version.  You'll need to test
with the 1.0.11 Technology Preview.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=102435&aid=1959974&group_id=2435