#786 Redirecting commands with binary streams

[Rony G. Flatscher]

Redirection of stdin and stdout ought to be possible with binary streams as well. The current implementation seems to take text streams only into account.

Here the information from the mail to the developer list (https://sourceforge.net/p/oorexx/mailman/message/37115232/):

-------- Forwarded Message --------
Subject:    Re: [Oorexx-devel] Question ad redirected commands
Date:   Thu, 24 Sep 2020 13:25:46 +0200
From:   Rony G. Flatscher <Rony.Flatscher@wu.ac.at>
Reply-To:   Open Object Rexx Developer Mailing List <oorexx-devel@lists.sourceforge.net>
To:     oorexx-devel@lists.sourceforge.net


The current implementation of the redirection of commands seems to assume that only text data gets feeded and output, which is not true. All operating systems allow one to pipe binary data as well and therefore there is a need to process binary data on input and output.

Now, it may be true that most of the times text gets piped, such that the current implementation in ooRexx is ideal for that.

However, there is a need for being able to process binary data as well.

Maybe one could indicate the fact that binary data is to be processed by adding an optional keyword BINARY, e.g.

    ...
    address system "readbinary.rex < in\bsf4oorexx_016.png" with input binary output using binary (xyz)
    ...

Or maybe another possibility would be to imply binary if the expression in "using (xyz)" yields a string value?

---rony



On 23.09.2020 15:34, Rony G. Flatscher wrote:
>
> Is it possible to redirect commands using binary data for input and output?
>
> If so how can one achieve this?
>
> ---
>
> Example: binary file (a graphic), size 976 can be processed with this Rexx filter program "readbinary.rex":
>
>     allchars=readall()        -- read from stdin using charin()
>     .output~charout(allchars) -- write binary
>
>     ::routine readall
>       mb=.mutableBuffer~new
>       signal on notready
>
>       do counter i forever
>          mb~append(.input~charin) -- supplying a length does not work
>       end
>
>     notready:
>       .error~say(i "charin reads")  -- in this sample yields: "977 charin reads"
>       return mb~string
>
> Running the above program with "readbinary.rex < image.png > output.png" works, the resulting file is identical to the original one.
>
> ---
>
> Trying to fetch the binary data with redirecting .output to an .array breaks up the binary data and swallows CR-LF characters it seems, e.g.:
>
>     out=.array~new
>     cmd="readbinary.rex < in\bsf4oorexx_016.png "
>     say "cmd:" cmd
>     address system cmd with output using (out)
>     say "out~items:" out~items
>     sum=0
>     do counter i data over out
>        sum += data~length
>        say "#" i":" edit(data)
>     end
>
>     .error~say("test2.rex: sum="sum)
>
>     ::routine edit
>       parse arg data
>
>       str1="length=["|| (data~length)~right(4)"]"
>       str1=str1 data~left(min(3,data~length))~c2x~left(6) "|"
>
>       if data~length>3 then
>           str1=str1 "right(3)~c2x:" data~right(min(3,data~length))~c2x~left(6)
>       else
>       do
>          len=data~length-3
>          if len>0 then
>             str1=str1 "right("len")~c2x:" data~right(min(3,len))~c2x~left(len*2)
>       end
>       return str1
>
> In my test case this results in eight array items, and adding up the eight chunks is eight characters short compared to the original. Here the output:
>
>     G:\test\oorexx\redirect>test2
>     cmd: readbinary.rex < in\bsf4oorexx_016.png
>     977 charin reads
>     --->
>     89504E470D0A1A0A0000000D49484452000000100000001008060000001FF3FF61000003974944415478DA4D936D6C53651886EFF7B4A767A7675DCB76D6F6281B04C73
>     6666580637E30995F307F8C1F10C090684C8C46C948FC831A2526447F982846D1CC294403314CD998A213AB9B83B10D261A452CD2B18F6EEBBAB6EBFABDF5F4F39CE329
>     29C437B9F224CFFBBC77F23EB96F82C25194F31AB5702A6695BBF335329BB6DCEC9F78686AF84AD55232E6BEEFB9AD97EA9B6D3EBD5EE7A528CAA7CE84C8FF047492833
>     4C9E3AE77820BEE8DA246A1E30B29B8A67DD9A5409099F7B889B98A975A9EBADF49B73E7C84E74BDC34AD9DBF2390EBB73F9FBA30D0E12461CAB4BB412BBA72983C338A
>     E0821F04125C0137529B56635B99193506665CF742EBBB66B3C97D4B20DBD7D79239D36DFF830AA6F9B6478B4A743C5C1F0DE1A66B0AA488401B4A603CE487A5750B569
>     6B2305E9ECC89D6E25EDB6B7BED44191E2E53D89999B495E3321C4D14686018F4E2EB8E6F20B204B9C412921111114941FDF626A4E76611BEEE9293D178ACF4959D5F10
>     D977EE8A6F31D478F1DAAC52B28225F5B66A08B2163FB6772332E5413012458052C0180CA8B158A111534878BC4A389B22630AFA883474DC1B1338E1A537BB70EA541B0
>     EEC3F8AF70EB7E1BBCE5FE0199FC6355544A635B0AD1260F12E83613930EA4EA6265CC808A62491FE39911BF2C4C8D163BDD4F66DF782058D7DFB5A70E4D3B3D8B2B906
>     F1641283E7FFC25AC104C37400E1313F38BD1E0A91E1F0F963449AEF513EFCEAA242E834191D75E274FBAB985C08A3B367100D0DEBB0F5914D387776045AF51BAB4C2C2
>     E74FC00D113458950A64CA4C4189918ED901EDFF316F5F1270770F2643F5E7F71077EFDDD09319385D95C0A8666D0B4BE1A3F5DFA1B2B2B2CC8BA7C18E91A80A4A3156B
>     635D8CC8735D628AA58BA8629AA8C680727506A9CA72D0460E5AAD0624964536BA0C452807A134C82C26E5C56088F89713F1CBBF5DFF9E78386EFF7CF55DEF177DF934B
>     77A4D3934DD7FE2DFCF46C01E7B0655211ED15D6F60EC502BB8C76A5197A8CCA4DF6ED7D92BF8D3D36B2B1C3CBFC243EA54FF7F5B2D0CC83B1A1F287DF941B04606E2A1
>     7E754926A07710B3CF6E86F4442D6A02463973F8736A408713BDB635F6CA4A73B0B979FD2D27521B01FE8327D7751A054BA3DE6CD1246ECC25020E678CB298AC86FA5A9
>     A5E1433C1AB8EC99FE3C9E33D06EE86D1581CDFB0E11EEFC1837BE3B7B3A0553114C8A79256D1AB30F99CA9A454A4C2ACA8125659CEF7EE84A9F0902988512AB7EF9442
>     250591743E3E2A72BEF91FA6218819E1AB32C90000000049454E44AE426082
>     <---
>     out~items: 8
>     # 1: length=[   4] 89504E | right(3)~c2x: 504E47
>     # 2: length=[   1] 1A     |
>     # 3: length=[ 300] 000000 | right(3)~c2x: 6DFF83
>     # 4: length=[  14] A6F9B6 | right(3)~c2x: E1A66B
>     # 5: length=[ 131] A48840 | right(3)~c2x: 9B2263
>     # 6: length=[ 103] FA8834 | right(3)~c2x: 38BD1E
>     # 7: length=[  90] 91E1F0 | right(3)~c2x: 85D95C
>     # 8: length=[ 325] 8666D0 | right(3)~c2x: 426082
>     test2.rex: sum=968
>
> It seems that in the redirection case CR and LF cause the data to be split (and eating up those control characters).
> ---rony