#754 'rexxc' binary (compiled/tokenized) data: please encode as plain text (7-Bit-ASCII) to allow it to be read as a string (e.g. from Java java.io.Reader)

[Rony G. Flatscher]

Currently a 'rexxc' compiled/tokenized Rexx program gets stored as binary data right after the eye-catcher string "/**/@REXX".

This RFE asks for encoding the binary data in a 7-bit string, e.g. with c2x() or String's encodeBase64() method. This way compiled/tokenized data can be read unchanged with e.g. a java.io.Reader (which expects plain text and applies codepage translations if necessary) and can be supplied e.g. from Java to native code as a java.lang.String object.

[Rony G. Flatscher]

After having analyzed various code-paths it turns out that really it is simply impossible for third-party library encoded rexxc images that constitute a x Rexx application consisting of many Rexx programs can be executed by the Rexx interpreter.

The root cause for this findings is that if a Rexx program that got decoded by a third-party library and run with a Rexx interpreter instance is directly or indirectly invoking another third-party library encoded rexxc'd program via a Rexx call keyword statement would cause a runtime error as ooRexx cannot process an encoding of the rexxc image which format is unknown to the the interpreter.

This problem could be alleviated if the Rexx interpreter could process a rexxc image that got encoded by a third party for any third party library. Maybe the encoding can be indicated by indicating the encoding (none, base16 or base64) the byte that immediately follows the rexxc-produced string right before the image:
/REXX/@
That byte is currently set to "00"x ("no encoding").

If that byte was set to "10"x (decimal 16, indicating a "base16 encoding") then the remaining data is base16 encoded, if it was set to "40"x (decimal 64, indicating a "base64 encoding") the remaining data is base64 encoded. This way the interpreter would be able to learn about a present encoding and if so, first decode the encoded image data appropriately before processing it.

[Rony G. Flatscher]

Just for exemplifying: attached please find an archive that contains a simple Rexx program ("test.rex"), its "rexxc.exe" version ("test_rexxc_binary.rex"), plus two versions that have the binary data encoded as base64, where the byte immediately following the string "/**/@REXX" got changed from "00"x to "64"x to indicate that the remaining bytes are base64 encoded.

Also, the quick sketch for a utility that would be able to add the info string as suggested in [feature-requests:#757] and encode the binary data as base64 is enclosed as well; it got used to create the encoded versions in the attached archive.

[Rick McGuire]

Initial code checked in [r11890]

[Rick McGuire]

Added MIME-like chunking format to the encoded data with [r11894]

[Rony G. Flatscher]

Doc commited with [r11994].