You can subscribe to this list here.
2019 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(4) |
Nov
(1) |
Dec
(2) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2020 |
Jan
|
Feb
|
Mar
(4) |
Apr
(3) |
May
(6) |
Jun
(7) |
Jul
(10) |
Aug
(9) |
Sep
(9) |
Oct
(1) |
Nov
(3) |
Dec
(5) |
2021 |
Jan
|
Feb
(1) |
Mar
(5) |
Apr
(1) |
May
|
Jun
|
Jul
|
Aug
(4) |
Sep
(3) |
Oct
(9) |
Nov
(1) |
Dec
(14) |
2022 |
Jan
(8) |
Feb
|
Mar
|
Apr
|
May
(1) |
Jun
(165) |
Jul
(8) |
Aug
|
Sep
|
Oct
(2) |
Nov
|
Dec
(5) |
2023 |
Jan
(19) |
Feb
(14) |
Mar
(2) |
Apr
(3) |
May
|
Jun
(2) |
Jul
(10) |
Aug
(6) |
Sep
|
Oct
|
Nov
|
Dec
|
2025 |
Jan
|
Feb
(10) |
Mar
(2) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: Jeff H. <Je...@Je...> - 2022-06-17 18:44:39
|
<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body> <p>My ERROR! I apologize.<br> </p> <p>In getting NetRexx Pipelines stages updated last year, I did not look at the working LOCATE stage, and did not inspect the IBM documentation to see it could be abbreviated to just L. I will supply the needed short class files (one is needed for each "word": L, LO, LOC, etc. to call the LOCATE stage) to do this. When I have time, I'll scan the IBM documentation to see if there are other such stages I have missed.</p> <p>Jeff Hennick<br> </p> <div class="moz-cite-prefix">On 6/14/2022 6:41 PM, <a class="moz-txt-link-abbreviated" href="mailto:hp...@we...">hp...@we...</a> wrote:<br> </div> <blockquote type="cite" cite="mid:753...@we...">Hi, <br> <br> at least I have done the first not-for-test-only NetRexx pipe. <br> A very simple one, read a file, discard comments, save first <br> record in a variable, save rest in a stem. <br> <br> Alas, some points displease: <br> i) the ooRexx integration per Address "" ... input ... output <br> force some changes, there is only one output connection possible, <br> but this pipe has two, a variable and a stem. Thus I had to solve <br> this in the post-processing. <br> ii) the input file consists of 10 records, 5 comment and 5 data, <br> what is close to nothing. Nonetheless it takes almost 1.7 seconds <br> to process, what is way too much for almost nothing. I hope there <br> is a chance to accelerate it notedly. <br> iii) labels require a blank after the colon, <br> iv) stage Locate can't be abbreviated to L, (yes, sure, this and <br> the point before is of minor importance, I know, but it adds to <br> the annoying differences to what I'm accustomed), <br> v) nonexistent input file ends processing of the ooRexx program <br> with an error. I assume this is due to the Address input/output <br> enclosure, Pipelines would just set the variable and the stem <br> accordingly. <br> <br> Here what it was (set as comment) and how I replaced it: <br> <br> <blockquote type="cite">-- call RxPipe '(end ?) file "' || vilif || '"!strip!l!nlocate 1 /*/!a:take!var conif', <br> -- '?a:!b:lookup 1 detail!stem livil.', <br> -- '?literal +-!fblock 1!b:' /* filter with lookup as locate missing option anyof */ <br> </blockquote> <br> <blockquote type="cite">f = .stream~new(vilif) <br> address "" ripe '"(sep ! end ?) cons', <br> '! strip!locate!nlocate 1 /*/', <br> '!a: drop!locate 1 any /+-/', <br> '!b: fanin!term', <br> '?a:!elastic!b:', <br> with input stream (f) output stem livil. <br> drop f <br> /* post-processing */ <br> conif = livil.[livil.0] <br> livil.0 -= 1 <br> </blockquote> <br> For a simpler post-processing I replaced take by drop. The changes <br> for a missing input file are not shown. <br> <br> All suggestions for speed-up are welcome. <br> <br> Best, <br> M. <br> <br> <br> _______________________________________________ <br> netrexx-pipelines mailing list <br> <a class="moz-txt-link-abbreviated" href="mailto:net...@li...">net...@li...</a> <br> <a class="moz-txt-link-freetext" href="https://lists.sourceforge.net/lists/listinfo/netrexx-pipelines">https://lists.sourceforge.net/lists/listinfo/netrexx-pipelines</a> <br> </blockquote> </body> </html> |
From: J L. T. <jlt...@ma...> - 2022-06-17 16:34:08
|
On 2022-06-16 04:38:46 René Jansen wrote: > Hi Mike, > > > On 16 Jun 2022, at 00:02, hp...@we... wrote: > > > > What indicates (for me), there is no error to report. > > > >> say out.1 > >> Error (specs) - pd673a59 - Missing data selector at field2 > > > > Ah -- big surprise. What signifies "pd673a59"? Roaring silence of > > NetRexx 4.03 Pipelines Guide and Reference regarding 'data selector’. > > That is an example of a generated pipe name. > > René. So if the source code for the pipeline included pipename in the options, that name would appear instead of pd673a59 in the message? Leslie -- Operating System: Linux Distribution: openSUSE Leap 15.4 x86_64 java version "18" 2022-03-22 NetRexx portable processor 4.03-GA build 260-20220503-1730 |
From: <hp...@we...> - 2022-06-17 11:00:16
|
Hello René! Totally overwhelmed by your huge and detailed reply I don't know where to start, as a simple thank you is just not enough. Should I bow down mumbling reverently 'Dear sir, most respected Esquire' or even apologize for stepping on your toes, because such an answer is also a little bit a defence, kind of "look we can, we just did not yet." Therefore I don't need to comment every paragraph, some words only where I feel it might complement your assertion. Am 16.06.2022 um 11:37 schrieb René Jansen: > to know what would be a good description of njp I would need the structure of the required documentation. That would maybe be a welcome addition to the document we have now. You may describe a subject just as it is, or by explaining the difference to what it was before. The later requires a target group which knows well enough what it once was. In case you decide to outline the subject in full, it will for sure be boring for the "flock of those knowing". So it is a significant decision, _who_ do you want/need/like to 'ramp up'. You need a structure of the required documentation? Just lean back (for the creative momentum) and try to see the forest not only many trees. Put yourself in the position of somebody without all your entirely knowledge -- where would you start? With a top-down-approach, an overview first, ok, chapter 1 Introduction, done. Next, more details, 2 - The Pipeline Concept, also done. Then, how to deal with it, a problem-oriented description of tasks and tools, all the chapters 3..18 what is IMO the 'guide' part, almost done, it lacks only the reference to the (currently missing) documentation. Chapters 19 and 20 are already part of the appendix with similar tables. So for my flavour between chapter 18 and 19 the reference could be inserted, documenting each and every stage, argument, option, keyword, error message and corresponding corrective action. Frankly, this is only my point of view, driven by the idea to look for what I want to know where I am accustomed to find it. Maybe there are simpler, cheaper, somehow better options to achieve the same result. > Let me try to make a start: > [...] IMHO, this is mainly history, important to understand what's up today, essential part of the know-why, to be filed in chapter 2. > To compile an .njp file to a class I guess, the following documentation of pipc's arguments and options, you did not write it down right now for this reply, it was available somewhere but did not make it to the manuals -- or I was not able to find it. > The compiler is invoked with the command: > > > pipc <arguments> > > Where <arguments> are: > > <compopt>|<fileid> {<compopt>|<fileid>} - compile the files listed > ( <name> { <compopt>} ) <pipe> - compile the <pipe> name The pipe might have options too, is the sequence of those and <compopt> irrelevant? > Where <compopt> is: > > -gen - keep the .nrx source of pipes following > -nogen - delete the .nrx source. This is the default Pipelines Guide and Reference, chapter 18 lists also option -keep. > -<option> - where <option> is a valid NetRexx option > > And <fileid> is: > > <name>.nrx - any NetRexx file > <name>.njp - any pipe or NetRexx or Java souce using pipes > <name>.grp - a file containing a list of <compopt>s and/or > <fileid>s. lines can be commented using '--' For the documentation of pipc this is sufficient, while the syntax of the files should be specified separately, in case of deviations also separately for each filetype. > The <options> are: > > runaway <milliseconds> - Interval to use for deadlock checking. This > or only detects true pipe deadlocks, not stages > stall <milliseconds> that are looping or hung. (Should this be > changed?) Ahem... question from the hobbyist: what has a stall to do with milliseconds? To my (uneducated) understanding, a stall is a situation, when the dispatcher can't find no stage to run while there is still data to flow. This is IMO a matter of strategy, not of milliseconds. In contrast to a stage hung, where I have to probe every now and then, if it consumes CPU without any I/O (or here pipe adequate, w/o data flow). > package <name> - insert a NetRexx package statement > import <name> - insert a NetRexx import statement > config <name> - override the default config file name (see below) > sep <char> - override the default stage separator '!' > end <char> - override the default end of pipe separator '?' The role model has no default end character. > cont <char> - override the default line continuation character ',' > debug <flag> - debugging options All documented in context of pipeline and listed here only for the record. > When the pipe is contained in a file the options must be set there. > > When a pipe is read from a file, text on the same line following a '-- ', > will be treated as a comment. > > Errors issued by the compiler. Ah -- at last ;) > Stages can issue additional messages. > > Error - name of pipe is missing ... and so on -- also most error messages are clear without further explanation, there might be some requiring few more words I assume, like > Error - problem reading group rc='r' for example. > Debugging pipes > > To find out what is happening in pipeline you have three tools. First, > you can set a debug flag when you compile the pipe. The bits you set > in the flag control what it does: > > 1 - Show all pipes starting > 2 - Show all pipes ending > 4 - Show all stages starting > 8 - Show all stages stopping > 16 - Show all Commit requests > 32 - Show all Commit completions > 64 - Show StageErrors raised via stage's Error(int,String) method. The > stage class uses Error for all its StageError signals. > 128 - Show the arguement that each stage is recieving. Handy since > shells have a habbit of doing unexpected thing to arguements. > (try: java findtext exit *.nrx vs java findtext "exit *.nrx") Wait a moment... single stages will receive arguments from the shell? On Windows from the cmd.exe? Where is this described? Piping as I know it _might_ set options for each stage separately (if necessary), typically arguments only, but they are all contained in the pipe specification part as argument to the pipe command (or compiler in this case), at least to my understanding. > - If there is a need I can easily add more events - just ask. Well, that depends on the dispatcher of NetRexx Pipelines, which states it knows in a stage's "lifetime", similar to those listed here http://vm.marist.edu/%7Epipeline/pipeline.pdf on p. 243, 'States of a Stage'. All possible transitions might be interesting for serious debugging. > To create a flag to see all stages starting and stopping you would > add 8+4 and use: > > pipc (apipe debug 12) ... > > The second option is to use the invoke the dump() method in a stage. > This dumps the status of the pipe using the same format you see when > a pipe deadlocks <file:///Users/rvjansen/Library/Mobile%20Documents/com~apple~CloudDocs/Documents/Programming_languages/Pipes/njpipes.webarchive#dead>. Using dump() does not normally cause a pipe to > terminate. Once in a while dump() will generate an exception. This > happens since dump() does not use protect or syncronize so it does not > stall. But that is only possible from within DIY stages, I assume. > When all else fails, you can recompile all the stages used by the pipe > with trace results enabled. So this was a good start, it should become a chapter in the reference part of Pipelines Guide and Reference. > There is a category of problems that is caused by the different ways of processing the pipe - most of them are very new, like using address in NetRexx. Some of them are older, because to enable calling from a commandline, a synthetic name will be generated. This is caused by the fact that you can call an unnamed pipe from the commandline in VM - if that were disallowed, we would not have this problem now. For the moment, I can only say that all .njp examples in the examples/pipes directory work as specified. Ah.. sorry, I have no clue about problems inventing names (I would convert the timestamp to allowed characters -- since it's for commandline entry an adequate coarse resolution would do). Example from VM/CMS: > 'callpipe literal!spec tod!spec 3-*!t62vld!var tft' with t62vld REXX translating to allowed characters like this: > * * * Top of File * * * > /* Translate FOLDed to valid characters of CMS file system */ > /**********************************************************************\ > | From VALIDATE HELPCMS: | > |The file name and file type can each be one to eight characters. The | > |valid characters are A-Z, a-z, 0-9, $, #, §, +, - (hyphen), : (colon),| > |and _ (underscore). | > \**********************************************************************/ > 'callpipe (end ?) *:!vchar 6 8!a:xlate!*:', > '?literal', /* transliteration table: */ > '81C1C2C3C4C5C6C7C8C9828384854E87' !!, /* aABCDEFGHIbcde+g */ > '88D1D2D3D4D5D6D7D8D9895B91929394' !!, /* hJKLMNOPQRi$jklm */ > '6095E2E3E4E5E6E7E8E99697986D99A2' !!, /* -nSTUVWXYZopq_rs */ > 'F0F1F2F3F4F5F6F7F8F97A7B7CA3A4A5' !!, /* 0123456789:#§tuv */ > '!spec 1-* x2c!a:' > * * * End of File * * * That is the way I once prepared unique filenames. Simple as that. > Stages have their own error messages - I must compile a pipe to list those all to the documentation. > Please let me know what more you need for documentation. Yes -- the messages recently: > Error (specs) - pd673a59 - Missing data selector at field2 and also > "Error - pipe as stage definition VILMA_1.CLASS is missing a period" For both issues I found no hint how to cure it. Well, I know, the documentation I ask for "c'est la galère, un boulot monstre" -- lots of work. Yesterday I met an emulator programmer (if you are interested in HP calculators you probably use either Emu48 or Emu42, ...), in his opinion it's not uncommon to program by trial and error, since sometimes the is no description at all or it's incomplete or wrong. But this approach seems to me like hunting for Easter eggs. When you say NetRexx works like its role model and I can't reproduce this experience, the effect is similar an empty promise. Best regards, M. |
From: René J. <rvj...@xs...> - 2022-06-16 09:39:00
|
Hi Mike, > On 16 Jun 2022, at 00:02, hp...@we... wrote: > > What indicates (for me), there is no error to report. > >> say out.1 >> Error (specs) - pd673a59 - Missing data selector at field2 > > Ah -- big surprise. What signifies "pd673a59"? Roaring silence of > NetRexx 4.03 Pipelines Guide and Reference regarding 'data selector’. That is an example of a generated pipe name. René. |
From: René J. <rvj...@xs...> - 2022-06-16 09:37:57
|
Hi Mike, to know what would be a good description of njp I would need the structure of the required documentation. That would maybe be a welcome addition to the document we have now. Let me try to make a start: NetRexx Pipelines are not interpreted; the implementation predates the NetRexx interpreter. Because the NetRexx translator, before version 2.00, only contained a compiler, the sourcefile that contains the pipe specification will be processed by a program called org.netrexx.njpipes.pipes.compiler. (Used to have another name but I do not have reason to include that info now- but that might pop up). We call this program the 'pipes compiler’. It rewrites your pipeline specification to a program that is composed of a number of threads and a synchronisation mechanism. These threads call the precompiled stages (from the org.netrexx.njpipes.stages package) and your own precompiled stages. The ‘command’ stage can start OS native commands and put their output into the pipe. This program source will now be passed on to the NetRexx compiler (we call the translator now a compiler, because we could also pass the source to the interpreter part of the translator, to be executed. These two, compilerr and translator, share most of the code, but are different things for different purposes). The compiler translates the generated pipeline program (NetRexx) to Java source code, and calls the Java compiler. The Java compiler is a compiler Java class - we call that, and not the platform executable, javac or javac.exe. We can also call an alternative Java compiler, the Eclipse Batch compiler, of which a modified version is delivered in the NetRexxC.jar archive (and not in the NetRexxC.jar archive, which you can use if you do not need the alternative compiler. We do this because a lot of companies provide systems to their workers that have a Java runtime and not a javac compiler.) The NetRexx translator was always made to support the human factors of programming and it was not designed to be a speed devil. Somewhere in the 2010’s - I think it was 2014 - the translator was modified (by Kermit Kiser) to keep all source and compiled Java classes in memory. Because of 64 bit Java and platform technology memory shortage ceased to be a problem. So now the NetRexx translator did not use files for its intermediate products anymore, the process became faster by avoiding disk I/O. Later developments as SSD disk diminished that speed gap, although the overall speed increased. At this point it became possible to offer a VM-like experience with typing the pipeline at a Ready prompt and having it executed immediately. This M1 mac I am typing this email on has a turnaround of 0.102 s for a short pipe in nrws. The NetRexx workspace came about when I tried to make a small REPL for this purpose, but while researching I found Martin Lafaix’ workspace from 20 years ago. This is very usable for development of pipelines where you up-arrow a lot and change the stage parameters (why count columns for a SPEC stage when you can just binary search until you find the right number.) Still, the original pipes compiler yields the best performance (around 0.05 s for execution of the compiled classfile) - because three compiles (and the writing of the .class file) are avoided.. To compile an .njp file to a class The compiler is invoked with the command: pipc <arguments> Where <arguments> are: <compopt>|<fileid> {<compopt>|<fileid>} - compile the files listed ( <name> { <compopt>} ) <pipe> - compile the <pipe> name Where <compopt> is: -gen - keep the .nrx source of pipes following -nogen - delete the .nrx source. This is the default -<option> - where <option> is a valid NetRexx option And <fileid> is: <name>.nrx - any NetRexx file <name>.njp - any pipe or NetRexx or Java souce using pipes <name>.grp - a file containing a list of <compopt>s and/or <fileid>s. lines can be commented using '--' The <options> are: runaway <milliseconds> - Interval to use for deadlock checking. This or only detects true pipe deadlocks, not stages stall <milliseconds> that are looping or hung. (Should this be changed?) package <name> - insert a NetRexx package statement import <name> - insert a NetRexx import statement config <name> - override the default config file name (see below) sep <char> - override the default stage separator '!' end <char> - override the default end of pipe separator '?' cont <char> - override the default line continuation character ',' debug <flag> - debugging options When the pipe is contained in a file the options must be set there. When a pipe is read from a file, text on the same line following a '-- ', will be treated as a comment. Errors issued by the compiler. Stages can issue additional messages. Error - name of pipe is missing Error - runaway time must be numeric Error - stall time must be numeric Error - debug level must be numeric, found 'lvl' Error - cache must be followed by a valid symbol, found 'work' Error - append or prefix stages cannot be labeled 'stg.word(1)' Error - missing stage/pipe after 'stg.word(1)' Error - missing range and/or stage/pipe after 'stg.word(1)' Error - specs has only one output stream, not requires two Error - pipe definition 'stg' must be terminated by 'pend' Error - pipe definition 'stg' must define a pipe Error - Label 'label' already used Error - Label 'label' must not be numeric Error - pipe 'stg' cannot have a label Error - connectors must be named Error - Label 'label' not defined Error - Use a nop stage between 'label' definition a second use Error - expected a colon after 'stg.word(1)' Error - pipe as stage definition 'stg' is missing a period Error - 'parms' incorrect. Use: {({class} {size})} {A|D} {target} Error - need to use a nop stage between 's[c[i,j-1]]' 'ssep' 's[c[i,j]]' Error - problem reading group rc='r' Error - 'w' is unreconized Warning - could not delete 'fileid' Error - 'key' is only valid in a class Warning - Possible netrexx exit in 'arg()' at 'l' Error - pipe name and parms must be on same line as 'key' Error - Pipe name missing for 'key' Error - Body of 'wp' is empty Error - *: connectors not implemented. Use *in: or *out: Error - Connector 'w1' should start with *in or *out Error - Missing colon at 'w1' Error - Connect 'w1' cannot contain a period Error - cannot connect 'in' to an input stream with 'key' Error - Pipe fragment 'sub' needs atleast one 'sep' Error - cannot connect 'out' to an output stream with 'key' Error - A object name cannot contain spaces, found: 'a' Debugging pipes To find out what is happening in pipeline you have three tools. First, you can set a debug flag when you compile the pipe. The bits you set in the flag control what it does: 1 - Show all pipes starting 2 - Show all pipes ending 4 - Show all stages starting 8 - Show all stages stopping 16 - Show all Commit requests 32 - Show all Commit completions 64 - Show StageErrors raised via stage's Error(int,String) method. The stage class uses Error for all its StageError signals. 128 - Show the arguement that each stage is recieving. Handy since shells have a habbit of doing unexpected thing to arguements. (try: java findtext exit *.nrx vs java findtext "exit *.nrx") - If there is a need I can easily add more events - just ask. To create a flag to see all stages starting and stopping you would add 8+4 and use: pipc (apipe debug 12) ... The second option is to use the invoke the dump() method in a stage. This dumps the status of the pipe using the same format you see when a pipe deadlocks <file:///Users/rvjansen/Library/Mobile%20Documents/com~apple~CloudDocs/Documents/Programming_languages/Pipes/njpipes.webarchive#dead>. Using dump() does not normally cause a pipe to terminate. Once in a while dump() will generate an exception. This happens since dump() does not use protect or syncronize so it does not stall. When all else fails, you can recompile all the stages used by the pipe with trace results enabled. There is a category of problems that is caused by the different ways of processing the pipe - most of them are very new, like using address in NetRexx. Some of them are older, because to enable calling from a commandline, a synthetic name will be generated. This is caused by the fact that you can call an unnamed pipe from the commandline in VM - if that were disallowed, we would not have this problem now. For the moment, I can only say that all .njp examples in the examples/pipes directory work as specified. Stages have their own error messages - I must compile a pipe to list those all to the documentation. Please let me know what more you need for documentation. best regards, René. > On 16 Jun 2022, at 08:18, hp...@we... wrote: > > Hi René! > > Am 15.06.2022 um 08:42 schrieb René Jansen: >> The way to make it faster would be to precompile the pipe to a java .class file using pipe and an .njp file, and instead of addressing pipe, to address java yourpipe.class. Of course, when called from a NetRexx program, you would have the java context already. > > I tried it, but found no documentation or at least a description > about coding njp. NetRexx 4.03 Pipelines Guide and Reference shows > some examples, but no detailed explanation. I prefer know-why, no > programming by trial and error. > > At the moment I'm stuck with: > >> "Error - pipe as stage definition VILMA_1.CLASS is missing a period" > > Sorry, if I am not able to solve this with the manual and have to > ask for help. > >> But to show the difference: >> [...] >> So compiling it speeds it up around 100 times, I expect the same thing to happen with your pipeline. > > I hope so, otherwise it's not worth the drudgery. > >> [...] >> >> There is one thing I am fixing at the moment, and that is an issue that Leslie Turriff reported first: with a portrait pipeline, the pipe separators must be at the end of the line for the pipe compiler (for some reason), and he (and I) like them at the start of the line. > > Same with me, up to now. But maybe I get used to something new ;) > > Best, > M. > > > _______________________________________________ > netrexx-pipelines mailing list > net...@li... > https://lists.sourceforge.net/lists/listinfo/netrexx-pipelines |
From: <hp...@we...> - 2022-06-16 06:57:07
|
Hi Rony! Am 15.06.2022 um 13:54 schrieb Rony G. Flatscher: > ... cut ... >> i) the ooRexx integration per Address "" ... input ... output >> force some changes, there is only one output connection possible, >> [...] > > No, there are two output connections possible! One for stdout > ("output") and one for stderr ("error"), cf. "rexxref.pdf", "2.1 > ADDRESS". LOL! You suggest to misuse stderr as ordinary output? Nice. And how may I make NetRexx Pipelines distinguish which one to use? Testing how to convert the following I also tried with two 'term' (as "opposite" console), which both sent output to stdout. This is why I opted for an _unambiguous_ output sequence, just to simplify the post-processing. > /* filter with lookup as OS/2 stage locate lacks option anyof */ > -- call RxPipe '(end ?) file "' || vilif || '"', > -- '! strip!l!nlocate 1 /*/!a:take!var conif', /** 1 **/ > -- '?a:!b:lookup 1 detail!stem livil.', /** 2 **/ > -- '?literal +-!fblock 1!b:' > if SysFileExists(vilif) then do > f = .stream~new(vilif) > address '' ripe '"(sep ! end ?) cons!strip!locate!nlocate 1 /*/', > '!a: drop!locate 1 any /+-/!b: fanin!term', > '?a: ! elastic !b:', > with input stream (f) output stem livil. > drop f > /* post-processing */ > conif = livil.[livil.0] > livil.0 -= 1 > ... Could stderr really be some serious help in this case? Another brain-twisting example of double-use stdin (original pipe as comment first): > -- call RxPipe '(end ?) var nck!a:lookup f1 master!spec f2 1!var doubl', > -- '?stem selid.!a:' > in. = selid.; in.0 += 1; in.[in.0] = nck /* prepare input */ > address '' ripe '"(sep ! end ?) cons!x: take last!a:lookup f1 master!spec f2 1!term', > '?x:!a:', > with input stem in. output stem out. error stem fail. Result (in trace mode): > 1145 *-* address '' ripe '"(sep ! end ?) cons!x: take last!a:lookup f1 master!spec f2 1!term', '?x:!a:', with input stem in. output stem out. error stem fail. > >>> "@java -cp "D:\PRGM\NetRexx\lib\NetRexxF.jar;.;D:\PRGM\NetRexx\lib\NetRexxC.jar;" org.netrexx.njpipes.pipes.runner "(sep ! end ?) cons!x: take last!a:lookup f1 master!spec f2 1!term ?x:!a:" > >K> "INPUT" => "0" > >K> "OUTPUT" => "OUT." > >K> "ERROR" => "FAIL." > say out.0 > 1 What could be the single line of output as expected. > say fail.0 > 0 What indicates (for me), there is no error to report. > say out.1 > Error (specs) - pd673a59 - Missing data selector at field2 Ah -- big surprise. What signifies "pd673a59"? Roaring silence of NetRexx 4.03 Pipelines Guide and Reference regarding 'data selector'. Then, rather an academical question, how may "address '' ... with input/output/error" handle all streams of lookup? Best, M. |
From: <hp...@we...> - 2022-06-16 06:57:03
|
Hi René! Am 15.06.2022 um 08:42 schrieb René Jansen: > The way to make it faster would be to precompile the pipe to a java .class file using pipe and an .njp file, and instead of addressing pipe, to address java yourpipe.class. Of course, when called from a NetRexx program, you would have the java context already. I tried it, but found no documentation or at least a description about coding njp. NetRexx 4.03 Pipelines Guide and Reference shows some examples, but no detailed explanation. I prefer know-why, no programming by trial and error. At the moment I'm stuck with: > "Error - pipe as stage definition VILMA_1.CLASS is missing a period" Sorry, if I am not able to solve this with the manual and have to ask for help. > But to show the difference: > [...] > So compiling it speeds it up around 100 times, I expect the same thing to happen with your pipeline. I hope so, otherwise it's not worth the drudgery. > [...] > > There is one thing I am fixing at the moment, and that is an issue that Leslie Turriff reported first: with a portrait pipeline, the pipe separators must be at the end of the line for the pipe compiler (for some reason), and he (and I) like them at the start of the line. Same with me, up to now. But maybe I get used to something new ;) Best, M. |
From: René J. <rvj...@xs...> - 2022-06-15 13:59:45
|
Now it can, in the upcoming 4.04. > On 15 Jun 2022, at 00:41, hp...@we... wrote: > > iv) stage Locate can't be abbreviated to L, (yes, sure, this and > the point before is of minor importance, I know, but it adds to > the annoying differences to what I'm accustomed), |
From: Rony G. F. <Ron...@wu...> - 2022-06-15 11:54:13
|
Hi Mike, On 15.06.2022 00:41, hp...@we... wrote: ... cut ... > Alas, some points displease: > i) the ooRexx integration per Address "" ... input ... output > force some changes, there is only one output connection possible, > but this pipe has two, a variable and a stem. Thus I had to solve > this in the post-processing. No, there are two output connections possible! One for stdout ("output") and one for stderr ("error"), cf. "rexxref.pdf", "2.1 ADDRESS". ---rony |
From: René J. <rvj...@xs...> - 2022-06-15 06:42:55
|
[and one for the list] Hi Mike, The way to make it faster would be to precompile the pipe to a java .class file using pipe and an .njp file, and instead of addressing pipe, to address java yourpipe.class. Of course, when called from a NetRexx program, you would have the java context already. But to show the difference: ➜ ~ time pipe "literal aap noot mies | split | sort | cons" aap mies noot java org.netrexx.njpipes.pipes.runner 1.76s user 0.08s system 325% cpu 0.567 total So that is 1.7 sec overall, but 0.5 sec net due to multiple cpu’s in this machine (that is the 325%) But when we compile this, ➜ ~ emacs cpipe.njp ➜ ~ cat cpipe.njp pipe (cpipe) literal aap noot mies | split | sort | cons ➜ ~ pipc cpipe pipe (cpipe ) literal aap noot mies | split | sort | cons ➜ ~ time java cpipe aap mies noot java cpipe 0.05s user 0.02s system 106% cpu 0.064 total So compiling it speeds it up around 100 times, I expect the same thing to happen with your pipeline. What I do, is using the pipe and nrws commands for development of the pipeline (up-arrowing and editing until it is working like I want it) and then compiling when it is “ready for production”. There is one thing I am fixing at the moment, and that is an issue that Leslie Turriff reported first: with a portrait pipeline, the pipe separators must be at the end of the line for the pipe compiler (for some reason), and he (and I) like them at the start of the line. This will be hopefully fixed in NetRexx 4.04 when it is GA. Until then, the line needs breaking at unfortunate places. Best regards, René. > On 15 Jun 2022, at 00:41, hp...@we... wrote: > > Hi, > > at least I have done the first not-for-test-only NetRexx pipe. > A very simple one, read a file, discard comments, save first > record in a variable, save rest in a stem. > > Alas, some points displease: > i) the ooRexx integration per Address "" ... input ... output > force some changes, there is only one output connection possible, > but this pipe has two, a variable and a stem. Thus I had to solve > this in the post-processing. > ii) the input file consists of 10 records, 5 comment and 5 data, > what is close to nothing. Nonetheless it takes almost 1.7 seconds > to process, what is way too much for almost nothing. I hope there > is a chance to accelerate it notedly. > iii) labels require a blank after the colon, > iv) stage Locate can't be abbreviated to L, (yes, sure, this and > the point before is of minor importance, I know, but it adds to > the annoying differences to what I'm accustomed), > v) nonexistent input file ends processing of the ooRexx program > with an error. I assume this is due to the Address input/output > enclosure, Pipelines would just set the variable and the stem > accordingly. > > Here what it was (set as comment) and how I replaced it: > >> -- call RxPipe '(end ?) file "' || vilif || '"!strip!l!nlocate 1 /*/!a:take!var conif', >> -- '?a:!b:lookup 1 detail!stem livil.', >> -- '?literal +-!fblock 1!b:' /* filter with lookup as locate missing option anyof */ > >> f = .stream~new(vilif) >> address "" ripe '"(sep ! end ?) cons', >> '! strip!locate!nlocate 1 /*/', >> '!a: drop!locate 1 any /+-/', >> '!b: fanin!term', >> '?a:!elastic!b:', >> with input stream (f) output stem livil. >> drop f >> /* post-processing */ >> conif = livil.[livil.0] >> livil.0 -= 1 > > For a simpler post-processing I replaced take by drop. The changes > for a missing input file are not shown. > > All suggestions for speed-up are welcome. > > Best, > M. > > > _______________________________________________ > netrexx-pipelines mailing list > net...@li... > https://lists.sourceforge.net/lists/listinfo/netrexx-pipelines |
From: <hp...@we...> - 2022-06-14 22:43:52
|
Hi, at least I have done the first not-for-test-only NetRexx pipe. A very simple one, read a file, discard comments, save first record in a variable, save rest in a stem. Alas, some points displease: i) the ooRexx integration per Address "" ... input ... output force some changes, there is only one output connection possible, but this pipe has two, a variable and a stem. Thus I had to solve this in the post-processing. ii) the input file consists of 10 records, 5 comment and 5 data, what is close to nothing. Nonetheless it takes almost 1.7 seconds to process, what is way too much for almost nothing. I hope there is a chance to accelerate it notedly. iii) labels require a blank after the colon, iv) stage Locate can't be abbreviated to L, (yes, sure, this and the point before is of minor importance, I know, but it adds to the annoying differences to what I'm accustomed), v) nonexistent input file ends processing of the ooRexx program with an error. I assume this is due to the Address input/output enclosure, Pipelines would just set the variable and the stem accordingly. Here what it was (set as comment) and how I replaced it: > -- call RxPipe '(end ?) file "' || vilif || '"!strip!l!nlocate 1 /*/!a:take!var conif', > -- '?a:!b:lookup 1 detail!stem livil.', > -- '?literal +-!fblock 1!b:' /* filter with lookup as locate missing option anyof */ > f = .stream~new(vilif) > address "" ripe '"(sep ! end ?) cons', > '! strip!locate!nlocate 1 /*/', > '!a: drop!locate 1 any /+-/', > '!b: fanin!term', > '?a:!elastic!b:', > with input stream (f) output stem livil. > drop f > /* post-processing */ > conif = livil.[livil.0] > livil.0 -= 1 For a simpler post-processing I replaced take by drop. The changes for a missing input file are not shown. All suggestions for speed-up are welcome. Best, M. |
From: <hp...@we...> - 2022-06-14 14:17:18
|
Hi René! Thank you for your prompt and detailed reply. Am 14.06.2022 um 13:28 schrieb René Jansen: > do OS/2 Pipelines an sich still work? Yes, from a cmd window I may do > D:\PRGM\...>pipe diskr vilma.ini!count lines!term > 32 But, triggered by your question I tried also: > D:\PRGM\...>pipe "(end ?) diskr vilma.ini!strip!a:l?a:!count lines!term" > PIP0562E: Unable to install REXX compiler runtime library (module REXX.DLL, return code 193). I never "tested" such things directly from the cmd window, typically from ooRexx. The puzzling fact, there it worked until about 15 hours ago. > As I understand for a long time OS/2 16 bit code was still supported under NT and successors - the 16bit code being OS/2 1.X and supported by MS. If this went away, it becomes harder - not impossible due to QEMU/UTM, Virtual Box and the likes. If you can still run 16 bit OS/2 executables with Windows 10, this is a PATH/DLL/loading problem. ooRexx uses address which can execute a shell in addition to a lot of other platform programs. If things stop with ooRexx 5.0.0 it can be fixed - dlopen() etc still work, maybe some linkage conventions are changed. Well, "our" first goal is not to bring back OS/2 Pipelines to work, "we" (in fact: I) haunt for NetRexx support for ooRexx. Thus, if it does not work any longer it's not a great loss. In contrast to Rexx/gd because I don't know what could replace it. > Mark should be able to help with Rexx/gd. This might be the RexxSAA.h thing that happened a few years ago. I cc’ed him because I do not think he is on this list. But it should be working. Thank you. > These are of course things that complicated matters; easiest would be just to do NetRexx with its pipes and a Java Swing or JavaFX interface. The Java Properties class is made for picking up parameters, and it is easiest to use those in a Java (NetRexx) program instead of marshalling them to ooRexx. To switch from ooRexx to NetRexx would need to redo all the shiny surface I did with ooDialog. > But I like complicated matters so just go agaed, and I am sorry I don’t have enough time as I would like to invest in this. There is also some code to write for the day job, and also NetRexx and CREXX are time intensive. It's me who has to beg your pardon, since my "problems" are hobby only, nothing serious. I may only try to compensate your kind support with a "nothing but REXX" simulation of HP-11C, -12C, 15C, -16C, and 41C/CV/CX -- see http://hp41.org/html/ooNUlb.htm > About OS/2 Pipelines - who did that, do we have source? No, I have no sources of it. In pipe.htm I find: > The OS/2 version is available as OS2PIPE package on the OS2TOOLS repository. > The (beta) Windows or AIX versions are requestable from the author. On a VM/CMS command line enter: > > REQUEST WINPIPE FROM FDB AT NLVM > > or: > > REQUEST AIXPIPE FROM FDB AT NLVM > > The initial versions of the OS/2 Pipelines program and this documentation (up to version 0.53) were written by Mark VanTassel during 1993 and 1994. Since Mark has left IBM (ISSC) in September 1994, ownership of the OS2PIPE package has been transferred to Frans de Bruijn (FDB at IBMNL or fd...@nl... ). I have used it only once for a "quick-n-dirty" program (with bias on quick), I published only its description but not the program for 2..3 reasons: i) OS/2 Pipelines is tagged 'internal use only', ii) IMO it's work in progress and thus very buggy, only for those who know the role model and are able to check it works for a specific use, iii) somebody told me about his idea and I showed him how fast it can be done with REXX (pst.. and pipelines). > Is/was it IBM internal, EWS? Yes. > Would someone invest the time in open sourcing it? From my point of view, no. Over time I saw several efforts to copy John's role model and I hoped they would succeed, just to keep going the same way using the PC I was used to on the mainframe. But! frankly, to achieve the maturity of CMS/TSO Pipelines is a huge task you can't finish between 5 and feierabend (closing time). Thus there must be a sponsor to convince about the advantages, who will pay for an able team, and in the end he likes to make some profit of it, therefore it will not be free. That is why it's highly unlikely something like Pipelines/PC will come true. Methinks. > The ooRexx pipe stuff is interesting on the concept level - [...] That's quite a very diplomatic way to tell the truth... ;) > Your perspective of having to convert *all* the pipelines is actually more fun to me, as it gives the opportunity to fix a lot of things in Rexx and Pipelines. Only it will take a lot longer. Honestly, the pace of the last few days was to fast for me. I've got so many hints and help and support from this list, I was not able to test all options and give an adequate reply immediately. So please don't take it as impolite if I slow down a bit. I have no hurry, I'm not at work. Best, M. |
From: J L. T. <jlt...@ma...> - 2022-06-14 11:37:52
|
OS/2 is still alive under a newer maintainer, eComStation <https://www.ecomstation.com/> Leslie On 2022-06-14 06:28:06 René Jansen wrote: > Hi Mike, > > do OS/2 Pipelines an sich still work? As I understand for a long time OS/2 > 16 bit code was still supported under NT and successors - the 16bit code > being OS/2 1.X and supported by MS. If this went away, it becomes harder - > not impossible due to QEMU/UTM, Virtual Box and the likes. If you can still > run 16 bit OS/2 executables with Windows 10, this is a PATH/DLL/loading > problem. ooRexx uses address which can execute a shell in addition to a lot > of other platform programs. If things stop with ooRexx 5.0.0 it can be > fixed - dlopen() etc still work, maybe some linkage conventions are > changed. > best regards, > > René. -- Operating System: Linux Distribution: openSUSE Leap 15.4 x86_64 java version "18" 2022-03-22 NetRexx portable processor 4.03-GA build 260-20220503-1730 |
From: René J. <rvj...@xs...> - 2022-06-14 11:28:30
|
Hi Mike, do OS/2 Pipelines an sich still work? As I understand for a long time OS/2 16 bit code was still supported under NT and successors - the 16bit code being OS/2 1.X and supported by MS. If this went away, it becomes harder - not impossible due to QEMU/UTM, Virtual Box and the likes. If you can still run 16 bit OS/2 executables with Windows 10, this is a PATH/DLL/loading problem. ooRexx uses address which can execute a shell in addition to a lot of other platform programs. If things stop with ooRexx 5.0.0 it can be fixed - dlopen() etc still work, maybe some linkage conventions are changed. Mark should be able to help with Rexx/gd. This might be the RexxSAA.h thing that happened a few years ago. I cc’ed him because I do not think he is on this list. But it should be working. These are of course things that complicated matters; easiest would be just to do NetRexx with its pipes and a Java Swing or JavaFX interface. The Java Properties class is made for picking up parameters, and it is easiest to use those in a Java (NetRexx) program instead of marshalling them to ooRexx. But I like complicated matters so just go agaed, and I am sorry I don’t have enough time as I would like to invest in this. There is also some code to write for the day job, and also NetRexx and CREXX are time intensive. Maybe we can ask Jeff to have a look at SPECS and its missing parts. At the moment I am trying to harmonize all the ways you can execute a pipe. About OS/2 Pipelines - who did that, do we have source? Is/was it IBM internal, EWS? Would someone invest the time in open sourcing it? The ooRexx pipe stuff is interesting on the concept level - it is nowhere near the NetRexx Pipelines functionality compared to CMS/TSO Pipelines, and that was a lot of work, even if you take into account what Ed and Jeff already had 20 years ago. Your perspective of having to convert *all* the pipelines is actually more fun to me, as it gives the opportunity to fix a lot of things in Rexx and Pipelines. Only it will take a lot longer. best regards, René. > On 14 Jun 2022, at 13:01, hp...@we... wrote: > > Hello René! > > The plan to migrate OS/2 to NetRexx Pipelines one by one while due > to access to both the program still works as a whole -- this plan > is thwarted since at a sudden OS/2 pipelines is not accessible any > more from ooRexx 5.oo under Windows 10. I have no clue why, even a > power cycle did not cure it. > >> call RxFuncAdd 'RxPipe', 'RXPIPE', 'RXPIPE' > > just fails. But also loading the Rexx/gd library: > >> Call RxFuncAdd 'GdLoadFuncs', 'rexxgd', 'GdLoadFuncs' > > in another program fails. While > >> rc = RxFuncAdd("SockLoadFuncs","rxsock","SockLoadFuncs") > > still works. > > So I _do have_ to convert all OS/2 pipelines and may test the > program in full only when completed the last one. Same task but > less fun. (In contrast, inaccessible Rexx/gd library is a > substantial loss.) > > Best, > M. > > > Am 07.06.2022 um 21:59 schrieb René Jansen: >>>>> [...] > > > _______________________________________________ > netrexx-pipelines mailing list > net...@li... > https://lists.sourceforge.net/lists/listinfo/netrexx-pipelines |
From: <hp...@we...> - 2022-06-14 11:02:56
|
Hello René! The plan to migrate OS/2 to NetRexx Pipelines one by one while due to access to both the program still works as a whole -- this plan is thwarted since at a sudden OS/2 pipelines is not accessible any more from ooRexx 5.oo under Windows 10. I have no clue why, even a power cycle did not cure it. > call RxFuncAdd 'RxPipe', 'RXPIPE', 'RXPIPE' just fails. But also loading the Rexx/gd library: > Call RxFuncAdd 'GdLoadFuncs', 'rexxgd', 'GdLoadFuncs' in another program fails. While > rc = RxFuncAdd("SockLoadFuncs","rxsock","SockLoadFuncs") still works. So I _do have_ to convert all OS/2 pipelines and may test the program in full only when completed the last one. Same task but less fun. (In contrast, inaccessible Rexx/gd library is a substantial loss.) Best, M. Am 07.06.2022 um 21:59 schrieb René Jansen: >>>> [...] |
From: <hp...@we...> - 2022-06-14 11:02:52
|
Hello Rony! Am 13.06.2022 um 13:29 schrieb Rony G. Flatscher: > ooRexx comes with a pipe.cls that defines the following stages: > > ::class pipeStage public -- base > pipeStage class > ::class SecondaryConnector subclass pipeStage > [...] Nice! It is also part of ooRexx 4.2.0 (Feb 22 2014), but recently somebody (bigrixx) added some more stages. > To see how one can use the ooRexx pipe.cls look up > "ooRexx\samples\usepipe.cls". Probably a typo: ...\usepipe.rex > If you look up pipe.cls you should > also see the programming pattern of the stages such that you might > be able to add new stages of your own. Yes -- if all else fails ;) > It may be the case that your needs can be met with the means > ooRexx has on board already. Honestly, what I "do need" (I'm hobbyist) can be done with ooRexx. My zest/request/demand for pipelines is because I am used to it. Formerly, at work, Pipelines _multiplied_ the value of the mainframe by 2..3 for me concerning my "challenge". Many tasks are solved much simpler by "pipe-thinking" it, less code is also less habitat for bugs. But -- frankly -- what is the big advantage, what makes the difference? For me it's foremost the vast number of possible sources and sinks for data flow, married with a practice-oriented set of stages that covers almost unlimited possibilities. This said (many words with just the intention not to debase any effort to copy the role model) I hope you fathom that for me "the Book" -- http://vm.marist.edu/%7Epipeline/pipeline.pdf -- is the point of reference. And assumedly you also know the warning about "those with one book only" ;) Best, M. |
From: J L. T. <jlt...@ma...> - 2022-06-13 11:45:52
|
I too was disappointed that specs field- and word-ranges aren't yet supported in NetRexx Pipelines; I always found it so very useful for rearranging fields of records, and there's no equivalent to that in the Unixish world. Leslie On 2022-06-13 05:04:49 hp...@we... wrote: > Am 13.06.2022 um 02:27 schrieb ((ich)): > > [...] > > Hmm... no, that looks quite similar like the first case, so this > > one too I will not convert to NetRexx Pipelines. But the third > > > > pipe is worth it: > >> call RxPipe '< "' || fn || '"!l fs ; f1!stem loc.!spec fs 09 f1 > >> 1!stem n.' > > No-no-no-no: > > Error (specs) - fs 09 f1 1 write - Unrecognized option: fs > > A quick glance into the documentation reveals: > > (3) CMS only. Not yet implemented in NetRexx Pipelines > > Therefore this OS/2 pipe is converted to plain ooRexx without > > support from NetRexx Pipelines. Same for next one: > > if syswinver() = 'WindowsNT 6.01' then, /* Win7 only */ > > gehon = '< C:\Windows\System32\drivers\etc\hosts', /* mappings of > > dotIP to host names */ '!spec fs # f1!strip!l!spec w2!' > > /* keep name only */ else gehon = '' > > call RxPipe gehon 'literal localhost!stem honam.' /* no sort as > > ComboBox will do so */ > > Again spec fs char is the show-stopper. Next one? > > > selni = ctlcbx~selected > > call RxPipe '(end ?) var selni!a:lookup f1 master!spec f2', > > '!spec fs ; f1!strip!var fid!reverse!spec fs . f1!reverse!xlate!var > > sfx', '?stem loc.!a:' > > Forget it, no way! Two times spec fs ... Next one is: > > call RxPipe '(end ?) var fid!strip /"/!get asis BIN', > > '!a:take 16 bytes!spec *-* c2x!var sig', > > '?a:!b:take 4 bytes!reverse!spec *-* c2x!var kml', > > '?b:!stem rst.' > > Finally, the perfect candidate for NetRexx Pipelines, read a file > binary, sip off two strings from the beginning, convert and > deliver them to ooRexx, also the untouched remnant of the file for > further investigation. New thereby, maybe a challenge, are the two > strings and an unknown number of records returned. > > But first I'll "rexxify" all skipped before I attack this one. > > /M. > > > _______________________________________________ > netrexx-pipelines mailing list > net...@li... > https://lists.sourceforge.net/lists/listinfo/netrexx-pipelines -- |
From: <hp...@we...> - 2022-06-13 11:36:52
|
Am 13.06.2022 um 02:27 schrieb ((ich)): > [...] > Hmm... no, that looks quite similar like the first case, so this > one too I will not convert to NetRexx Pipelines. But the third > pipe is worth it: >> call RxPipe '< "' || fn || '"!l fs ; f1!stem loc.!spec fs 09 f1 >> 1!stem n.' No-no-no-no: > Error (specs) - fs 09 f1 1 write - Unrecognized option: fs A quick glance into the documentation reveals: > (3) CMS only. Not yet implemented in NetRexx Pipelines Therefore this OS/2 pipe is converted to plain ooRexx without support from NetRexx Pipelines. Same for next one: > if syswinver() = 'WindowsNT 6.01' then, /* Win7 only */ > gehon = '< C:\Windows\System32\drivers\etc\hosts', /* mappings of dotIP to host names */ > '!spec fs # f1!strip!l!spec w2!' /* keep name only */ > else gehon = '' > call RxPipe gehon 'literal localhost!stem honam.' /* no sort as ComboBox will do so */ Again spec fs char is the show-stopper. Next one? > selni = ctlcbx~selected > call RxPipe '(end ?) var selni!a:lookup f1 master!spec f2', > '!spec fs ; f1!strip!var fid!reverse!spec fs . f1!reverse!xlate!var sfx', > '?stem loc.!a:' Forget it, no way! Two times spec fs ... Next one is: > call RxPipe '(end ?) var fid!strip /"/!get asis BIN', > '!a:take 16 bytes!spec *-* c2x!var sig', > '?a:!b:take 4 bytes!reverse!spec *-* c2x!var kml', > '?b:!stem rst.' Finally, the perfect candidate for NetRexx Pipelines, read a file binary, sip off two strings from the beginning, convert and deliver them to ooRexx, also the untouched remnant of the file for further investigation. New thereby, maybe a challenge, are the two strings and an unknown number of records returned. But first I'll "rexxify" all skipped before I attack this one. /M. |
From: Rony G. F. <Ron...@wu...> - 2022-06-13 11:29:35
|
Hi Mike, On 6/13/2022 2:44 AM, hp...@we... wrote: > Am 12.06.2022 um 12:47 schrieb Rony G. Flatscher: >> sorry a glitch in the demo code that would not check for an >> asterisk in the first column, here the corrected (hypothetical and >> untested code sample: >> [...] > > Thank you for the correction. It shows, also you gave up > "pipe-thinking" in this first case because you prefer to filter in > the post-process than rather (and much simpler) in the pipeline ;) not necessarily. :) ooRexx comes with a pipe.cls that defines the following stages: ::class pipeStage public -- base pipeStage class ::class SecondaryConnector subclass pipeStage ::class after subclass pipeStage public -- write only records from first trigger record ::class all public subclass pipeStage -- a string selector pipeStage ::class arraycollector subclass pipeStage public -- collect items in an array ::class before subclass pipeStage public -- write only records before first trigger record ::class between subclass pipeStage public -- write only records from first trigger record ::class bitbucket public subclass pipeStage -- just consume the records ::class buffer subclass pipeStage public -- write only records before first trigger record ::class changestr public subclass pipeStage -- a string replacement pipeStage ::class charCount subclass pipeStage public -- count number of characters passed through the pipeStage ::class delstr public subclass pipeStage -- a string deletion pipeStage ::class displayer subclass pipeStage public ::class dropFirst public subclass pipeStage -- drop the first n records ::class dropLast public subclass pipeStage -- drop the last n records ::class dropnull public subclass pipeStage -- drop null records ::class duplicate public subclass pipeStage -- duplicate each record N times ::class fanin public subclass pipeStage -- process main stream, then secondary stream ::class fanout public subclass pipeStage -- write records to both output streams ::class insert public subclass pipeStage -- insert a string into each line ::class left public subclass pipeStage -- a splitter pipeStage ::class lineCount subclass pipeStage public -- count number of records passed through the pipeStage ::class lower public subclass pipeStage -- a lowercasing pipeStage ::class merge public subclass pipeStage -- merge the results from primary and secondary streams ::class notall public subclass pipeStage -- a string de-selector pipeStage ::class overlay public subclass pipeStage -- insert a string into each line ::class pivot subclass pipeStage public ::class reverse public subclass pipeStage -- a string reversal pipeStage ::class right public subclass pipeStage -- a splitter pipeStage ::class sort public subclass pipeStage -- sort piped data ::class sortWith public subclass pipeStage -- sort piped data ::class splitter subclass pipeStage public ::class startsWith public subclass pipeStage -- a string selector pipeStage ::class stemcollector subclass pipeStage public -- collect items in a stem ::class takeFirst public subclass pipeStage -- take the first n records ::class takeLast public subclass pipeStage -- take the last n records ::class upper public subclass pipeStage -- a uppercasing pipeStage ::class wordCount subclass pipeStage public -- count number of characters passed through the pipeStage ::class x2c public subclass pipeStage -- translate records to hex characters To see how one can use the ooRexx pipe.cls look up "ooRexx\samples\usepipe.cls". If you look up pipe.cls you should also see the programming pattern of the stages such that you might be able to add new stages of your own. It may be the case that your needs can be met with the means ooRexx has on board already. HTH, ---rony |
From: <hp...@we...> - 2022-06-13 01:12:25
|
Hi! Am 12.06.2022 um 17:55 schrieb Rony G. Flatscher: > Learned from René that it is possible to use external programs, > also Rexx programs in NetRexx pipes like: > > pipe literal curl http://example.com | command | cons > > So you can write oo/Rexx filter programs to be used as NetRexx > pipe stages. That is a very handy hint, thank you René and Rony for forwarding it. > The oo/Rexx filter (reading from stdin, writing to stdout) would > have to be written like: > > do until line="" > parse pull line /* read from stdin */ > ... do something with line and if you want to pass on data > do ... > say editedData /* writes to stdout */ > end > > alternatively, one could write the Rexx filter also as (preferable): > > signal on notready /* raised if stdin gets closed */ > do forever > parse linein line > ... do something with line and if you want to pass on data > do ... > say editedData /* writes to stdout */ > end > notready: /* label to jump to */ That looks promising :) > If you also want to write to stderr then you could either do that > in classic Rexx style with "call lineout stderr,data" or in ooRexx > ".error~say(data)". > > You may want to look through these slides: > <https://wi.wu.ac.at/rgf/wu/lehre/slides/BusinessProgramming/060_ooRexx_commands_V14.pdf>. Rony! I took the chance and browsed also through the other sets of slides available there -- and besides 2..3 things about REXX I learned two facts about i) Prof. Rony G. Flatscher that ii) he was part of the team negotiating with IBM about ooRexx. So I get help from the top-notch expert himself. Hope your students are less demanding than I might be as mainframe dinosaur. Best, M. |
From: <hp...@we...> - 2022-06-13 01:12:25
|
Hello Rony, Am 12.06.2022 um 12:47 schrieb Rony G. Flatscher: > sorry a glitch in the demo code that would not check for an > asterisk in the first column, here the corrected (hypothetical and > untested code sample: > [...] Thank you for the correction. It shows, also you gave up "pipe-thinking" in this first case because you prefer to filter in the post-process than rather (and much simpler) in the pipeline ;) Best, M. |
From: <hp...@we...> - 2022-06-13 01:12:25
|
Hello Rony! Am 12.06.2022 um 12:39 schrieb Rony G. Flatscher: > many questions... :) Sorry, but because ooRexx 5.oo is still Beta I installed it on a spare PC without mail system, so the questions piled up. > Firstly one thing you should know: ooRexx is backwardly compatible > with Rexx, this way one can develop and run classic Rexx programs > including the Rexx scoping rules (which variables and labels can > be seen from which part of a Rexx program). That's for me the most important characteristic of ooRexx -- and NetRexx from 4.04 on will also offer 'old-school' REXX, René said. > Ad ooRexx scopes (from my slides): > [...] Thank you very much for this complete and concise overview, my description was limited to variables only. Maybe I should review all my published REXX programs with your "scopes cheat sheet" besides the PC. > --- > [...] > ooRexx directives are not available to NetRexx and will cause a > syntax error. Thus I can't use BsfContextVariables() in a stage of filetype NJP which is to be translated by PIPC. Consequently I need another way to put (or push) records from the pipe to something (var, array, ...) within ooRexx. > --- > [...] > You can include BSF.CLS (which loads all external BSF4ooRexx > functions and establishes the camouflaging support such that > strictly typed Java classes and Java objects can be interacted > with as if they were dynamically typed ooRexx classes and ooRexx > objects) in two ways: > [...] Ok, I would go with the second possibility you mentioned, > ::requires BSF.CLS but since this is impossible within the "DIY" stages, I may only set it in the ooRexx program which starts a NetRexx pipeline. > BsfContextVariables() allows one to get all currently defined > variables as a directory (index is name of the variable, item the Wait a moment, I need Java (BSF.CLS is concealed Java, isn't it) to get the variable pool of a running ooRexx? -- Oops, gotcha! Honestly, you got me: exactly as on VM/CMS I use on the PC too the stage REXXVARS which is part of OS/2 Pipelines too. > Pipe rexxvars!> prgm.ini and I have a file with current variable pool saved for a future session to be used by Pipe < prgm.ini!varload -- simple as that. > current value) for which in the meantime alternatives exist > (.context~package~variables). That's ooRexx 5.oo exclusively, I assume. > BsfContextVariables() allows you to set multiple Rexx variable in > the scope where you invoke it, if you supplied a directory with > all variables that should get defined. For this usage I would > suggest BsfContextVariables(). With pleasure, if NetRexx could deliver that directory in question. But! - sorry for still not being convinced -- if I have a directory in ooRexx, aren't there other, actually better/faster/leaner processes to set the variables in accordance with the directory? > --- > Ad fetching data from external sources. ooRexx 5 gained the > ability to use the ADDRESS keyword instruction to redirect stdin, > stdout and stderr from/to ooRexx if desired. To do so you define > what collection you want to use from ooRexx, e.g. a stem or an array. That is only possible within ooRexx 5.oo, which will remain for some reasons in Beta further on. > Not being familar with Netrexx pipes here a hypothetical pipe > where the data comes from Rexx and the result is piped into Rexx > again. So please excuse if the following pipe is syntactically > wrong, but I think you get the idea: > > command='pipe console | sort | console' -- intensions read > from the console, sort, output to console > arrIn =.array~of("Max", "und", "Moritz", "diese", "beiden", > "konnt", "...") > arrOut=.array~new > -- command gets input from ooRexx' arrIn, places stdout into > arrOut > ADDRESS SYSTEM command WITH INPUT USING (arrIn) OUTPUT USING > (arrOut) That looks like temporarily redefining stdin and stdout for command calls. > -- now we need to postprocess the data received from the > NetRexx pipe Post-processing? -- well, I should restrain from malicious glee or schadenfreude because highly likely NetRexx Pipelines was not designed to work with ooRexx as smooth as Pipelines on VM/CMS. > -- assuming that each line is in the form of "varName=varValue", > -- lines that start with an asterisk (*) get ignored > -- create a directory of varName/varValue entries > dir=.directory~new > do line over arrOut -- process output of pipe line by line > parse var line varName '=' varValue > dir~setentry(varName,varValue) -- setentry will > uppercase the index varName automatically > end Rony! that's a good idea, thank you. Since this first pipeline of VilMA.rx (with more than 20 others waiting for migration) consists solely of two stages, i) reading INI file from disk, ii) set vars accordingly, _and_ the need of post-processing the pipe's output, it appears as if it's much simpler (in that artless case) to do entirely without pipe. The remaining are still good candidates for NetRexx Pipelines, worth to apply the a. m. drudgeries. > -- now you decide what do to with the directory: In this case the decision is taken, next pipe to migrate is now: > call RxPipe '< "' || .constDir[VMD_LILOC] || .constDir[VMD_TYPCP] || '"!stem tycap.' Hmm... no, that looks quite similar like the first case, so this one too I will not convert to NetRexx Pipelines. But the third pipe is worth it: > call RxPipe '< "' || fn || '"!l fs ; f1!stem loc.!spec fs 09 f1 1!stem n.' but, instead of spec a substring would do too/better. (For René, who has a ZIP of it, fn here is VilMA.Controllers.) BTW, stage L is Locate abbreviated to the max. > [...] > HTH Yes, it helped a lot. Until next hurdle I can't take on my own. Best, M. |
From: Rony G. F. <Ron...@wu...> - 2022-06-12 15:55:25
|
Learned from René that it is possible to use external programs, also Rexx programs in NetRexx pipes like: pipe literal curl http://example.com | command | cons So you can write oo/Rexx filter programs to be used as NetRexx pipe stages. The oo/Rexx filter (reading from stdin, writing to stdout) would have to be written like: do until line="" parse pull line /* read from stdin */ ... do something with line and if you want to pass on data do ... say editedData /* writes to stdout */ end alternatively, one could write the Rexx filter also as (preferable): signal on notready /* raised if stdin gets closed */ do forever parse linein line ... do something with line and if you want to pass on data do ... say editedData /* writes to stdout */ end notready: /* label to jump to */ If you also want to write to stderr then you could either do that in classic Rexx style with "call lineout stderr,data" or in ooRexx ".error~say(data)". You may want to look through these slides: <https://wi.wu.ac.at/rgf/wu/lehre/slides/BusinessProgramming/060_ooRexx_commands_V14.pdf>. ---rony On 6/12/2022 12:47 PM, Rony G. Flatscher wrote: > > Hi Mike, > > sorry a glitch in the demo code that would not check for an asterisk in the first column, here the > corrected (hypothetical and untested code sample: > > command='pipe console | sort | console' -- intensions: read from the console, sort, output to console > arrIn =.array~of("Max", "und", "Moritz", "diese", "beiden", "konnt", "...") > arrOut=.array~new > -- command gets input from ooRexx' arrIn, places stdout into arrOut > ADDRESS SYSTEM command WITH INPUT USING (arrIn) OUTPUT USING (arrOut) > > -- now we need to postprocess the data received from the NetRexx pipe > -- assuming that each line is in the form of "varName=varValue", > -- lines that start with an asterisk (*) get ignored > -- create a directory of varName/varValue entries > dir=.directory~new > do line over arrOut -- process output of pipe line by line > *if left(line,1)="*" then iterate -- ignore lines starting with an asterisk* > parse var line varName '=' varValue > dir~setentry(varName,varValue) -- setentry will uppercase the index varName automatically > end > > -- now you decide what do to with the directory: use it to set Rexx variables in the current > -- context (in this very section of the code) or maybe return it to a caller who could use > -- the directory to set variables in its context > > -- call BsfSetContextVariable("set",dir) -- will set the Rexx variable using dir > -- or: > --return dir -- will return the directory to the caller > > And an additional remark: if it is possible from NetRexx to call external programs in pipes, then > you could always inject ooRexx filters that read from stdin and write to stdout as demonstrated in > the above code. > > HTH, > > ---rony > |
From: Rony G. F. <Ron...@wu...> - 2022-06-12 10:47:24
|
Hi Mike, sorry a glitch in the demo code that would not check for an asterisk in the first column, here the corrected (hypothetical and untested code sample: command='pipe console | sort | console' -- intensions: read from the console, sort, output to console arrIn =.array~of("Max", "und", "Moritz", "diese", "beiden", "konnt", "...") arrOut=.array~new -- command gets input from ooRexx' arrIn, places stdout into arrOut ADDRESS SYSTEM command WITH INPUT USING (arrIn) OUTPUT USING (arrOut) -- now we need to postprocess the data received from the NetRexx pipe -- assuming that each line is in the form of "varName=varValue", -- lines that start with an asterisk (*) get ignored -- create a directory of varName/varValue entries dir=.directory~new do line over arrOut -- process output of pipe line by line *if left(line,1)="*" then iterate -- ignore lines starting with an asterisk* parse var line varName '=' varValue dir~setentry(varName,varValue) -- setentry will uppercase the index varName automatically end -- now you decide what do to with the directory: use it to set Rexx variables in the current -- context (in this very section of the code) or maybe return it to a caller who could use -- the directory to set variables in its context -- call BsfSetContextVariable("set",dir) -- will set the Rexx variable using dir -- or: --return dir -- will return the directory to the caller And an additional remark: if it is possible from NetRexx to call external programs in pipes, then you could always inject ooRexx filters that read from stdin and write to stdout as demonstrated in the above code. HTH, ---rony |
From: Rony G. F. <Ron...@wu...> - 2022-06-12 10:39:50
|
Hi Mike, many questions... :) Firstly one thing you should know: ooRexx is backwardly compatible with Rexx, this way one can develop and run classic Rexx programs including the Rexx scoping rules (which variables and labels can be seen from which part of a Rexx program). Ad ooRexx scopes (from my slides): * "Standard Scope": Determines which labels are visible o Labels are only visible within a program (until the end of the program or until the first directive led in by a double colon ::, whatever comes first) o Labels within of ::ROUTINE and ::METHOD directives are only visible within the code of these directives * "Procedure Scope": Determines, which variables of the caller are visible (accessible) from within the called internal routine (procedure/function) o Internal routines (labels), without a PROCEDURE statement: All variables of the calling part of the program are accessible + Internal routines (labels), followed by a PROCEDURE statement – Variables of the calling part of the program are not accessible (are hidden) # "Local scope" However: with the help of the EXPOSE subkeyword on a PROCEDURE statement one can deliberately define direct access to variables of the calling part of the program * "Program Scope" (new in ooRexx): Determines that all classes and routines defined in a program are accessible + Local classes and routines cannot be hidden/overwritten + Classes and routines can be defined to be public o In addition, this scope determines, that public classes and public routines of called or required (::REQUIRES directive) programs become accessible ● Attention! + If different programs are called one after the other, and contain public classes or public routines with the same names, then those classes/routines are accessible that are defined in the last called program * "Routine Scope" (new in ooRexx): Defines its own scope for + Labels ("standard scope") and + Variables ("procedure scope") o Accessing classes and routines is determined by the "program scope" * "Method Scope" (new in ooRexx): same as "Routine Scope" plus o Direct access to attributes ● Within a method it is possible to use the EXPOSE statement (must be the first statement in the method routine) to list those attributes of the class which should be made directly available for access from within the method routine o Determines which attributes can be accessed directly from within a method + There are two types of methods which determine the accessibility of attributes # Methods that are assigned to classes * Method and attribute directives defined after a class directive get assigned to that class * Expose and share the same set of instance/object attributes # "Floating methods" (advanced concept, can be used for dynamic programming) * Methods which are defined before a class directive are called "floating methods" * All floating methods can expose and share the same attributes with each other * Hint: accessing floating methods is possible via the environment symbol .METHODS from within the program where they are defined --- Ad ooRexx directives: ooRexx directives start with two colons (::) and instruct the ooRexx interpreter to do something at that point in time for us, before starting our program with line # 1. E.g. the "::requires someRexxProgram" causes the interpreter to call "someRexxProgram" if it was not yet called, otherwise it would reuse all public classes and public routines of "someRexxProgram". ooRexx directives are not available to NetRexx and will cause a syntax error. --- Ad BsfContextVariables(): this is a normal external Rexx function, hence you could load it with RxFuncAdd(), but I would advise against it. You can include BSF.CLS (which loads all external BSF4ooRexx functions and establishes the camouflaging support such that strictly typed Java classes and Java objects can be interacted with as if they were dynamically typed ooRexx classes and ooRexx objects) in two ways: * call BSF.CLS /* each call will run BSF.CLS and replace definitions of prior calls of BSF.CLS */ * ::requires BSF.CLS /* the first time ooRexx sees this directive it will call BSF.CLS but reuse it if it encounters later ::requires BSF.CLS in other ooRexx programs */ BsfContextVariables() allows one to get all currently defined variables as a directory (index is name of the variable, item the current value) for which in the meantime alternatives exist (.context~package~variables). BsfContextVariables() allows you to set multiple Rexx variable in the scope where you invoke it, if you supplied a directory with all variables that should get defined. For this usage I would suggest BsfContextVariables(). --- Ad fetching data from external sources. ooRexx 5 gained the ability to use the ADDRESS keyword instruction to redirect stdin, stdout and stderr from/to ooRexx if desired. To do so you define what collection you want to use from ooRexx, e.g. a stem or an array. Not being familar with Netrexx pipes here a hypothetical pipe where the data comes from Rexx and the result is piped into Rexx again. So please excuse if the following pipe is syntactically wrong, but I think you get the idea: command='pipe console | sort | console' -- intensions read from the console, sort, output to console arrIn =.array~of("Max", "und", "Moritz", "diese", "beiden", "konnt", "...") arrOut=.array~new -- command gets input from ooRexx' arrIn, places stdout into arrOut ADDRESS SYSTEM command WITH INPUT USING (arrIn) OUTPUT USING (arrOut) -- now we need to postprocess the data received from the NetRexx pipe -- assuming that each line is in the form of "varName=varValue", -- lines that start with an asterisk (*) get ignored -- create a directory of varName/varValue entries dir=.directory~new do line over arrOut -- process output of pipe line by line parse var line varName '=' varValue dir~setentry(varName,varValue) -- setentry will uppercase the index varName automatically end -- now you decide what do to with the directory: use it to set Rexx variables in the current -- context (in this very section of the code) or maybe return it to a caller who could use -- the directory to set variables in its context -- call BsfSetContextVariable("set",dir) -- will set the Rexx variable using dir -- or: --return dir -- will return the directory to the caller You probably get the idea and can assess better how you can use the ooRexx and BSF4ooRexx infrastructure. HTH ---rony On 6/12/2022 4:56 AM, hp...@we... wrote: > Hi Rony! > > Once started from the promise/allegation, NetRexx Pipelines would > work the same as the role model on VM/CMS I planned to eliminate > OS/2 Pipelines in a ooRexx program -- and René agreed to help -- > we are now at the detail point, how do I get BSFContextVariables > to do what I need? Currently I'm not convinced to be still on > track, but that's my impression which could be wrong. > > Let me picture the idea I have about REXX variable pools. It may > be the source of failure because too coarse or completely wrong or > too much influenced by REXX as I know it from VM/CMS. So please > holler if I have the wrong picture. > Every REXX program has its own variable pool. Subroutines within > the same file share this pool _unless_ protected by "procedure" > which optional may share a selected set of variables by "expose". > Within ooRexx it's just the other way round, objects and methods > are protected "like lepers", what might help in vast program files > where by chance use of same variable names might occur. So each > "entity" has a variable pool of its own. > > This "keep 'em separated" is absolute when running NetRexx > Pipelines from ooRexx, since stages like var, stem and alike all > fail (from an ooRexx-centric point of view). The solution using > the stack to bring an ooRexx stem's content to the NetRexx pipe > shown by Gil Barmwater works, highly likely it could also work > backwards. Alas, NetRexx Pipelines lack the stack stage. > > At the moment I have trouble with BsfContextVariables() in two > ways, theoretically and in practice. > Theoretically with respect to the current variable pool involved: > > Am 10.06.2022 um 16:00 schrieb Rony G. Flatscher: >>> [...] >>> Please define "current" ;) >> >> It is the context of the Rexx code executing >> BsfContextVariables(). ;) > > In that case a simple a = b would set the same variable A as does > BsfContextVariables('Set', a, b). So there is for sure something I > did not grasp yet. Nonetheless I gave it a try -- the trouble in > practice: > The first replacement of OS/2 Pipelines by NetRexx P. would be: > >> ripe = '@java -cp "', >> || value("NETREXX_HOME", , "ENVIRONMENT"), >> || '\lib\NetRexxF.jar;', >> || value("CLASSPATH", , "ENVIRONMENT"), >> || '"' 'org.netrexx.njpipes.pipes.runner' >> fn = vilma.ini >> '' ripe '"(sep !) <' fn '!nxvarload' > > with nxvarload being a draft of a future varload stage, a current > snapshot of it: > >> /* nxvarload.njp: NetrexX VARLOAD, still incomplete */ >> import org.netrexx.njpipes.pipes. >> class nxvarload extends stage >> method run() >> do /* who needs this? */ >> addpipe (bs end ? sep !) *in:!a: fanout!strip!locate!*in:?a:!*out: >> loop forever >> line = Rexx readto() >> parse line.upper sc +1 vn (sc) sv /* UPPER to force keyword SYMBOLIC */ >> if sc <> '*' then BSFContextVariables("Set", vn, sv) >> end >> catch StageError >> rc = rc() >> end >> exit(rc*(rc<>12)) >> ::requires BSF.CLS > > (In case you wonder about the addpipe, 'the Book' specifies, > "varload strictly does not delay the record." That implies, even > without explicit operation description, varload copies records to > the primary output stream.) > > Problem: without ::requires directive BSFContextVariables() is > unknown, with it the PIPC compiler or translator dislikes the > colons of the directive. Frankly, methinks it's a bit contorted to > use a ooRexx directive and function within a NetRexx stage, but > without know-why I am pretty helpless. > >>> [...] >> There is no choice available to use any other pool than the >> context one, i.e. the variable pool of the location where your >> program runs BsfContextVariables(). (You could have a routine >> which collects the pipe results and returns it and at the receiver >> side issue the BsfContextVariables() with that data.) > > When I have collected pipe results in ooRexx I see no need for > BsfContextVariables(). Sorry, but I would access the results > collection directly. > >> [...] >> If you want to set variables (instead of interpret) you can use >> the value() BIF (built-in function) like this: >> >> call value hi, 'hi, my beloved world' >> say hi /* will say 'hi, my beloved world' > > Nice, but how do I get that "hi, my beloved world" from NetRexx > Pipelines to that BIF value in line one? > > Please advice. > M. |