[netrexx-pipelines] Fwd: Migrate OS/2 pipelines to NetRexx Pipes

[René J. <rvj...@xs...>]

[and one for the list]


Hi Mike,

The way to make it faster would be to precompile the pipe to a java .class file using pipe and an .njp file, and instead of addressing pipe, to address java yourpipe.class. Of course, when called from a NetRexx program, you would have the java context already.

But to show the difference:

➜  ~ time pipe "literal aap noot mies | split | sort | cons"
aap
mies
noot
java org.netrexx.njpipes.pipes.runner   1.76s user 0.08s system 325% cpu 0.567 total

So that is 1.7 sec overall, but 0.5 sec net due to multiple cpu’s in this machine (that is the 325%)

But when we compile this, 

➜  ~ emacs cpipe.njp
➜  ~ cat cpipe.njp
pipe (cpipe) literal aap noot mies | split | sort | cons
➜  ~ pipc cpipe
pipe (cpipe ) literal aap noot mies | split | sort | cons
➜  ~ time java cpipe
aap
mies
noot
java cpipe  0.05s user 0.02s system 106% cpu 0.064 total

So compiling it speeds it up around 100 times, I expect the same thing to happen with your pipeline.

What I do, is using the pipe and nrws commands for development of the pipeline (up-arrowing and editing until it is working like I want it) and then compiling when it is “ready for production”.

There is one thing I am fixing at the moment, and that is an issue that Leslie Turriff reported first: with a portrait pipeline, the pipe separators must be at the end of the line for the pipe compiler (for some reason), and he (and I) like them at the start of the line. This will be hopefully fixed in NetRexx 4.04 when it is GA. Until then, the line needs breaking at unfortunate places.

Best regards,

René.

> On 15 Jun 2022, at 00:41, hp...@we... wrote:
> 
> Hi,
> 
> at least I have done the first not-for-test-only NetRexx pipe.
> A very simple one, read a file, discard comments, save first
> record in a variable, save rest in a stem.
> 
> Alas, some points displease:
> i) the ooRexx integration per Address "" ... input ... output
> force some changes, there is only one output connection possible,
> but this pipe has two, a variable and a stem. Thus I had to solve
> this in the post-processing.
> ii) the input file consists of 10 records, 5 comment and 5 data,
> what is close to nothing. Nonetheless it takes almost 1.7 seconds
> to process, what is way too much for almost nothing. I hope there
> is a chance to accelerate it notedly.
> iii) labels require a blank after the colon,
> iv) stage Locate can't be abbreviated to L, (yes, sure, this and
> the point before is of minor importance, I know, but it adds to
> the annoying differences to what I'm accustomed),
> v) nonexistent input file ends processing of the ooRexx program
> with an error. I assume this is due to the Address input/output
> enclosure, Pipelines would just set the variable and the stem
> accordingly.
> 
> Here what it was (set as comment) and how I replaced it:
> 
>> -- call RxPipe '(end ?) file "' || vilif || '"!strip!l!nlocate 1 /*/!a:take!var conif',
>> --  '?a:!b:lookup 1 detail!stem livil.',
>> --  '?literal +-!fblock 1!b:'           /* filter with lookup as locate missing option anyof */
> 
>> f = .stream~new(vilif)
>> address "" ripe '"(sep ! end ?) cons',
>> '! strip!locate!nlocate 1 /*/',
>> '!a: drop!locate 1 any /+-/',
>> '!b: fanin!term',
>> '?a:!elastic!b:',
>> with input stream (f) output stem livil.
>> drop f
>> /* post-processing */
>> conif = livil.[livil.0]
>> livil.0 -= 1
> 
> For a simpler post-processing I replaced take by drop. The changes
> for a missing input file are not shown.
> 
> All suggestions for speed-up are welcome.
> 
> Best,
> M.
> 
> 
> _______________________________________________
> netrexx-pipelines mailing list
> net...@li...
> https://lists.sourceforge.net/lists/listinfo/netrexx-pipelines