Menu
▾
▴
Re: [netrexx-pipelines] Migrate OS/2 pipelines to NetRexx Pipes
|
From: <hp...@we...> - 2022-06-17 11:00:16
|
Hello René!
Totally overwhelmed by your huge and detailed reply I don't know
where to start, as a simple thank you is just not enough. Should I
bow down mumbling reverently 'Dear sir, most respected Esquire' or
even apologize for stepping on your toes, because such an answer
is also a little bit a defence, kind of "look we can, we just did
not yet."
Therefore I don't need to comment every paragraph, some words only
where I feel it might complement your assertion.
Am 16.06.2022 um 11:37 schrieb René Jansen:
> to know what would be a good description of njp I would need the structure of the required documentation. That would maybe be a welcome addition to the document we have now.
You may describe a subject just as it is, or by explaining the
difference to what it was before. The later requires a target
group which knows well enough what it once was. In case you decide
to outline the subject in full, it will for sure be boring for the
"flock of those knowing". So it is a significant decision, _who_
do you want/need/like to 'ramp up'.
You need a structure of the required documentation? Just lean back
(for the creative momentum) and try to see the forest not only
many trees. Put yourself in the position of somebody without all
your entirely knowledge -- where would you start? With a
top-down-approach, an overview first, ok, chapter 1 Introduction,
done. Next, more details, 2 - The Pipeline Concept, also done.
Then, how to deal with it, a problem-oriented description of tasks
and tools, all the chapters 3..18 what is IMO the 'guide' part,
almost done, it lacks only the reference to the (currently
missing) documentation. Chapters 19 and 20 are already part of the
appendix with similar tables. So for my flavour between chapter 18
and 19 the reference could be inserted, documenting each and every
stage, argument, option, keyword, error message and corresponding
corrective action.
Frankly, this is only my point of view, driven by the idea to look
for what I want to know where I am accustomed to find it. Maybe
there are simpler, cheaper, somehow better options to achieve the
same result.
> Let me try to make a start:
> [...]
IMHO, this is mainly history, important to understand what's up
today, essential part of the know-why, to be filed in chapter 2.
> To compile an .njp file to a class
I guess, the following documentation of pipc's arguments and
options, you did not write it down right now for this reply, it
was available somewhere but did not make it to the manuals -- or I
was not able to find it.
> The compiler is invoked with the command:
>
>
> pipc <arguments>
>
> Where <arguments> are:
>
> <compopt>|<fileid> {<compopt>|<fileid>} - compile the files listed
> ( <name> { <compopt>} ) <pipe> - compile the <pipe> name
The pipe might have options too, is the sequence of those and
<compopt> irrelevant?
> Where <compopt> is:
>
> -gen - keep the .nrx source of pipes following
> -nogen - delete the .nrx source. This is the default
Pipelines Guide and Reference, chapter 18 lists also option -keep.
> -<option> - where <option> is a valid NetRexx option
>
> And <fileid> is:
>
> <name>.nrx - any NetRexx file
> <name>.njp - any pipe or NetRexx or Java souce using pipes
> <name>.grp - a file containing a list of <compopt>s and/or
> <fileid>s. lines can be commented using '--'
For the documentation of pipc this is sufficient, while the syntax
of the files should be specified separately, in case of deviations
also separately for each filetype.
> The <options> are:
>
> runaway <milliseconds> - Interval to use for deadlock checking. This
> or only detects true pipe deadlocks, not stages
> stall <milliseconds> that are looping or hung. (Should this be
> changed?)
Ahem... question from the hobbyist: what has a stall to do with
milliseconds? To my (uneducated) understanding, a stall is a
situation, when the dispatcher can't find no stage to run while
there is still data to flow. This is IMO a matter of strategy, not
of milliseconds. In contrast to a stage hung, where I have to
probe every now and then, if it consumes CPU without any I/O (or
here pipe adequate, w/o data flow).
> package <name> - insert a NetRexx package statement
> import <name> - insert a NetRexx import statement
> config <name> - override the default config file name (see below)
> sep <char> - override the default stage separator '!'
> end <char> - override the default end of pipe separator '?'
The role model has no default end character.
> cont <char> - override the default line continuation character ','
> debug <flag> - debugging options
All documented in context of pipeline and listed here only for the
record.
> When the pipe is contained in a file the options must be set there.
>
> When a pipe is read from a file, text on the same line following a '-- ',
> will be treated as a comment.
>
> Errors issued by the compiler.
Ah -- at last ;)
> Stages can issue additional messages.
>
> Error - name of pipe is missing
... and so on -- also most error messages are clear without
further explanation, there might be some requiring few more words
I assume, like
> Error - problem reading group rc='r'
for example.
> Debugging pipes
>
> To find out what is happening in pipeline you have three tools. First,
> you can set a debug flag when you compile the pipe. The bits you set
> in the flag control what it does:
>
> 1 - Show all pipes starting
> 2 - Show all pipes ending
> 4 - Show all stages starting
> 8 - Show all stages stopping
> 16 - Show all Commit requests
> 32 - Show all Commit completions
> 64 - Show StageErrors raised via stage's Error(int,String) method. The
> stage class uses Error for all its StageError signals.
> 128 - Show the arguement that each stage is recieving. Handy since
> shells have a habbit of doing unexpected thing to arguements.
> (try: java findtext exit *.nrx vs java findtext "exit *.nrx")
Wait a moment... single stages will receive arguments from the
shell? On Windows from the cmd.exe? Where is this described?
Piping as I know it _might_ set options for each stage separately
(if necessary), typically arguments only, but they are all
contained in the pipe specification part as argument to the pipe
command (or compiler in this case), at least to my understanding.
> - If there is a need I can easily add more events - just ask.
Well, that depends on the dispatcher of NetRexx Pipelines, which
states it knows in a stage's "lifetime", similar to those listed
here http://vm.marist.edu/%7Epipeline/pipeline.pdf on p. 243,
'States of a Stage'. All possible transitions might be interesting
for serious debugging.
> To create a flag to see all stages starting and stopping you would
> add 8+4 and use:
>
> pipc (apipe debug 12) ...
>
> The second option is to use the invoke the dump() method in a stage.
> This dumps the status of the pipe using the same format you see when
> a pipe deadlocks <file:///Users/rvjansen/Library/Mobile%20Documents/com~apple~CloudDocs/Documents/Programming_languages/Pipes/njpipes.webarchive#dead>. Using dump() does not normally cause a pipe to
> terminate. Once in a while dump() will generate an exception. This
> happens since dump() does not use protect or syncronize so it does not
> stall.
But that is only possible from within DIY stages, I assume.
> When all else fails, you can recompile all the stages used by the pipe
> with trace results enabled.
So this was a good start, it should become a chapter in the
reference part of Pipelines Guide and Reference.
> There is a category of problems that is caused by the different ways of processing the pipe - most of them are very new, like using address in NetRexx. Some of them are older, because to enable calling from a commandline, a synthetic name will be generated. This is caused by the fact that you can call an unnamed pipe from the commandline in VM - if that were disallowed, we would not have this problem now. For the moment, I can only say that all .njp examples in the examples/pipes directory work as specified.
Ah.. sorry, I have no clue about problems inventing names (I would
convert the timestamp to allowed characters -- since it's for
commandline entry an adequate coarse resolution would do). Example
from VM/CMS:
> 'callpipe literal!spec tod!spec 3-*!t62vld!var tft'
with t62vld REXX translating to allowed characters like this:
> * * * Top of File * * *
> /* Translate FOLDed to valid characters of CMS file system */
> /**********************************************************************\
> | From VALIDATE HELPCMS: |
> |The file name and file type can each be one to eight characters. The |
> |valid characters are A-Z, a-z, 0-9, $, #, §, +, - (hyphen), : (colon),|
> |and _ (underscore). |
> \**********************************************************************/
> 'callpipe (end ?) *:!vchar 6 8!a:xlate!*:',
> '?literal', /* transliteration table: */
> '81C1C2C3C4C5C6C7C8C9828384854E87' !!, /* aABCDEFGHIbcde+g */
> '88D1D2D3D4D5D6D7D8D9895B91929394' !!, /* hJKLMNOPQRi$jklm */
> '6095E2E3E4E5E6E7E8E99697986D99A2' !!, /* -nSTUVWXYZopq_rs */
> 'F0F1F2F3F4F5F6F7F8F97A7B7CA3A4A5' !!, /* 0123456789:#§tuv */
> '!spec 1-* x2c!a:'
> * * * End of File * * *
That is the way I once prepared unique filenames. Simple as that.
> Stages have their own error messages - I must compile a pipe to list those all to the documentation.
> Please let me know what more you need for documentation.
Yes -- the messages recently:
> Error (specs) - pd673a59 - Missing data selector at field2
and also
> "Error - pipe as stage definition VILMA_1.CLASS is missing a period"
For both issues I found no hint how to cure it.
Well, I know, the documentation I ask for "c'est la galère, un
boulot monstre" -- lots of work. Yesterday I met an emulator
programmer (if you are interested in HP calculators you probably
use either Emu48 or Emu42, ...), in his opinion it's not uncommon
to program by trial and error, since sometimes the is no
description at all or it's incomplete or wrong. But this approach
seems to me like hunting for Easter eggs. When you say NetRexx
works like its role model and I can't reproduce this experience,
the effect is similar an empty promise.
Best regards,
M.
|
