Menu

How to run NetRexx-Pipelines from ooREXX?

m-stgt
2021-08-19
2021-09-07
  • m-stgt

    m-stgt - 2021-08-19

    Hi René! :)

    It appears to me, there is a difference in forums and mailing lists, for the later I would have to subscribe also in ooREXX.

    Next, so much help (not only you) puts me under pressure to look here and there and test this and that. In addition, I remark some deficiency in my knowledge. Simple example: you write "you can always ’shell’ a pipeline with oorexx" and I think -- "shell"? that's something from Linux or macOS, but not Windows. Then I read in 'Pipelines Guide and Reference', Version 4.01-GA of March 20, 2021, chapter 3.3 "...in the command shells of Linux, Windows and macOS." Bang! Shell within Windows -- what is it?! The DOS-like interface of cmd.exe? Or something else I never missed so far?

    I am sorry, at work I was a class G user, and at home I am not an expert using PCs. What works for me are examples, at best with explanations. Currently I scan through tha a. m. 'Pipelines Guide and Reference' of NetRexx, but only theoretically as installing it will probably jeopardize my ooREXX programs. I have not yet found a recipe how to install both, ooREXX and NetRexx, together with some shell (Classic Shell, Open Shell, Aero Flip 3D? seems there are several options).

    Best,
    M.

     
  • Gil Barmwater

    Gil Barmwater - 2021-08-20

    In the hope that you will not be deterred in pursuing your objective, I decided to attempt to get it working myself. But first a comment re. "shell" in the context of Windows. You do not need to install anything extra as the console window that opens when you run cmd.exe is in fact the "shell" that René referenced and that is mentioned in the Pipelines Guide.

    So starting from scratch this morning, I retrieved the documentation, studied parts of it, installed and tested NetRexx 4.01-GA, tested pipelines which is included and wrote a simple ooRexx script to run a NetRexx pipeline! This had no effect on my ooRexx installation or programs so that is not a concern. I documented my steps and am attaching a file named journal.txt that describes what I did. I am also attaching the ooRexx program that I wrote named nrxpipe.rex. I am happy to explain any parts or answer other questions on my process via this forum.

     
    • m-stgt

      m-stgt - 2021-08-20

      Hi Gil!
      Thank you very much for your time and effort to create a stepwise recipe. I hope, not only me will use it. Also thank you for clarification of "shell" being just the DOS-remembrance command window.

      Currently I am stuck at step 7 of your journal.txt. Retracing steps 6, 4 and 2 if such a cmd was mentioned failed. Your hint in brackets for step 7 names NETREXX_HOME and bin subdirectory, so I search for these at steps 6, 4 and 2. Strike -- NETREXX_HOME is mentioned in chapter 3.1 "Configuration" of "Pipelines Guide and Reference", in files pipe.bat and pipc.bat -- but... it appears to me that there NETREXX_HOME it both batch files is used and already set. Unpacking a ZIP bundle is not an installation, so it is obvious, I missed something. Scrolling a bit down shows a grey box wich tells me what I just guessed, alas no hint or reference about how to do as required.

      So the first 6 steps are done, unless "study" should result in activity, what is a bit more than I understand by your journal steps 2, 4, and 6.

      I you still have hope and faith, please advice :) In the meantime I continue to study.

      Edit: Wading at slow pace through the Quickstart Guide, I included bin subdirectory to the path instead of copying its content to a directory already listed in path. See no harm in this modification. But I do not yet grasp what must/could/should/may be set in CLASSPATH environment variable. Description in chapter 3.6 is a bit too "quick" for me. (May continue on Sunday earliest)

       

      Last edit: m-stgt 2021-08-20
      • Gil Barmwater

        Gil Barmwater - 2021-08-20

        Yes, sorry I did not include the nrxstart.cmd as an attachment since it is specific to the folder/directory structure that I used. So I have the following set up: under my root folder (C:) I have a Rexx folder and then a projects folder. For this project I created a nrxpipes folder and placed everything in it. So I have C:\Rexx\projects\nrxpipes and in it is nrxstart.cmd. Here is its contents:

        @rem Setup for using NetRexx
        @echo off
        if defined netrexx_home goto :eof
        set netrexx_home=C:\Rexx\projects\nrxpipes
        path=%netrexx_home%\bin;%path%

        The third line allows for accidental rerunning of the batch file with no ill effects. The next sets NETREXX_HOME to point to that directory and the last adds its bin subdirectory to the beginning of the path so the executables are available. Hopefully that will let you create an equivalent version that matches your configuration.

         

        Last edit: Gil Barmwater 2021-08-21
      • Gil Barmwater

        Gil Barmwater - 2021-08-23

        Not having heard about any further progress, let me add the following. Step 7 - creating a batch file named nrxstart.cmd - is not required IF you 1) add the \bin subdirectory to your path (as you have done) and 2) set NETREXX_HOME to the directory where you unpacked the ZIP file. (See the fourth line in my nrxstart.cmd for the syntax.) After that you should be able to move to step 8!

         
        • m-stgt

          m-stgt - 2021-08-25

          Gil! Hello again!
          After verifying top 1) \bin is set in path, and executing set netrexx_home=D:\PRGM\NetRexx(this is not permanent, next session I do have to enter this again, yes?) I updated your journal.txt for step 7 not to create the nrxstart.cmd, then I stepped through the read.me.first and executed all as described there. All worked with no error.

          Ah -- wait! -- that was step 8 already. Question: is that \lib directory the work directory for all future tests/programs/routines or may I structure my work in directories somewhere else?

          Step 9 of jour journal.txt -- section 3.1 of the Pipelines Guide is titled "3.1 Configuration" and starts with .bash_aliases: what fails. Maybe it's not for Windows, so I move on to the batch files, but there is a CLASSPATH used, which caused me some trouble last week already. There is a paragraph 3.6 Setting the CLASSPATH in NetRexx Quick Start Guide, alas -- not for me, I do not belong to that target group.

          So I give up. Disappointed I just did an edit/copy-edit/paste of the pipe on next page. Result: "Der Befehl "dup" ist entweder falsch geschrieben oder" and so on. So it was not pipe for what it complains. Can't remember where I saw something about double quotes, so I test again with double quotes. Took quite a while until it returned 100000. So it worked without all the back and forth of ClassPath and more described before. Not somethng I have to understand.

          Next, step 10, run pipe under rexxtry. Took a while to find it. Drag-n-drop to the cmd window, ends it emmediately. Doubleklick in explorer asked how to run it. Strange. So i created my own:

          /* kind of rexxtry */
          trace '?R'
          address ''
          

          and typed in the pipe again. Alas it fails, no matter if single or doble quotes.

          That's it for today, more this or next week.

           
          • Gil Barmwater

            Gil Barmwater - 2021-08-25

            Hi Mike. Sorry that you are still struggling with getting this running. I think I know part of the problem that I've been having in trying to help you. It seems that you normally start your programs by double-clicking on them while I've been assuming you were running them from a command prompt. Unfortunately, the NetRexx documents also assume you are at a command prompt so we are at cross purposes! Before I suggest anything else, let me ask how you added the bin subdirectory to your path? Did you use Control Panel to modify the current value?

             
            • m-stgt

              m-stgt - 2021-08-26

              Hi Gil,
              no need to be sorry, it's not your fault when I am not an expert on Windows.
              Sure, most programs I start in the explorer by double-clicking a related file with its suffix linked to it, e. g. .txt and .log then are shown in an editor. When it comes to REXX it is a bit different, while developing I open a cmd window and start the program I am working at from the cmd window, finished programs I either start in the explorer by sending to rexxpaws or -- if its a dialog application I use rexxc to convert it also changing the suffix, so it is recognized as GUI program.

              Now I found one mistake I overlooked yesterday. The rexxtry routine has the suffix of .rex what is on this machine for GUI programs, "normal" REXX has a .rx suffix. So when I started rexxtry it did nothing visibly, so I tried several times to start it what made all CPUs work 100% for nothing. Changing the suffix of rexxtry did the trick.

              Another point that could cause some trouble, I have OS/2 Pipelines installed (now 20 years old), so to avoid any interference I renamed it now to hipe.exe in the hope this will be sufficient.

              How I added \bin to path? Well, I wanted that permanently so I searched in "PC-Einstellungen" (sorry, who installed this machine did it in german) for 'path' what gave me two options, one for the system and one for my account. One or the other worked for me, can't remember which one. What wonders me, when I open a cmd window today and query path, this \bin is mentioned in the middle of many others, yesterday (or before ?) it was at the end.

              Now I wanted to continue where I was too tired yesterday. I've learned so far, I have to start a cmd window and the pipelines specification must be double-quoted. So to test I entered pipe "literal a b c|split|term" what failed. "Hauptklasse org.netrexx.njpipes.pipes.runner konnte nicht gefunden oder geladen werden" -- blah-blah org.netrexx.njpipes.pipes.runner not found. Why? Path to \bin is set. A look to your journal.txt shows for point 7: set netrexx_home=D:\PRGM\NetRexx -- Hmm... is it possible to make this permantent like \bin in pasth too? Would help a lot. It made my pipe test work.

              Now back to rexxtry -- if I'm able to run a pipe in the cmd window, the same pipe must run from rexx too. Otherwise it's not "my" Rexx.

              D:\PRGM\NetRexx\lib>"C:\Program Files (x86)\ooRexx\rexxtry.rx"
              REXX-ooRexx_4.2.0(MT)_32-bit 6.04 22 Feb 2014
                rexxtry.rx lets you interactively try REXX statements.
                  Each string is executed when you hit Enter.
                  Enter 'call tell' for a description of the features.
                Go on - try a few...            Enter 'exit' to end.
              pipe "literal a b c|split|term"
              Der Befehl "split" ist entweder falsch geschrieben oder
              konnte nicht gefunden werden.
                rc = 255 ................................... rexxtry.rx on WindowsNT
              
                rexxtry.rx:  Enter 'exit' to end.       Or '?' for online REXX help.
              

              So not pipe is the problem, "split" is criticised. If I remove split, term is "misspelled or not found". Very questionable. Same if I prefix with address ''. At least I found single quotes do the trick: 'pipe "literal a b c|term"'.

              Next step (not from your journal.txt), access to REXX variables from pipes is essential. So I did in rexxtry set an unused variable, nixbix = 'Hello world!' and then 'pipe "var nixbix|term"' Alas, no Hello!

              What did I wrong?

               
              • m-stgt

                m-stgt - 2021-08-26

                Hello Gil!
                Intrigued by my "progress" tonite, I tried to advance a bit more. So your 'journal.txt' step-by-step recipe would currently read after step 6
                - add \bin subdirectory of NetRexx installation to your path within Windows system properties (once in lifetime),
                - open a Windows cmd window,
                - execute set netrexx_home=D:\PRGM\NetRexx (all the time),
                - work in a subdirectory at will (that's a query to be confirmed)
                - run pipe with the specification enclosed in double quotes like pipe "query|term"
                - run pipe from an ooREXX exec as 'pipe "query|term"' or address command 'pipe "query|term"'

                Nice. Alas access to variables fails this way,

                nixbix = 'Hello world!'
                say nixbix
                Hello world!
                'pipe "var nixbix|term"'
                

                returns a blank line, so var nixbix sourced an empty record, what is more than nothing (but questionable).
                There is a nice overview in ...\NetRexx\documents\stages.pdf that compares to CMS-Pipelines. For var I find a not so nice note: Pipes for NetRexx: this can only read vars -- Ah... hmm... well, no work around mentioned? Could varover be of some help? The note with it says NetRexx Pipelines only; not CMS Pipelines. And what about ooREXX?

                Question: do you see a chance to get something like following to work with NetRexx Pipelines?

                   call RxPipe '(end ?) file "' || videv || '"!strip!l!x:inside /[Settings]/ /[/',
                   '!a:find AddrOut=!b:faninany!insert /=/!varset',
                   '?a:!c:find PortOut=!b:',
                   '?c:!find PortIn=!b:',
                   '?x:!inside /[Run]/ /[/!xlate!find PLACEHOLDER!take!count lines!var dmy'
                

                Seems it requires replacements for varset and var to set variables.

                M.

                 
              • Gil Barmwater

                Gil Barmwater - 2021-08-26

                So, it looks like lots of progress! There is a lot to comment on in this post and the next one so I've decided to split them up into multiple posts.

                First, your explanation of how you normally work is very helpful and will guide me in making suggestions. Just a note on running rexxtry - while your solution to rename the suffix certainly works, you could have instead typed rexx rexxtry.rex. Now the normal rexx.exe is invoked rather than rexxhide. Not a big deal, just for your information.

                Next, the existence of OS/2 pipelines could have caused issues had you not renamed it. It depends on where in the path it is located relative to the NetRexx lib subdirectory which also contains a 'pipe' executable. The first one found is the one executed.

                Finally, your question on making NETREXX_HOME permanent. Yes this is possible and I would recommend that you do so. The NetRexx team has made running their Pipelines package easy by shielding users from the details of Java, classpaths - more on that later - and the name of the actual program that needs to be executed. But in order to do so, the user needs to set the environment variable NETREXX_HOME to the directory where he installed the package. In my next post I will try to give you the steps to make this environment variable always available.

                 
                • Gil Barmwater

                  Gil Barmwater - 2021-08-26

                  On my system I did a google search for 'setting environment variables windows 10' which led to a series of steps which I followed. Once the dialog opens showing two windows - one for user variables above a second one for system variables - you are ready to add NETREXX_HOME. Click on the 'new' button under the user window. A new dialog opens with two entry boxes. In the first type 'netrexx_home' - case does not matter. In the second box type the path to the directory where you 'installed' NetRexx. Based on your posts that might be D:\PRGM\NetRexx. Click OK on all open dialogs. Open a new command window and type 'set ne' and press enter. You should see NETREXX_HOME=D:\PRGM\NetRexx. If so, you never have to worry about this again!

                   
                  • Gil Barmwater

                    Gil Barmwater - 2021-08-26

                    On to the issues you encountered that you described in the first post: 'org.netrexx.njpipes.pipes.runner not found'. This is the NetRexx program that actually does the work. It is located in the NetRexxF.jar file. The 'pipe' command - actually pipe.bat on Windows - is found because it is in the path. It tries to invoke Java with an argument -cp (which stands for classpath) that makes that .jar file available. If NETREXX_HOME is not set (or set incorrectly), then Java can't find the .runner program and you get this message.

                    The problem with Rexxtry is that the pipe.bat batch file needs to "see" the double quotes so it knows that the entire string is one argument. However, Rexx thinks that you have defined a literal string within the quotes and just passes the contents so pipe.bat receives multiple arguments instead of just the pipeline. In your next sentence you show the solution - surround the whole line with single quotes.

                    Finally, for this post, you try to pass a variable to the 'pipe' command: 'pipe "var nixbix|term"'. However, the solution to passing the double quotes has now bitten you because the variable is inside the single quotes and Rexx does not do the substitution you wanted. This, although hard to read, should work:
                    'pipe "var' nixbix'|term"'. To Rexx this is a literal - up through var - blank concatenated to the variable nixbix and then concatenated with the second literal.

                     

                    Last edit: Gil Barmwater 2021-08-26
                    • m-stgt

                      m-stgt - 2021-08-26

                      Hello Gil!
                      Your advice for using Rexx variables by inserting some more quotes -- 'pipe "var' nixbix'|term"' -- do you use that often? And it works for you? Here under Windows 10 it makes not difference if more or less quotes.
                      The only reasonable way I could imagine would be 'pipe "literal' nixbix'|term"' so the content of the variable is used as argument of the literal stage. But transfering data this way is error prone, depending of the variables content it may have unwanted effects. Thus "injecting" directly with no chance for Rexx to interpret the data is the canonical way (on CMS).

                      Remains the hurdle how to get data from the pipe to the Rexx viariable pool.
                      M.

                       
                      • Gil Barmwater

                        Gil Barmwater - 2021-08-26

                        No, it does not work for me and I should have tested it before posting! I did some reading about the VAR stage after(!) I posted and discovered that VAR is supposed to retrieve the value of the variable NAMED nixbix. So your solution of using LITERAL rather than VAR seems more reasonable. Remember that when the NetRexx Pipeline documentation talks about "REXX", it is NetRexx NOT ooRexx. So unless there is a way for one to 1) create NetRexx variables and 2) assign them their ooRexx values, then the VAR stage seems to be not usable in this context. Perhaps the gurus have a work around?

                        As to your other question, getting data from the pipeline INTO ooRexx variables, I will again defer to the NetRexx guys. (ooRexx 5 has an enhancement that will put the output that normally goes to the screen via TERM or CONSOLE into, say, a stem but that is not available in 4.2.)

                         
                  • m-stgt

                    m-stgt - 2021-08-26

                    Hello Gil!
                    Thank you very much for your detailed explanations. So I now have created and set NETREXX_HOME I may test once more step 8 of your journal.txt recipe. Task: perform all tests mentioned in read.me.first found in NetRexx installation directory.
                    As my goal is not only to run the tests, but use NetRexx-Pipelines in any subdirectory, I intentionally do not conduct the tests from \lib as stipulated but from a test-dir somewhere else. Result: all works, even compilation by nrc hello, it only failed i) java -jar NetRexxF.jar hello and ii) java -cp NetRexxF.jar;. hello

                    Question: even though path and netrexx_home are now set permanently, running compiled routines is only possible from the \lib subdirectory? For what reason?
                    M.

                     
                    • Gil Barmwater

                      Gil Barmwater - 2021-08-26

                      I will defer to the NetRexx guys for the answer to this one as they can probably answer right away while I would need to do a lot of research and testing to figure it out. Sorry but this is not my area of experience ;-(

                       
                      • m-stgt

                        m-stgt - 2021-08-27

                        Hi Gil!
                        Thank you very much for your time, effort and the perseverance to help me regardless of not being so familiar with PCs. With your support I now have two Pipeline systems available in parallel, OS/2 Pipelines and NetRexx-Pipelines.
                        Let me show you what was my goal for this exercise. It's a little tool to select, arrange and launch several programs and ensure they talk together. Nothing of global importance. I developed it with ooDialog but quick-n-dirty, and to be real quick I used Pipelines. Alas with OS/2 Pipelines which are i) internal use only and ii) awesome flawy. Each reason on its own is enough not to distribute it. For this I hoped to find in NetRexx a Pipeline that does well together with ooDialog.
                        Another option would be to redo the Pipelines by "flat" REXX. But... no, no, no. No way for a pipe-addicted ;)
                        Once again, thank you.
                        M.

                         
                      • m-stgt

                        m-stgt - 2021-08-27

                        While the NetRexx guys seem to be out for holidays (vital, no doubt) or elsewhere (even more vital), I stumble over something remarkable, strange. Executing not from command line of a cmd windown but from a REXX so there is a variable pool for the var stage:

                        'pipe "var|term"'
                        
                        say nixbix
                        NIXBIX
                        'pipe "var nixbix|term"'
                        
                        'pipe "literal|term"'
                        null
                        

                        i) var stage does not complain if the required argument is missing?
                        ii) how does it get the blank line?
                        iii) say takes nonexistent variable as literaral, uppercase as all variable names, in contrast var stage makes no difference if without argument or nonexistent variable (what makes a distinct difference to CMS),
                        iv) literal stage evolves NetRexx makes a difference between blank line and null record. That makes me nosy:

                        'pipe "var nixbix|count bytes|term"'
                        0
                        'pipe "literal|count bytes|term"'
                        java.lang.NullPointerException
                                at org.netrexx.njpipes.stages.count.run(count.java:249)
                                at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:76)
                                at java.lang.Thread.run(Unknown Source)
                        Exiting due to uncaught Exception null
                        

                        Does this work as designed?
                        M.

                         
                        • Gil Barmwater

                          Gil Barmwater - 2021-08-28

                          Glad you have had success w/ NetRexx Pipelines! As to your recent discovery - note my earlier comment regarding the difference between variables in NetRexx and ooRexx. So, even though you ran the 'pipe' command from within ooRexx, the ooRexx interpreter created a new process to run Java which ran the .runner program interpreting the pipeline. That process has no access to the ooRexx process or its variable pool so the VAR stage cannot "see" the ooRexx variable. Instead it tries to find a NetRexx/Java variable and, since it doesn't exist, apparently returns the null string. It seems to me that it probably should return the upper-cased value of the variable name but it doesn't. Now the error caused when the output of LITERAL with no value is passed to COUNT is something the NetRexx guys will need to investigate.

                           
                          • m-stgt

                            m-stgt - 2021-09-07

                            Sorry for my delayed response, missed its email alert.
                            If NetRexx-Pipelines has only access to NetRexx variables, allow me an uneducated question: how about using NetRexx as primary Rexx to have full use of its Pipeline feature, and take ooDialog as an add-on to it? I dare to ask as I have a program based on ooRexx/ooDialog using Rexx/gd, what apparently is not part of the "oo-suite". And it works (quite well).

                             
  • m-stgt

    m-stgt - 2021-08-20

    Note: NetRexx 4.01-GA QuickStart Guide.pdf, chapter3.3 "First steps with NetRexx", p. 21, step 2: do not edit/copy-edit/paste the proposed program line from the PDF since the quotation marks are not those NetRexx accepts.

     
    • Gil Barmwater

      Gil Barmwater - 2021-08-20

      Yes, it appears that the line in the PDF uses back quotes. I simply followed the recommendation in the following sentence and copied the file so did not run into that issue.

      BTW, congratulations on getting all the way to step 7!

       
  • Rene Vincent Jansen

    I have read and will address all the document remarks. On CLASSPATH you only need the NetRexxF.jar file (the file and its path). As opposed to PATH, the jar file must be mentioned there. There are ways to specify a directory there, but putting the path to the .jar, including the .jar itself, is simplest and the most safe approach. After that, shortcuts, aliases and/or batchfiles can be used to start the java classes.

     
    • Gil Barmwater

      Gil Barmwater - 2021-08-21

      Hi René! I continue to play with my experiment and have now modified the nrxstart.cmd batch file to also set classpath. This allows me to use "address path java ..." in my nrxpipe.rex program which eliminates the overhead of cmd.exe and the pipe.bat batch file. NB The 'path' option on address is new on ooRexx 5.0.0.

       

      Last edit: Gil Barmwater 2021-08-21

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.