Menu
▾
▴
4 Attachments
#882 ADDRESS environments written in Rexx
ADDRESS is one of the most characteristic things in Rexx, but today a
command environment can only be written in native code: classic subcommand
handlers (RexxRegisterSubcomExe) or, since 5.0, AddCommandEnvironment.
Rexx code can send commands to an environment, but it cannot be one. This
RFE proposes that it can: a Rexx object as a command environment, with the
same powers as a native one (ADDRESS ... WITH redirection, as Rexx
streams; access to the variables of the code that issued the command;
raising conditions in it).
Where it should live
This is a question for the RFE, not something it settles. The
implementation attached is an extension package, cmdhandler
(::requires "cmdhandler.cls"), because it can be: it uses only the public
native API (AddCommandEnvironment and the I/O redirector context), and
nothing in the interpreter changes. That makes it a working reference, and
shows that the API is enough. But the feature itself looks like it belongs
to the language, and in the interpreter it would be simpler and more
complete: the command would go straight to the Rexx object, with no native
handler in between, and removing an environment would just work (see the
limitation at the end). Whether .CommandHandler should be a core class,
or stay an extension, is for us to decide.
The extension has been in use since early October in the WebAssembly port
(RFE #879), where the ADDRESS environments for the browser are written
with it (a first version, with a simpler API). It became robust with the
fixes for #2111 (r13274) and #2112 (r13275): registering an environment
while other threads run Rexx, and resolving ADDRESS names, were not safe
before.
Example
.CommandHandler~register("UPPER", .Upper~new)
in.1 = "hello"; in.2 = "world"; in.0 = 2
address upper "" with input stem in. output stem out.
say "rc="rc "out.0="out.0 out.1 out.2 -- rc=0 out.0=2 HELLO WORLD
address upper "COUNT"
address upper "COUNT"
say "rc="rc "count="count -- rc=0 count=2
address upper "" -- no WITH: ERROR, RC 1
say "rc="rc -- rc=1
call wrong
.CommandHandler~deregister("UPPER") -- "UPPER" is now an unknown environment
exit
wrong:
signal on syntax
address upper "FROB"
return
syntax:
c = condition("O")
say "SYNTAX" c~code "at line" c~position":" c~message
-- SYNTAX 98.900 at line 21: UPPER: unknown command FROB.
return
::requires "cmdhandler.cls"
::class Upper inherit CommandHandler
::method command
use arg command, caller
select
when command = "" then do
if \caller~isRedirected("Input") | \caller~isRedirected("Output") then
return caller~raiseError(1)
in = caller~input; out = caller~output
do while in~lines > 0
out~lineout(in~linein~upper)
end
return 0
end
when command = "COUNT" then do
if caller~hasVar("count") then n = caller["count"] -- the caller's
else n = 0
caller["count"] = n + 1
return 0
end
otherwise
return caller~raise("SYNTAX", 98.900, .array~of("UPPER: unknown command" command))
end
The CommandHandler class
A mixin class. A command environment is an object of a class that
inherits it.
register(name, handler) (class method) makes handler the
environment name (not case-sensitive) of the current interpreter
instance, replacing any previous one. handler must be an instance of a
class that inherits .CommandHandler; .nil or any other object is
SYNTAX 93.900.
deregister(name) (class method) removes it: name then behaves as an
unknown environment (FAILURE, RC 30). A name with no handler is SYNTAX
93.900.
command(command, caller) is abstract: the handler's class
implements it. It is called once for every command sent to the
environment, on the thread that issued it, with:
- command: the command, a String (the value of the expression after
ADDRESS name, or of the command clause); - caller: a
.CommandContextfor this command (below): the code that
issued it, as the handler sees it.
What it returns is the command's RC (0 if it returns nothing). Any value is
allowed; its sign raises nothing. To raise a condition, return
caller~raise(...) (below). A SYNTAX error inside command (a bug in the
handler, or a RAISE SYNTAX) reaches the code that issued the command,
with the handler's frames in the traceback.
The CommandContext class
Valid only while its command runs, and only on that thread (afterwards, or
from another thread, its methods and its streams raise SYNTAX 98.900;
isValid tells).
| Method | |
|---|---|
input |
the WITH INPUT source, as an .InputStream: linein, lines, charin, chars, arrayIn; at the end, linein returns "" and raises NOTREADY, as a stream does. Without WITH INPUT: SYNTAX 98.900 |
output, error |
the WITH OUTPUT / WITH ERROR targets, as .OutputStreams: lineout, charout, arrayOut. Without the WITH: SYNTAX 98.900 |
isRedirected(which) |
"Input", "Output", "Error", or "Same" (output and error to the same target) |
[name], [name] = value |
a variable of the code that issued the command: simple, stem or compound (the tail resolved there, as in Rexx); names are not case-sensitive. A variable whose value is .nil gives .nil. With no value, its name, as Rexx gives it (FOO; A.x y when i = "x y"); hasVar tells. Not a variable name: SYNTAX 93.900 |
getVar(name), setVar(name, value) |
exact synonyms of [name] and [name] = value |
hasVar(name), dropVar(name) |
whether it has a value; drops it, as DROP |
variables |
a Directory with a copy of all of them |
raise(condition, rc [, description] [, additional]) |
the command raises condition (any that RAISE accepts but SYNTAX: ERROR, FAILURE, HALT, LOSTDIGITS, NOMETHOD, NOSTRING, NOTREADY, NOVALUE, USER name) in the caller, with RC = rc, when command returns; description defaults to the command; returns rc. Only one per command: a second call is SYNTAX 93.900 (the native RaiseCondition keeps the last one; here two are taken for a mistake, and allowing them later would be easier than forbidding them) |
raise("SYNTAX", code [, additional]) |
when command returns, the caller gets SYNTAX code (major.minor) at its ADDRESS clause, without the handler's frames: "the command you sent is wrong", as a built-in function's argument error. additional is the Array of substitutions for the message. The result of command is ignored |
raiseError(rc), raiseFailure(rc) |
raise("ERROR", rc), raise("FAILURE", rc) |
environment |
the ADDRESS name the command was sent to |
isValid |
whether the command is still running |
The streams exist only with redirection: without it, the handler already
knows where to read and write. Code that wants both cases is plain Rexx,
since .input~current and .output~current follow the same protocol:
if caller~isRedirected("Input") then in = caller~input
else in = .input~current
The streams are not .input and .output redirected: those monitors
belong to the interpreter instance, not to the thread, so concurrent
commands and other threads would mix.
Registrations are per interpreter instance, like the environments
themselves.
Implementation
extensions/cmdhandler: cmdhandler.cpp (one native redirecting handler
serves every name registered from Rexx) and cmdhandler.cls; built and
installed like the other extensions (libcmdhandler, cmdhandler.cls).
The attached patch applies to trunk r13283 and touches nothing else but
CMakeLists.txt.
Tests (trunk r13283, Ubuntu 24.04 x86_64):
extensions/cmdhandler/tests/cmdhandler.rex, 65 checks, all pass; two
interpreter instances in one process (instances.cpp), ok; the full test
suite shows no other changes. If this is accepted, the tests would become a
.testGroup in the official suite, and the reference below a chapter of
the documentation.
raise relies on #2119, fixed in r13282: before it, under
::OPTIONS ERROR SYNTAX, a USER condition raised this way became SYNTAX
98.970 even when trapped.
One limitation, which comes from being an extension: there is no API to
remove an environment added with AddCommandEnvironment, so deregister
keeps the native handler and makes it answer RC 30. A Rexx program cannot
tell the difference, except that a classic subcommand handler registered
under the same name stays hidden. In the interpreter this would not arise;
as an extension, it would need a RemoveCommandEnvironment API (a separate
RFE).
Related
Discussion
Anonymous
+1
Because of its importance as a fundamental enhancement to the language it should be part of the language!
One more request: if possible add a class method to the RexxCommand class that returns a StringTable of currently registered command handlers using the name as index, and in the case of a Rexx command handler, the command handler object, otherwise a string like "NATIVE".
As discussed on oorexx-devel, here is a version that lives in the interpreter instead of being an extension. Two patches are attached, both for r13285 (
patch -p0):cmdhandler-in-interpreter-r13285.patch(main/trunk)cmdhandler-in-interpreter-tests-r13285.patch(test/trunk)The Rexx API is the same as the extension's:
.CommandHandler(register,deregister,command) and.CommandContext. No::requiresis needed. A program that still requirescmdhandler.clskeeps working, because the classes in its own package take precedence (its 65 tests pass on top of this version).What changes inside:
CommandHandler, a Rexx object. The command is sent straight to it, on the same activity, with no native handler and no native frames in between.CommandIOContextthatADDRESS ... WITHalready builds.caller~raise(...)hands its condition back toRexxActivation::command, exactly as a native handler does.deregisterreally removes the environment. The limitation described in the RFE is gone: a classic subcommand handler registered under the same name becomes visible again..CommandHandler~environments(Rony's request) returns a StringTable from the environment name to the handler object, or"NATIVE"for the others. One caveat: a classic subcommand handler registered through rxapi enters the instance's table only when its name is first used, and the API cannot list the registry, so it shows up from then on.caller~raise("HALT", ...), or byRAISE HALTincommand, each one underSIGNAL ON HALT,CALL ON HALT, or untrapped (SYNTAX 4.1). This needs no special handling, because there is no native boundary to cross.The classes live in a new
CommandClasses.orx, loaded byCoreClasses.orxright afterStreamClasses.orx, because the context's streams inherit.InputStreamand.OutputStream.Tests: the new
ooRexx/base/class/CommandHandler.testGrouphas 61 tests and 116 assertions. It covers the extension's checks plus recursion, concurrency, the nine HALT cases,environmentsand deregistration.MethodArgs.testGroupgains one line: the three context classes are created only by the interpreter, like.RexxContextand.StackFrame, so they join that test's skip list. Full suite on r13285 (Ubuntu 24.04 x86_64): 24,165 tests, exactly the same failures with and without the patch (the usual ones when running as root).Documentation (a reference section for rexxref) will follow once the interface is settled.
Just tested the original "cmdhandler.cls" and "cmdhandler.cpp" on Windows 10 (64-bit). There was a small problem in building the .dll from the .cpp as it had some extraneous HTML at the end(!) but when I removed it all was well. I created two cmdhandlers similar to those I presented at the Vienna Symposium - RESULT! and ECHO!. The entire package that made them both available consisted of: two lines to register both environments, one ::requires for cmdhandler.cls, and two class definitions - one for each handler - that each had only 4 lines (the ::class directive, the ::method directive, a use arg ... statement and the line that did the work). Amazingly simple to use! And the built-in version will be just as easy I'm sure. Thanks for this great addition to ooRexx!
(Anyone on Windows 64-bit that would like to play with this but isn't comfortable building a .dll, please contact me and I'm happy to share it.)
Glad that you like it. I'm pretty sure this will be used a lot.
Developing the WASM port and the Lua, .Net and Python bridges implies running really strong torture tests. It will take some time to fix all the bugs, but we will get a nice hardening of the interpreter :)
Version 2, after Rony's comments on oorexx-devel. Two patches, both for r13285 (
patch -p0); they replace the ones of 22/10:cmdhandler-in-interpreter-v2-r13285.patch(main/trunk)cmdhandler-in-interpreter-tests-v2-r13285.patch(test/trunk)What changes from version 1:
.CommandHandler~register(name, handler)covers the handler in use under that name, written in Rexx or native (a classic subcommand handler not used yet is looked up first, so it ends up below too), and returns its object, or .nil.deregister(name)removes the handler in use, written in Rexx or native, uncovers the one below and returns the object removed; with nothing left the name is an unknown environment (FAILURE, RC 30)..CommandHandler~handlers(name)returns the stack, the handler in use first;.CommandHandler~handler(name)the one in use..NativeCommandHandlerobjects (made only by the interpreter:newis private), with~name(the name they were registered under) and~kind(SUBCOM, DIRECT or REDIRECTING); the same native handler is always the same object..CommandHandler~environmentsmaps every name to an object; there is no"NATIVE"string any more.nativeHandler~command(command, caller)sends the command to the native handler as if the code that issued it had: the native handler gets that code's activation as its caller (variables through RexxVariablePool and the exit context), the sameADDRESS ... WITHredirection (already set up, shared with the Rexx handler), and its own registered name (the Unix handler runs the shell of that name). Its RC and condition become the command's. Measured with the system handler, a classic subcommand handler, and DIRECT and REDIRECTING handlers registered by a host application: same results with and without a Rexx handler forwarding in between, including SYNTAX and HALT, trapped or not.Inside:
CommandHandlerhas apreviousfield (the handler it covers);Activityhas acallbackParentused byActivity::run(CallbackDispatcher &)for the forwarded call.Tests:
CommandHandler.testGrouphas 68 tests (150 assertions): stacking, what register and deregister return,handlers, identity, forwarding to a Rexx handler and to the default system environment, deregistering and re-registering a native environment, the privatenew.MethodArgs.testGroupskips.NativeCommandHandlertoo. Full suite on r13285 (Ubuntu 24.04 x86_64): 24,233 tests, exactly the same failures with and without the patch (the usual ones when running as root).If nobody objects, I'll write the docs and implement that tomorrow, and then I'll announce it to the members list.
Hi Joseph Maria,
This is NOT an objection. At the moment there's a lot of innovation going
on, and I read about it on the lists, and mainly I feel a lack of the
context I need to truly understand what the development means for a user.
I'm sure you will do a brilliant job, but can I request that the docs are
not just reference, but also user guide, perhaps with a simple example of a
command handler written in oorexx please.
Thanks,
Jon
On Sat, 24 Oct 2026, 09:00 Josep Maria Blasco, jmblasc2@users.sourceforge.net wrote:
Related
Feature Requests:
#882Thanks, Jon. Yes, I will add examples in rexxpg, apart from updating rexxref. Here's a simple example, of an extension that can be used to detect unwanted commands:
and here's the tested output:
Last edit: Josep Maria Blasco 2 days ago
I've attached drafts of the documentation for the version in the interpreter
(v2): rexxref-cmdhandler.patch adds the five new classes (CommandHandler,
NativeCommandHandler, CommandContext, CommandInput and CommandOutput) to the
Utility Classes chapter, with their railroad diagrams, plus a short paragraph
in ADDRESS; rexxpg-cmdhandler.patch adds a section "Command Environments
Written in Rexx" at the end of the Commands chapter, as a user guide, with
five small examples: an ECHO environment, the "no commands" one above, a
key-value store that reads and sets the caller's variables, a word count
using WITH INPUT/OUTPUT, and a "safe" wrapper around the system environment
that refuses some commands and passes the rest on. All the examples were run
against the patched interpreter, and the PDFs of both books are attached too.
Comments welcome.
Last edit: Josep Maria Blasco 2 days ago
I've had a good look at the proposed documentation and it is excellent IMHO. Just another thought on a use for the Echo environment: if you are debugging a program and need to examine the values of a lot of variables as you are tracing, Echo can be helpful. If you register it when you start tracing (or before - it doesn't hurt to have it in your program if you don't use it), then when you get to the point where you want to see the values of a bunch of variables, enter "address echo" and then instead of typing "say var1", just type "var1". If you have to look at lots of variables, this will save you 4 keystrokes for each one.
Thanks, Gil. Ah, nice trick, it reminds me of the "expression=" syntax used by oorexxshell.
Yes, indeed! In fact, if you slightly modify the ECHO command method so that it "say's"
'['command']'- the variable wrapped in square brackets - you can distinguish variables that are blank or the empty string as in Rony's pp() function.Last edit: Gil Barmwater 2 days ago
I've taken this idea and expanded it a bit. The command environment is now called SHOW. In addition to the capability noted above to display variables (or expresions e.g. myArray~items) enclosed in brackets, it now 1) permits one to change the "wrapper" -
'[<>]'when SHOW is the address environment would set (and remember) that "commands" are to be enclosed in '<' and '>' instead of '[' and ']'. The "command"'[]'would turn off "wrapping". And 2) the "command" SHOWVARS will display all of the caller's variables along with their values. I've attached a file that implements the SHOW environment and registers it so one couldcall debugShowwhile tracing to make it available followed byaddress showto make it the current address environment. Might be a candidate for a sample program(?).Thanks, Gil! You made me think that we need some examples in the samples/ directory.. This is an important feature, it deserves to have some samples. I'll work on this later today.
Committed in revisions [r13288] (interpreter), [r13290] (documentation) and [r13291] (tests); [r13289] sets svn:eol-style on the new CommandClasses.orx.
Related
Commit: [r13288]
Commit: [r13289]
Commit: [r13290]
Commit: [r13291]
Some samples to put in samples/. Comments, anyone?
Very nice! Seeing the source code and the output is very informative, and a Rexx programmer can understand, what happens, even if they have not been exposed to ooRexx yet (although it would be a little challenge for them).
Question ad WordCount: why is output redirection forced?
Question ad Store: why using stems only for keys, and not arrays ("keys array") as well?
The only thing I would change is the order of the demonstrated command handlers to:
Thanks, Rony! Good points, all three. New versions attached:
WordCount: no good reason. It now does what Sorter does: it uses WITH
INPUT and WITH OUTPUT when they are there, and the default input and
output streams when they are not. Its RC is now the number of words.
The demo shows it both ways.
Store: KEYS can now give either. "KEYS name." (with a period) fills a
stem, as before; "KEYS name" (no period) sets the variable to an Array
of the keys. Plain "KEYS" is still KEYS KEYS.; the demo shows both.
The order: done, yours, in the class file and in the program: Echo,
WordCount, Sorter, Show, NoCommands, Safe, Store.
+1
Added samples with [r13292] and [r13293].
Related
Commit: [r13292]
Commit: [r13293]
Last edit: Josep Maria Blasco 1 day ago
Just a heads-up re. the samples: the Programmer Guide lists all of the sample programs in Appendix B with a brief description. I didn't see that listed as being updated.
Thanks for spotting this! Working on it.
Fixed with [r13296].
Related
Commit: [r13296]