Menu
▾
▴
1 Attachments
#1889 Prolog code of a required package on a different Rexx instance run twice (Required required package does not get prolog code run on additional Rexx instances
Executing Rexx programs on different Rexx instances that require packages does not correctly work, if a required package requires an additional package that was already required on a different Rexx instance as the second required package's prolog code does not get exectued.
ooRexx maintains required packages on a Rexx instance base. If within a Rexx instance different Rexx programs require the same package, then all its public classes and public routines get immediately re-used if the package got resolved (with call semantics) already.
This reported bug surfaces when in e.g. R1 (Rexx instance #1) a package P1 gets required that itself requires a package P2, and if concurrently on R2 (Rexx instance #2) the same package P1 gets later required. P1 requires P2 on R2 but unlike in R1 the prolog code in P2 does not get executed!
If one makes sure that R1 and R2 get terminated and one runs the same test program afterwards, then P2's prolog code gets executed again.
The enclosed zip-file contains the following files:
G:\oorexx.tmp\requiresRequiresBug>unzip -v requiresRequiresBug.zip
Archive: requiresRequiresBug.zip
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
12183 Defl:X 3392 72% 09.04.2023 14:43 841e2488 runRexxProgram.cpp
5099 Defl:X 1878 63% 09.04.2023 14:55 16948e0f Makefile.windows
114688 Defl:X 61277 47% 09.04.2023 14:54 ede8666b runRexxProgram.exe
339 Defl:X 174 49% 09.04.2023 15:13 f6225b3b main_ok.rex
549 Defl:X 215 61% 09.04.2023 15:14 34b0f6a0 main_not_ok.rex
322 Defl:X 158 51% 09.04.2023 15:10 fae61ddd useful_stuff.cls
361 Defl:X 184 49% 09.04.2023 15:15 81978320 useful_stuff_indirect.cls
-------- ------- --- -------
133541 67278 50% 7 files
runRexxProgram.exe is a 64-bit compilation and runs any supplied Rexx program in the following manner:
- R1 gets created and the supplied program executes on it, R2 gets concurrently created and the supplied program executes on it,
- R1 and R2 get terminated and afterwards R3 gets created and the supplied program executes on it.
main_ok.rex: this program requires "useful_stuff.cls" directly, all interpreter instances resolve this package and run the prolog code.
main_not_ok.rex: this program requires "useful_stuff_indirect.cls" which itself requires "useful_stuff.cls". R1 will run the prolog code for "useful_stuff.cls" and "useful_stuff_indirect.cls", whereas R2 will only run the prolog code for "useful_stuff_indirect.cls", not the prolog code for "useful_stuff.cls" (its public routines and public classes are made available).
As the prolog code in Rexx packages is used to initialize the package, not running the prolog code causes (at first sight unexplainable) runtime errors.
After terminating R1 and R2 running the Rexx program on R3 will again have the prolog code of "userful_stuff.cls" executed.
Here the output of running "runRexxProgram.exe main_not_ok.rex":
G:\oorexx.tmp\requiresRequiresBug>runRexxProgram.exe main_not_ok.rex
Created interpreter instance version=5.1.0 language level=6.05
Created interpreter instance version=5.1.0 language level=6.05
---------------------------------------------------
Using interpreter1 to execute main_not_ok.rex
useful_stuff.cls # 1: source=<<WindowsNT REQUIRES G:\oorexx.tmp\requiresRequiresBug\useful_stuff.cls>>
<<PP2>> # 5: source=<<WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\useful_stuff.cls>>
useful_stuff.cls # 2: testing pp2(): <<WindowsNT REQUIRES G:\oorexx.tmp\requiresRequiresBug\useful_stuff.cls>>
useful_stuff_indirect.cls # 1: source=<<WindowsNT REQUIRES G:\oorexx.tmp\requiresRequiresBug\useful_stuff_indirect.cls>>
<<<PP3>>> # 7: source=<<<WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\useful_stuff_indirect.cls>>>
useful_stuff_indirect.cls # 2: testing pp3(): <<<WindowsNT REQUIRES G:\oorexx.tmp\requiresRequiresBug\useful_stuff_indirect.cls>>>
main_not_ok.rex # 1: source=[WindowsNT COMMAND G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex]
[PP] # 9: source=[WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex}
main_not_ok.rex # 2: testing pp(): [WindowsNT COMMAND G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex]
<<PP2>> # 5: source=<<WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\useful_stuff.cls>>
main_not_ok.rex # 3: testing pp2(): <<WindowsNT COMMAND G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex>>
<<<PP3>>> # 7: source=<<<WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\useful_stuff_indirect.cls>>>
main_not_ok.rex # 4: testing pp3(): <<<WindowsNT COMMAND G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex>>>
---------------------------------------------------
Using interpreter2 to execute main_not_ok.rex
useful_stuff_indirect.cls # 1: source=<<WindowsNT REQUIRES G:\oorexx.tmp\requiresRequiresBug\useful_stuff_indirect.cls>>
<<<PP3>>> # 7: source=<<<WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\useful_stuff_indirect.cls>>>
useful_stuff_indirect.cls # 2: testing pp3(): <<<WindowsNT REQUIRES G:\oorexx.tmp\requiresRequiresBug\useful_stuff_indirect.cls>>>
main_not_ok.rex # 1: source=[WindowsNT COMMAND G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex]
[PP] # 9: source=[WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex}
main_not_ok.rex # 2: testing pp(): [WindowsNT COMMAND G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex]
<<PP2>> # 5: source=<<WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\useful_stuff.cls>>
main_not_ok.rex # 3: testing pp2(): <<WindowsNT COMMAND G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex>>
<<<PP3>>> # 7: source=<<<WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\useful_stuff_indirect.cls>>>
main_not_ok.rex # 4: testing pp3(): <<<WindowsNT COMMAND G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex>>>
Terminating interpreter1
Terminating interpreter2
Created interpreter instance version=5.1.0 language level=6.05
---------------------------------------------------
Using interpreter3 to execute main_not_ok.rex
useful_stuff.cls # 1: source=<<WindowsNT REQUIRES G:\oorexx.tmp\requiresRequiresBug\useful_stuff.cls>>
<<PP2>> # 5: source=<<WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\useful_stuff.cls>>
useful_stuff.cls # 2: testing pp2(): <<WindowsNT REQUIRES G:\oorexx.tmp\requiresRequiresBug\useful_stuff.cls>>
useful_stuff_indirect.cls # 1: source=<<WindowsNT REQUIRES G:\oorexx.tmp\requiresRequiresBug\useful_stuff_indirect.cls>>
<<<PP3>>> # 7: source=<<<WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\useful_stuff_indirect.cls>>>
useful_stuff_indirect.cls # 2: testing pp3(): <<<WindowsNT REQUIRES G:\oorexx.tmp\requiresRequiresBug\useful_stuff_indirect.cls>>>
main_not_ok.rex # 1: source=[WindowsNT COMMAND G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex]
[PP] # 9: source=[WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex}
main_not_ok.rex # 2: testing pp(): [WindowsNT COMMAND G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex]
<<PP2>> # 5: source=<<WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\useful_stuff.cls>>
main_not_ok.rex # 3: testing pp2(): <<WindowsNT COMMAND G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex>>
<<<PP3>>> # 7: source=<<<WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\useful_stuff_indirect.cls>>>
main_not_ok.rex # 4: testing pp3(): <<<WindowsNT COMMAND G:\oorexx.tmp\requiresRequiresBug\main_not_ok.rex>>>
Terminating interpreter3
As one can see the prolog output from "useful_stuff.cls" in interpreter2 (R2) above is not shown, because the prolog code of "useful_stuff.cls" does not get executed! The following lines are therefore missing from R2's execution:
useful_stuff.cls # 1: source=<<WindowsNT REQUIRES G:\oorexx.tmp\requiresRequiresBug\useful_stuff.cls>>
<<PP2>> # 5: source=<<WindowsNT FUNCTION G:\oorexx.tmp\requiresRequiresBug\useful_stuff.cls>>
useful_stuff.cls # 2: testing pp2(): <<WindowsNT REQUIRES G:\oorexx.tmp\requiresRequiresBug\useful_stuff.cls>>
As the Rexx programs are short, here their contents for direct inspection (they are contained in the attached zip-archive):
"main_not_ok.rex":
G:\oorexx.tmp\requiresRequiresBug>type main_not_ok.rex
parse source s; say filespec('n',.context~name)~left(25) "#" .line":" "source=["s"]"
say filespec('n',.context~name)~left(25) "#" .line":" "testing pp():" pp(s)
say filespec('n',.context~name)~left(25) "#" .line":" "testing pp2():" pp2(s)
say filespec('n',.context~name)~left(25) "#" .line":" "testing pp3():" pp3(s)
::requires "useful_stuff_indirect.cls" -- this will then require "useful_stuff.cls"
::routine pp
parse source s; say ("["filespec('n',.context~name)"]")~left(25) "#" .line":" "source="||"["s"}"
return "["arg(1)"]"
"useful_stuff_indirect.cls" (which requires "useful_stuff.cls"):
G:\oorexx.tmp\requiresRequiresBug>type useful_stuff_indirect.cls
parse source s; say filespec('n',.context~name)~left(25) "#" .line":" "source=<<"s">>"
say filespec('n',.context~name)~left(25) "#" .line":" "testing pp3():" pp3(s)
::requires "useful_stuff.cls"
::routine pp3 public
parse source s; say ("<<<"filespec('n',.context~name)">>>")~left(25) "#" .line":" "source="||"<<<"s">>>"
return "<<<"arg(1)">>>"
"useful_stuff.cls":
G:\oorexx.tmp\requiresRequiresBug>type useful_stuff.cls
parse source s; say filespec('n',.context~name)~left(25) "#" .line":" "source=<<"s">>"
say filespec('n',.context~name)~left(25) "#" .line":" "testing pp2():" pp2(s)
::routine pp2 public
parse source s; say ("<<"filespec('n',.context~name)">>")~left(25) "#" .line":" "source="||"<<"s">>"
return "<<"arg(1)">>"
Related
Discussion
Anonymous
Required packages are NOT loaded and maintained on an instance bases, they are managed globally and the prologs are only ever executed once.
So, why is it then that the prolog code of P1 gets executed on R1 and R2, but the prolog code of P2 does not get executed on R2?
Prpbably a race condition in the loading process that results in it getting
run twice. That's a bug, not the other way.
Rick
On Sun, Apr 9, 2023 at 10:12 AM Rony G. Flatscher orexx@users.sourceforge.net wrote:
Related
Bugs: #1889
Adjusting the title of this bug to reflect Rick's comment.
It’s the “otherwise” part :
InterpreterInstance::loadRequires(package)
if the interpreter instance has already the package in its cache (with short or full name) then return immediatly.
otherwise
ask the package manager to load it
run the prolog (always, even if the package was already in the static cache of PackageManager)
If this is modified to no longer run the prolog, then I think BSF will no longer be initialised correctly when running several interpreters.
Related
Bugs: #1889
Indeed, currently if using an intermediate package that requires BSF.CLS then on additional interpreter instances errors occur. Not sure exactly why this is the case as I tried to remove any dependency on .local a couple of years ago.
There is currently also another unexplainable observation in this particular use case when defining a debug class in BSF.CLS like :
The above debug_class is meant to produce debug output when this (in this use case indirectly required) version of BSF.CLS gets processed: upon return of running this version of BSF.CLS (indirectly via another requires) currently an error gets raised, commenting as seen in the above snippet "fixes" the raising of this error condition.
It may be the case that once the resolution of required packages gets fixed correctly, that it then will work, speculating ("black-box analyisis") that some side effects are caused by the current behavior. (Also speculating and hoping that the security manager currently not getting triggered will trigger requires directives again, cf. [bugs:#1886].)
Therefore I would appreciate it, if this particular bug could get fixed ASAP in order to become able to debug BSF4ooRexx850 with it.
Related
Bugs:
#1886O.K., the "DEBUG_CLASS" and the "unexplainable" runtime error occurring is not ooRexx fault but introduced with the position where it got created in BSF.CLS for debugging purposes. It should have been defined after the definition of floating methods.
(Background: there is a floating method defined in BSF.CLS which causes .methods to be created which later gets used to decorate imported Java class objects. Introducing a class before that intended floating method makes it instead become part of the .debug_class hence .methods will not get created by ooRexx, hence referring to entries in .methods cause the - misleading - error "Error 93 running G:\test\orx\require\fxml_99\indirect\BSF_INDIRECT.CLS line 21: Incorrect call to method.
Error 93.924: Invalid position argument specified; found "NEWSTRICT".", an error reported by the string class as in this case the environment symbol will be returned as an uppercased string ".METHODS". The reported line number 21 is the line in which the requires to BSF.CLS is contained.)
Ad problem with BSF4ooRexx if indirectly requiring BSF.CLS (i.e. not running prolog code on additional Rexx interpreter instances). Although all dependencies to .local got removed a couple of years ago, there was a subtle problem caused by a missing configuration call to the Rexx instance's Java peer because the prolog code is not executed (call bsf 'bsfPrefixReturnValue',1). Solution will put that call into a proper public routine in BSF.CLS which gets called by RexxScriptEngine upon creating a new Rexx instance implicitly.
There are scenarii where not executing the prolog code in required Rexx programs in different interpreter instances may cause runtime problems and may not be solvable by Rexx programmers.
E.g. writing a Rexx package that gets required in different Rexx interpreter instances and which need state data or configuration changes in each of these instances (either in the respective .local or in peers in native code).
If it was possible to define some "::OPTIONS FORCEPROLOG" causing the prolog code to be run in new Rexx interpreters even if that package has already been required in other Rexx instances.
This would be the prolog related settings:
1. PROLOG: execute prolog code on first requires (default behavior)
2. NOPROLOG: do not execute prolog at all (needs to be explicitly run: .context~package~prolog~call
3. FORCEPROLOG: always run when first required in a new Rexx instance
Would that be acceptable for everyone?
Without doing a lot of research, and I generally dislike comments/suggestions made without doing the appropriate "homework", let me offer my take on the situation.
Lets assume there is a program, lets call it pgmA, that needs to run on multiple Rexx interpreter instances. pgmA needs another program, call it pgmB, and so pgmA ::REQUIRES pgmB. However, the prolog code of pgmB must run on every interpreter instance. From the above comments by Rick, this will NOT be the case. Assuming I have the situation stated correctly why not do the following? Add a ::OPTIONS NOPROLOG to pgmB so the prolog code is never executed automatically. Then at the very beginning of pgmA add something like:
do aPkg over .context~package~importedPackagesif aPkg~name = "pgmB" then doaPkg~prolog~callleaveendendNote that this is untested code and it may require modifications in order to do what it is supposed to do. One drawback that I see already is if the prolog code of pgmB must run before another ::REQUIRES is processed in pgmA, then this solution will not work.
FWIW, YMMV.
Gil
Last edit: Gil Barmwater 2023-04-19
Well, you assume too much! :)
If one writes an ooRexx package (utility package, bridge package, etc.) xyz.pkg then anyone wanting to access routines, classes from those packages should use "::requires xyz.pkg". Such a package would be used by programmers who have not created it, nor having a will to know anything about its implementation, just requiring xyz.pkg should make all documented functionality available.
If a package writer places Rexx instance related information into .local (specific for each instance) or has a need to configure a peer program dependent on the Rexx instance (e.g. if using BSF.CLS then the Java peer needs to be configured to prepend string values with an indicator which is expected by the routines and classes of BSF.CLS) then it is expected that these setup steps get carried out in its prolog code as that is its purpose.
Currently the prolog code when doing a "::requires xyz.pkg" in the very first Rexx program gets executed in new Rexx interpreter instances, but not the prolog code of required packages, which is a bug. The current design, if understanding Rick correctly, does globally cache such required packages and only runs the prolog code the very first time that package got required.
In the case of BSF4ooRexx it is possible to initialize the Java peer only, if the Java scripting framework gets used to create a Rexx instance (had to change the logic to do so). However, if a Rexx instance gets created using BSF (the Apache Bean scripting framework) then the programmer is free to not use BSF.CLS at all! If however one of the Rexx programs the programmer executes via BSF requires BSF.CLS then there is no way to determine this fact as Rexx does not run the prolog code in the case that BSF.CLS got required somewhere else by a different Rexx instance!
Again, this is one specific use case but there are other use cases conceivable (and then to be expected). In order to prevent the Rexx programmer from making (unconscious) errors because he is not aware of the need to initialize a required package if that required package gets required on a different Rexx instance I propose to allow the package author to define an "::OPTIONS FORCEPROLOG" to have the Rexx interpreter to run the prolog code of such packages when required in a new Rexx instance. This way a Rexx package having a need to initialize itself for whatever reasons for each Rexx instance can do so. A Rexx programmer taking advantage of such a package does not need to know anything about the implementation (and therefore the need to initialize for some packages) and hence cannot cause errors because he was not aware of such an implementation detail.