[Oorexx-devel] Questions, RFI on multithreading

[Rony G. F. <Ron...@wu...>]

Hi there,

trying to get BSF4Rexx 4.0 into beta-test shape (32-bit and 64-bit
Windows and Linux versions). Using developed test units I have been able
to assert that it runs stable, with the exception of employing
multithreading heavily.

"32-bit" means: operating system is 32-bit, hence ooRexx and BSF4Rexx.
Windows is XP SP3, Linux is Ubuntu.
"64-bit" means: operating system is 64-bit, ooRexx is 64-bit and
BSF4Rexx is 64-bit  Windows is Vista Business, Linux is Ubuntu.

Multithreading in this context means:

    * using multiple Rexx threads, each interfacing with Java using
      BSF4Rexx (e.g. 100 Rexx threads per test, in each thread two
      invocations of BSF4Rexx),
    * using multiple Java threads, which each uses the same Rexx
      interpreter instance to run off Rexx scripts that themselves
      dispatch multiple Rexx threads, all (Java and Rexx code)
      interfacing with each other using BSF4Rexx (e.g. 23 Java threads,
      each running a Rexx script that creates 29 Rexx threads; summing
      up all threads gives 712 threads, a heavy version may use a total
      of 2,601 threads),
    * there are individual Java objects that Rexx interacts with via
      BSF4Rexx,
    * there are individual Rexx objects that Java interacts with via
      BSF4Rexx; access to these Rexx objects may be concurrently from
      Rexx threads and Java threads, where the latter get routed via
      BSF4Rexx.


Here are three issues where I would to ask for advice, ideas:

    * Scenario 1: 32-bit Windows (not tested on 64-bit Windows),
      exception in "rexx.exe":
          o Here I may get the following exception
          o

            >	rexx.dll!00337328() 	
             	[Frames below may be incorrect and/or missing, no symbols loaded for rexx.dll]	
             	rexx.dll!002ff218() 	
             	rexx.dll!002e9490() 	
             	rexx.dll!00303462() 	
             	rexx.dll!003473c9() 	
             	kernel32.dll!7c80b729() 

            Unhandled exception at 0x00337328 in rexx.exe: 0xC0000005: Access violation reading location 0x0000000c.


            or:

            >	rexx.dll!002e86f8() 	
             	[Frames below may be incorrect and/or missing, no symbols loaded for rexx.dll]	
             	rexx.dll!002e880b() 	
             	rexx.dll!002e9438() 	
             	rexx.dll!0030d2ae() 	
             	rexx.dll!00318639() 	
             	BSF4Rexx.dll!00c44269() 	

            Unhandled exception at 0x002e86f8 in rexx.exe: 0xC0000005: Access violation reading location 0x028fd522.

                  

    * Scneario 2: 64-bit Linux, *"buffer overflow"*: ran the same tests
      as above and in the multithreaded test unit I get a
      non-redirectable output in the shell reading:
          o

            rony@ronylinux:/mnt/root_e/rony/dev/bsf/src/source_cc_40/testUnits$ rexx testOORexx -R bsf4rexx -s 2>&1 | gvim -
            Vim: Reading from stdin...
            **** buffer overflow detected ***: rexx terminated*
            ======= Backtrace: =========
            /lib/libc.so.6(__fortify_fail+0x37)[0x7fb7374ee2c7]
            /lib/libc.so.6[0x7fb7374ec170]
            /lib/libc.so.6[0x7fb7374eb519]
            /lib/libc.so.6(_IO_default_xsputn+0x96)[0x7fb737465426]
            /lib/libc.so.6(_IO_vfprintf+0x348d)[0x7fb737437e2d]
            /lib/libc.so.6(__vsprintf_chk+0x99)[0x7fb7374eb5b9]
            /lib/libc.so.6(__sprintf_chk+0x80)[0x7fb7374eb500]
            /mnt/root_e/rony/dev/bsf/src/bsf4rexx/libBSF4Rexx.so(BSF_impl+0x348)[0x7fb734d90fb8]
            /mnt/root_e/rony/dev/bsf/src/bsf4rexx/libBSF4Rexx.so(BSF+0x21)[0x7fb734d914e1]
            /opt/ooRexx/lib/ooRexx/librexx.so.4(_ZN20RexxNativeActivation17callNativeRoutineEP12RoutineClassP17RexxNativeRoutineP10RexxStringPP10RexxObjectmR15ProtectedObject+0xc2)[0x7fb738634e12]
            /opt/ooRexx/lib/ooRexx/librexx.so.4(_ZN12RoutineClass4callEP12RexxActivityP10RexxStringPP10RexxObjectmR15ProtectedObject+0x55)[0x7fb738607495]
            /opt/ooRexx/lib/ooRexx/librexx.so.4(_ZN14PackageManager17callNativeRoutineEP12RexxActivityP10RexxStringPP10RexxObjectmR15ProtectedObject+0x62)[0x7fb73864bca2]
            /opt/ooRexx/lib/ooRexx/librexx.so.4(_ZN17SystemInterpreter22invokeExternalFunctionEP14RexxActivationP12RexxActivityP10RexxStringPP10RexxObjectmS5_R15ProtectedObject+0x9e)[0x7fb73867e4fe]
            /opt/ooRexx/lib/ooRexx/librexx.so.4(_ZN14RexxActivation12externalCallEP10RexxStringmP19RexxExpressionStackS1_R15ProtectedObject+0x13a)[0x7fb738629aba]
            /opt/ooRexx/lib/ooRexx/librexx.so.4(_ZN22RexxExpressionFunction8evaluateEP14RexxActivationP19RexxExpressionStack+0x243)[0x7fb73865a003]
            /opt/ooRexx/lib/ooRexx/librexx.so.4(_ZN21RexxExpressionMessage8evaluateEP14RexxActivationP19RexxExpressionStack+0x59)[0x7fb73865ac29]
            ... cut ...
            ======= Memory map: ========
            00400000-00401000 r-xp 00000000 08:01 24936487                           /opt/ooRexx/bin/rrony@ronylinux:/mnt/root_e/rony/dev/bsf/src/source_cc_rony@ronylinux:/mnt/root_e/rony/dev/bsf/src/source_cc_40/testUnits$ 
                  


    * Scenario 3: only on 32- and 64-bit Linux, not on Windows:
          o all "regular" test units and examples that use the new
            callback from Java to Rexx mechanism work with one sole
            exception: that exception, if it occurs will always occur on
            the Java "awt-thread", which AFAIK is special in Java as it
            interfaces directly with the operating system/graphic
            system. This example extends an abstract Java class and uses
            a Rexx object's method as its implementation (this in
            principle works for all examples, tests and experiments,
            even the most stressing ones, such that I have come to
            believe that the basic implementation works bullet-proofly,
            just not in this special awt-case on Linux). Again this
            example works reliably on Windows 32- and 64-bit.
            Java tells me that the problem lies in BSF4Rexx, however
            after having traced it down (BSF4Rexx currently consists of
            more conditional debug code then real code, it seems to me
            :( ) the spot where the exception gets thrown is right at
            invoking a JNI-function, invoking NewObjectArray() and
            making sure that all arguments have a meaningful/valid
            value! So after thinking about it, it looks as if something
            goes wrong right there switching to JNI, where the JNI
            watching code assumes the problem is is right there, but in
            BSF4Rexx.


Now having thrown a lot of resources, efforts and time against this area
in order to figure out the places where the errors occur, and then go on
to find a pattern, in order to become able to locate the cause of the
problem and fix it.

However, in these three occurrences I am stucked at the moment and would
like to ask about insights, theories (could the operating system,
ooRexx, Java, JNI be culprits here too?) and ideas what may cause this
and how one could/should tackle and solve them.

TIA,

---rony