#17 "System Resources Exhausted"

411.20130714
closed-fixed
None
5
2013-10-08
2013-05-08
Erik Duijs
No

We're sometimes seeing "System Resources Exhausted" messages appear that seem to be caused by a leak somewhere.
I know you've seen this message before wrt .package.new() and made a report at the ooRexx guys, but it seems to be a more general issue: It even occurs after some time when repeatedly running a completely empty REXX script on the same interpreter instance for example.

I've prepared a small test program that demonstrates a few different scenarios that will all lead to this error condition.

1 Attachments

Related

Bugs: #17

Discussion

  • Hi Erik,

    was away at the RexxLA symposium (http://www.rexxla.org/events/2013/schedule.html with links to slides presented), hence my late answer.

    AFAICT the problem resides within Rexx, hence it would be sensible to report the bug there. However, as my last bug report supplying the Java example you supplied has not caused any reaction, I assume that one needs to make the examples that lead to the exhausting system resource problem in C++ form. Would you or a colleague of you be able to code the necessary (rather short) C++ code to make it "Java-free" and report it with the ooRexx project bug tracker?

    (FYI: I am asking this as I am totally swamped currently with work, ca. 80 hrs per week, so my resources are currently almost non-existent, especially in May and June which are the heaviest loaded months for my lecturing and meetings at the University. If you are not able to come up with a C++ version of this and your previous report, please let me know and I try to find another way to create the C++ version or interest the ooRexx developers to look into it, as I am quite confident that this has nothing to do with BSF4ooRexx nor Java.)

     
  • Erik Duijs
    Erik Duijs
    2013-05-10

    Hi Rony,

    Thanks for your reply.
    I see what we can do regarding a C++ based test scenario (no promises but I'll do my best; my C++ skills are a bit rusty now :)
    We've also tried to reproduce from within REXX (stand-alone) but without luck so far.

    Best regards,
    Erik

     
  • Hi Erik,

    great!

    The sooner pure C++ versions can get created and reported with the ooRexx bug tracker the better, as it looks like a new bug-fix release of ooRexx might be considered in the near future, hence if your bug reports can be easily reproduced by the ooRexx team, the hopes would be that a fix may be possible soon.

    Best regards,

    ---rony

     
  • First of all: one needs to terminate() the BSFManager, such that it can terminate() the RexxEngine!

    It turns out that BSF4ooRexx.cc did not terminate Rexx interpreter instances (RII), even if doing a terminate() on the BSFManager object. Therefore ooRexx created a couple of hundred-thousands (!) RIIs until it rightfully runs out of resources!

    A fix for this is in commit # 138 (cf. http://sourceforge.net/p/bsf4oorexx/code/138).

     
    • status: open --> pending-fixed
    • Group: BSF4ooRexx (general) --> Next Release
     
  • Hi Erik,

    there is a new beta version available which addresses the reported memory leaks at:
    https://sourceforge.net/projects/bsf4oorexx/files/beta/20130629/.

    A few remarks:

    • please always use the BSFManager.terminate() method, not directly the RexxEngine.terminate(); if you do, then from now on there will be an exception thrown once you try to reuse that engine indicating that it got already terminated (BSFManager caches the RexxEngine and will return the terminated one, if you do not use BSFManager.terminate())

    • all your observed memory leaks should be gone

    • if possible for your applications, try to reuse an existing RexxEngine as that is the fastest way to dispatch a Rexx program

    If you find any new errors or still run out of system resources, then please report back ASAP.

    ---rony

     
  • Erik Duijs
    Erik Duijs
    2013-06-29

    Hi Rony,

    Thanks for the heads-up. 
    I'll test it this monday or tuesday (I accidently destroyed my laptop at home so this weekend I'm busy re-installing things on a new one...).

    I'll make sure to use BSFManager.terminate() instead of RexxEngine.terminate().
    The way it works in our implementation now is that the RexxEngine is reused per Rexx program invocation (so included scripts and sub-programs all use the same RexxEngine). When the main program has finished, the RexxEngine is discarded (and we'll call BSFManager.terminate() to do so). Would you reckon this is a good approach, or should we aim to even have just 1 RexxEngine that is reused globally?

    Anyway, I think I can say a big "thank you" on behalf of many people at the E.P.O. too for supporting BSF4ooRexx so relentlessly all this time; it's certainly very much appreciated :)
    Have a nice weekend!

    Erik


    From: Rony G. Flatscher orexx@users.sf.net
    To: [bsf4oorexx:bugs] 17@bugs.bsf4oorexx.p.re.sf.net
    Sent: Saturday, June 29, 2013 8:41 PM
    Subject: [bsf4oorexx:bugs] #17 "System Resources Exhausted"

    Hi Erik,
    there is a new beta version available which addresses the reported memory leaks at:
    https://sourceforge.net/projects/bsf4oorexx/files/beta/20130629/.
    A few remarks:
    * please always use the BSFManager.terminate() method, not directly the RexxEngine.terminate(); if you do, then from now on there will be an exception thrown once you try to reuse that engine indicating that it got already terminated (BSFManager caches the RexxEngine and will return the terminated one, if you do not use BSFManager.terminate())
    * all your observed memory leaks should be gone
    * if possible for your applications, try to reuse an existing RexxEngine as that is the fastest way to dispatch a Rexx program
    If you find any new errors or still run out of system resources, then please report back ASAP.
    ---rony
    ________

    [bugs:#17] "System Resources Exhausted"
    Status: pending-fixed
    Created: Wed May 08, 2013 03:31 PM UTC by Erik Duijs
    Last Updated: Mon Jun 10, 2013 11:29 AM UTC
    Owner: nobody
    We're sometimes seeing "System Resources Exhausted" messages appear that seem to be caused by a leak somewhere.
    I know you've seen this message before wrt .package.new() and made a report at the ooRexx guys, but it seems to be a more general issue: It even occurs after some time when repeatedly running a completely empty REXX script on the same interpreter instance for example.
    I've prepared a small test program that demonstrates a few different scenarios that will all lead to this error condition.


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/bsf4oorexx/bugs/17/
    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

     

    Related

    Bugs: #17

  • Replying the e-mail directly has not worked, hence copying my answer via the SourceForge interface:


    Hi Erik,

    On 29.06.2013 21:59, Erik Duijs wrote:

    Thanks for the heads-up.

    I'll test it this monday or tuesday (I accidently destroyed my laptop at home so this weekend I'm busy re-installing things on a new one...).
    Oops, good luck with recreating your laptop!

    I'll make sure to use BSFManager.terminate() instead of RexxEngine.terminate().
    The way it works in our implementation now is that the RexxEngine is reused per Rexx program invocation (so included scripts and sub-programs all use the same RexxEngine). When the main program has finished, the RexxEngine is discarded (and we'll call BSFManager.terminate() to do so). Would you reckon this is a good approach, or should we aim to even have just 1 RexxEngine that is reused globally?
    That should be fine with the fix. You are really free to use any strategy that you see to fit best with your application deployment scenarios.


    If you use a mixture of "classic" BSF4Rexx and "oo" BSF4ooRexx programs, then I would advise to pick a strategy, that from time to time terminates the RexxEngine via its BSFManager, then nullify the BSFManager and create a new one for new deployments. The reason being, that BSFManager is used to hold Java objects that get referred to by Rexx. In the classic approach (using the BSF() external Rexx function) the Rexx programmer is responsible for removing the Java objects from the (BSFManager's) registry once they are not needed anymore. However, for short lived programs it does not really matter. But for long termed executions and re-executions this may matter. Using the oo-approach (requiring BSF.CLS) each Java object gets an ooRexx proxy object, that has a destructor. So in the oo-case whenever a Rexx proxy is destroyed its destructor makes sure that its peer Java object gets removed from the (BSFManager) registry. So using the oo-approach not only makes programming much easier for the ooRexx programmer, but it also makes sure that the maintenance of the BSF registry is done automatically.

    In either scenario, if you nullify a BSFManager after calling its terminate() method, will also make the BSF registry available for garbage collection to Java. Whenever there are classic BSF4Rexx programs running, I would advise to do so.

    Anyway, I think I can say a big "thank you" on behalf of many people at the E.P.O. too for supporting BSF4ooRexx so relentlessly all this time; it's certainly very much appreciated :)
    Thank you very much for your kind feedback, which is highly appreciated as well!
    :-)

    Good luck with your laptop and have a nice weekend!

    ---rony

    From: Rony G. Flatscher orexx@users.sf.net
    To: [bsf4oorexx:bugs] 17@bugs.bsf4oorexx.p.re.sf.net
    Sent: Saturday, June 29, 2013 8:41 PM
    Subject: [bsf4oorexx:bugs] #17 "System Resources Exhausted"

    Hi Erik,
    there is a new beta version available which addresses the reported memory leaks at:
    https://sourceforge.net/projects/bsf4oorexx/files/beta/20130629/.
    A few remarks:
    please always use the BSFManager.terminate() method, not directly the RexxEngine.terminate(); if you do, then from now on there will be an exception thrown once you try to reuse that engine indicating that it got already terminated (BSFManager caches the RexxEngine and will return the terminated one, if you do not use BSFManager.terminate())
    all your observed memory leaks should be gone
    * if possible for your applications, try to reuse an existing RexxEngine as that is the fastest way to dispatch a Rexx program
    If you find any new errors or still run out of system resources, then please report back ASAP.
    ---rony
    __

    [bugs:#17] "System Resources Exhausted"
    Status: pending-fixed
    Created: Wed May 08, 2013 03:31 PM UTC by Erik Duijs
    Last Updated: Mon Jun 10, 2013 11:29 AM UTC
    Owner: nobody
    We're sometimes seeing "System Resources Exhausted" messages appear that seem to be caused by a leak somewhere.
    I know you've seen this message before wrt .package.new() and made a report at the ooRexx guys, but it seems to be a more general issue: It even occurs after some time when repeatedly running a completely empty REXX script on the same interpreter instance for example.
    I've prepared a small test program that demonstrates a few different scenarios that will all lead to this error condition.

    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/bsf4oorexx/bugs/17/
    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

    [bugs:#17] "System Resources Exhausted"

    Status: pending-fixed
    Created: Wed May 08, 2013 03:31 PM UTC by Erik Duijs
    Last Updated: Sat Jun 29, 2013 06:41 PM UTC
    Owner: nobody

    We're sometimes seeing "System Resources Exhausted" messages appear that seem to be caused by a leak somewhere.
    I know you've seen this message before wrt .package.new() and made a report at the ooRexx guys, but it seems to be a more general issue: It even occurs after some time when repeatedly running a completely empty REXX script on the same interpreter instance for example.

    I've prepared a small test program that demonstrates a few different scenarios that will all lead to this error condition.

    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/bsf4oorexx/bugs/17/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

    --


    Prof. Dr. Rony G. Flatscher
    Department Informationsverarbeitung und Prozessmanagement
    Institut für Betriebswirtschaftslehre und Wirtschaftsinformatik
    WU Wien
    Augasse 2-6
    A-1090 Wien/Vienna, Austria/Europe

    http://www.wu.ac.at (English: http://www.wu.ac.at/start/en)


     

    Related

    Bugs: #17

    • status: pending-fixed --> closed-fixed
    • assigned_to: Rony G. Flatscher
    • Group: Next Release --> version 411.20130714