Re: [Swig-user] Low memory conditions and Java exceptions

[Brian C. <co...@ey...>]

The good news is that this seems to work better with the garbage collector (objects get cleaned faster). The bad news is that
 - it no longer seems to obey the -Xmx flag. If I specify 1Gb it goes to 2Gb and then dies in the jvm (on 32bit Windows)
Strange, our experience is that the -Xmx flag is still obeyed. If you have code that calls malloc instead of operator new it could be circumventing the overload.

 - it's much, much slower. We use STL a lot and possibly have loads of memory allocations, or maybe there is some other issue that I've not tracked down. .
 - it seems to be generally less memory efficient.
Our C++ already has some memory caching mechanisms in place that largely alleviates the problem, though not completely. Essentially, we override operator new and delete for specific small C++ classes we know get created and destroyed often. This shields us from slow memory allocators (java or libstdc++), especially in the case of multi-threading, which your allocators have to be in Java since the allocation and de-allocation occur in different threads. We're toying with the idea of writing our own stl allocator as well: http://www.sgi.com/tech/stl/alloc.html There are a few places this would help (particularly when objects are stored in vectors).

Be a good time to recommend you to use VisualVM as well: https://visualvm.dev.java.net/ Helped us a lot in figuring out what the hell the JVM was doing with all that memory and CPU cycles.

-Brian


Brian Cole wrote:
Re: [Swig-user] Low memory conditions and Java exceptions Using a %exception directive will wrap all JNI calls automatically. We use something like this to catch OutOfMemory exceptions:
%exception %{
    try {
      $action
    } catch (std::bad_alloc &) {
      return $null;
    } catch (std::exception &e) {
      jclass clazz = jenv->FindClass("java/lang/Exception");
      jenv->ThrowNew(clazz, e.what());
      return $null;
    } catch (...) {
      jclass clazz = jenv->FindClass("java/lang/Exception");
      jenv->ThrowNew(clazz, "Unknown exception");
      return $null;
    }
%}

We also overload operator new and delete for all our libraries to use the Java Heap as described here:  http://www.swig.org/Doc1.3/Java.html#java_heap_allocations

The operator new and delete uses the Java heap. When Java runs out of memory it will force those implementations to throw bad_alloc. The bad_alloc can then propagate through C++ correctly to then be caught by the above exception directive. That's why the bad_alloc catch block simply returns null, because the operator new and delete already set the java exception in the java environment.

Remember there are pitfalls defining operator new and delete in shared libraries. We do something similar to what is described here: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n1976.html#problem_01 Basically, make operator new and delete local to each shared library, that implementation than calls through to some implementation of a JAVA_malloc and JAVA_free that is located in some other mutual shared library.

Works beautifully for catching OutOfMemory.

Even with all that we still had cases where we could create objects faster than the JAVA finalizer thread could clean them up. We're in the middle of implementing something similar to http://java.sun.com/developer/technicalArticles/javase/finalization/. Works well, but we had to make significant modifications. I'll post more on it once I have time past our release date.

-Brian

On 1/28/09 12:03 PM, "David Creasy" <dc...@ma...> wrote:


Hi,

I'm using SWIG in a simple way and it does what we need just perfectly.
Almost.. In very low memory conditions, we get crashes while we'd like
something a 'little more refined.'

The java wrapper produced has for example:

public class myClass {
   private long swigCPtr;
   protected boolean swigCMemOwn;


In the java constructor, it calls the C++ myClass constructor and
assigns it to the swigCPtr. This then obviously gets used in the later
functions, for example:

    public String getValue() {
     return mylibJNI.myClass_getValue(swigCPtr, this);
   }

The problem is that if the C++ constructor fails (out of memory), then
the swigCPtr is null, but this is never checked and it then crashes when
trying to use the null pointer in, for example the jni getValue function.

So, it would be great if it could throw a java exception when the
constructor fails. We've got a very large number of classes, and I don't
want to have to add a %exception or something for each one. I guess in
an ideal world it would be nice if the C++ constructor failed, it could
call System.gc() and then retry the creation of the C++ object.

Apologies if I've missed something obvious.

David

------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
 http://p.sf.net/sfu/sf-spreadtheword
_______________________________________________
Swig-user mailing list
 Swi...@li...
 https://lists.sourceforge.net/lists/listinfo/swig-user




________________________________

------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword


________________________________

_______________________________________________
Swig-user mailing list
Swi...@li...
https://lists.sourceforge.net/lists/listinfo/swig-user