Thread: [Sablevm-developer] Incrasing SVM_FRAME_NATIVE_REFS_MIN

Brought to you by: egagnon

sablevm-developer

[Sablevm-developer] Incrasing SVM_FRAME_NATIVE_REFS_MIN

From: Grzegorz B. P. <ga...@de...> - 2004-03-03 22:52:37

Hi Etienne,

In Debian packages I want to incrase the number of native references
to something nearer to what Sun's JVM makes available (max about 100
AFAIR, can't find it now). Putting aside fact that apps that don't work
with 16 refs are in 99% of cases broken, I'd like to make sure it won't
hurt too much.

$ grep -R SVM_FRAME_NATIVE_REFS_MIN . 
./constants.h:#define SVM_FRAME_NATIVE_REFS_MIN 16
./native.c:      method->data.native_method->refargs_count +
SVM_FRAME_NATIVE_REFS_MIN;
./native.c:      method->data.native_method->refargs_count +
SVM_FRAME_NATIVE_REFS_MIN;
./prepare.c:	    (((SVM_FRAME_NATIVE_REFS_MIN + 2) +
./thread.c:  jint lrefs_count = SVM_FRAME_NATIVE_REFS_MIN;

So I don't see anything that would effectively bind maximum
value "by design" (like using some bit operations where each
bit is for one reference or similar). So besides incrasing
memory usage, possibly decrasing execution speed, what can
be the other consequences of doing that? Anything I should
really be afraid of?

Hoping for quick and short answer,

				Grzegorz B. Prokopski

PS: I want to upload something in shape of the final packages
that would make it into testing, before we release 1.1.1 with
GNU Classpath 0.08, which I think should happen soon after
0.08 release, next monday.

-- 
Grzegorz B. Prokopski <ga...@de...>
Debian GNU/Linux      http://www.debian.org
SableVM - LGPLed JVM  http://www.sablevm.org
Why SableVM ?!?       http://devel.sablevm.org/wiki/WhySableVM

Re: [Sablevm-developer] Incrasing SVM_FRAME_NATIVE_REFS_MIN

From: Archie C. <ar...@de...> - 2004-03-03 23:20:36

Grzegorz B. Prokopski wrote:
> In Debian packages I want to incrase the number of native references
> to something nearer to what Sun's JVM makes available (max about 100
> AFAIR, can't find it now). Putting aside fact that apps that don't work
> with 16 refs are in 99% of cases broken, I'd like to make sure it won't
> hurt too much.

Just curious.. are you encountering a lot of native code out there
that breaks with only 16 native refs per frame? I'd be interested
in knowing what native code does this.

Cheers,
-Archie

__________________________________________________________________________
Archie Cobbs      *        CTO, Awarix        *      http://www.awarix.com

Re: [Sablevm-developer] Incrasing SVM_FRAME_NATIVE_REFS_MIN

From: Grzegorz B. P. <ga...@de...> - 2004-03-04 03:00:00

W li=B6cie z =B6ro, 03-03-2004, godz. 17:47, Archie Cobbs pisze:=20
> Grzegorz B. Prokopski wrote:
> > In Debian packages I want to incrase the number of native references
> > to something nearer to what Sun's JVM makes available (max about 100
> > AFAIR, can't find it now). Putting aside fact that apps that don't wo=
rk
> > with 16 refs are in 99% of cases broken, I'd like to make sure it won=
't
> > hurt too much.
>=20
> Just curious.. are you encountering a lot of native code out there
> that breaks with only 16 native refs per frame? I'd be interested
> in knowing what native code does this.

It was a few weeks ago, I didn't have to look far.

$ apt-get install debbuggtk

Apparently it's gnome-java's bindings BUG. Feel free to report it :-)

Cheers,

					Grzegorz B. Prokopski

PS: But given what I've just read today in "The Java Native Interface
Programmer's Guide and Specification" I am pretty sure there's plenty
of such apps.

--=20
Grzegorz B. Prokopski <ga...@de...>
Debian GNU/Linux      http://www.debian.org
SableVM - LGPLed JVM  http://www.sablevm.org
Why SableVM ?!?       http://devel.sablevm.org/wiki/WhySableVM

Re: [Sablevm-developer] Incrasing SVM_FRAME_NATIVE_REFS_MIN

From: Grzegorz B. P. <ga...@de...> - 2004-03-04 05:05:44

W li=B6cie z =B6ro, 03-03-2004, godz. 17:47, Archie Cobbs pisze:=20
> Grzegorz B. Prokopski wrote:
> > In Debian packages I want to incrase the number of native references
> > to something nearer to what Sun's JVM makes available (max about 100
> > AFAIR, can't find it now). Putting aside fact that apps that don't wo=
rk
> > with 16 refs are in 99% of cases broken, I'd like to make sure it won=
't
> > hurt too much.
>=20
> Just curious.. are you encountering a lot of native code out there
> that breaks with only 16 native refs per frame? I'd be interested
> in knowing what native code does this.

I incrased it to 64 and I was able to get this:

http://gadek.debian.net/SableVM-BugLister.png !

Works like a charm ;-) Besides maybe that at some point it died on
sablevm: INTERNAL ERROR (source file "native_interface.c", line 22950):
todo
which is

static JNIEXPORT jint JNICALL
Throw (JNIEnv *_env, jthrowable obj)

David, by any chance is this one of the functions you implemented but
not merged yet? Or would it be one of those "trivial to implement,
harder to test"?

				GBP

--=20
Grzegorz B. Prokopski <ga...@de...>
Debian GNU/Linux      http://www.debian.org
SableVM - LGPLed JVM  http://www.sablevm.org
Why SableVM ?!?       http://devel.sablevm.org/wiki/WhySableVM

Re: [Sablevm-developer] Incrasing SVM_FRAME_NATIVE_REFS_MIN

From: David <db...@CS...> - 2004-03-04 19:14:49

On Wed, Mar 03, 2004 at 11:31:56PM -0500, Grzegorz B. Prokopski wrote:
> static JNIEXPORT jint JNICALL
> Throw (JNIEnv *_env, jthrowable obj)
>=20
> David, by any chance is this one of the functions you implemented but
> not merged yet? Or would it be one of those "trivial to implement,
> harder to test"?
>=20

Yes, I did one a long time ago.  It happened that I just merged it
in staging as well as 5 other JNI as I want to remove a branch.

The exec/wait merge is coming also.



David


---

David B=E9langer
Graduate Student
School of Computer Science
McGill University
Office: MC226

Web page:   http://www.cs.mcgill.ca/~dbelan2/
Public key: http://www.cs.mcgill.ca/~dbelan2/public_key.txt

Re: [Sablevm-developer] Incrasing SVM_FRAME_NATIVE_REFS_MIN

From: Etienne G. <gag...@uq...> - 2004-03-04 01:26:46

This minimum is encoded in the JNI specification.  You may petition Sun to
increase it, or alternatively, you could implement the related JNI
calls that allows native code to ask for more native references
*explicitly*.

I do NOT think that changing SableVM in order to support BROKEN JNI code
is the right thing to do.  Instead, SableVM could be made more helpful to
the broken code developer by giving him a better error message, including
a stack trace and an explicit message telling him about the need to either
free unused local references (in a loop, for example), or to request explicitly
more local references.

Etienne

Grzegorz B. Prokopski wrote:
> Hi Etienne,
> 
> In Debian packages I want to incrase the number of native references
> to something nearer to what Sun's JVM makes available (max about 100
> AFAIR, can't find it now). Putting aside fact that apps that don't work
> with 16 refs are in 99% of cases broken, I'd like to make sure it won't
> hurt too much.
> 
> $ grep -R SVM_FRAME_NATIVE_REFS_MIN . 
> ./constants.h:#define SVM_FRAME_NATIVE_REFS_MIN 16
> ./native.c:      method->data.native_method->refargs_count +
> SVM_FRAME_NATIVE_REFS_MIN;
> ./native.c:      method->data.native_method->refargs_count +
> SVM_FRAME_NATIVE_REFS_MIN;
> ./prepare.c:	    (((SVM_FRAME_NATIVE_REFS_MIN + 2) +
> ./thread.c:  jint lrefs_count = SVM_FRAME_NATIVE_REFS_MIN;
> 
> So I don't see anything that would effectively bind maximum
> value "by design" (like using some bit operations where each
> bit is for one reference or similar). So besides incrasing
> memory usage, possibly decrasing execution speed, what can
> be the other consequences of doing that? Anything I should
> really be afraid of?
> 
> Hoping for quick and short answer,
> 
> 				Grzegorz B. Prokopski
> 
> PS: I want to upload something in shape of the final packages
> that would make it into testing, before we release 1.1.1 with
> GNU Classpath 0.08, which I think should happen soon after
> 0.08 release, next monday.
> 


-- 
Etienne M. Gagnon, Ph.D.             http://www.info.uqam.ca/~egagnon/
SableVM:                                       http://www.sablevm.org/
SableCC:                                       http://www.sablecc.org/

Re: [Sablevm-developer] Incrasing SVM_FRAME_NATIVE_REFS_MIN

From: Grzegorz B. P. <ga...@de...> - 2004-03-04 02:59:19

W li=B6cie z =B6ro, 03-03-2004, godz. 20:09, Etienne Gagnon pisze:=20
> This minimum is encoded in the JNI specification.  You may petition Sun=
 to
> increase it, or alternatively, you could implement the related JNI
> calls that allows native code to ask for more native references
> *explicitly*.
It'd be nice to have them. I'll return to that in a second.

I haven't had much time to write this email, but before writing I took
a look at http://java.sun.com/docs/books/jni/html/refs.html, now I have
time to actually put here these snippets:

"In most cases, you do not have to worry about freeing local references
 when implementing a native method. The Java virtual machine frees them
 for you when the native method returns to the caller."
(honestly, I think it's bad introduction :-/ it encourages not to care.
It explains why we'll find tons of such programs that "don't care".)

"The JNI specification dictates that the virtual machine automatically
 ensures that each native method can create AT LEAST 16 local
 references." [*]

"If, however, there is a need to create additional local references, a
 native method MAY issue an EnsureLocalCapacity call to make sure that
 space for a sufficient number of local references is available." [*]

"The native code MAY create local references beyond the default
 capacity of 16 or the capacity reserved in a PushLocalFrame or
 EnsureLocalCapacity call. The virtual machine implementation WILL TRY
 to allocate the memory needed for the local reference. There is no
 guarantee, however, that memory will be available." [*]

So, I think Jonny programmer reads it as: "Sun's JVM will allocate
additional references as you use them. Don't worry."

Therefor I believe that allowing for usage of additional references
_without_ calling EnsureLocalCapacity is still in line w/ specs!
(Whether it's good idea to depend on implementation-specific and not
guaranteed behavior is another story, and I believe we fully agree on
how harmful it is.)

> I do NOT think that changing SableVM in order to support BROKEN JNI cod=
e
> is the right thing to do.  Instead, SableVM could be made more helpful =
to
> the broken code developer by giving him a better error message, includi=
ng
> a stack trace and an explicit message telling him about the need to eit=
her
> free unused local references (in a loop, for example), or to request ex=
plicitly
> more local references.
For _today_ I am inclined take the pragmatic approach.

_IF_ SableVM supported EnsureLocalCapacity, I could probably somehow call
it internally, when the basic, 16-references limit is exceeded. It would
be fully within the freedom that specs leave. BUT it doesn't have this
functionality implemented, so the only way is to define something bigger
at compile time. I don't mean putting that change in "upstream" SableVM,
but for Debian, where current usability of SableVM is the primary objecti=
ve,
I'll do it, especially if the risk of breaking things is low.

I share your pain because of that, but I think it's the right thing to do=
 ATM.

Thanks and cheers,

					Grzegorz B. Prokopski

[*] The emphasises are mine.

PS: On the other hand there'd be (almost) nothing bad to print a warning
on stderr, when a native thread exceeds the number of references it has
guaranteed according to the specs (and I'd say that at the return of
such function it should be printed/noted how many references it actually
needed). Sun's JVM has already an option to do this: -verbose:jni.

--=20
Grzegorz B. Prokopski <ga...@de...>
Debian GNU/Linux      http://www.debian.org
SableVM - LGPLed JVM  http://www.sablevm.org
Why SableVM ?!?       http://devel.sablevm.org/wiki/WhySableVM

Re: [Sablevm-developer] Incrasing SVM_FRAME_NATIVE_REFS_MIN

From: Etienne G. <gag...@uq...> - 2004-03-04 02:20:19

I mean, have a [deep ;-)] look at:
http://java.sun.com/docs/books/jni/html/functions.html#70207

Specifically:

Global and Local References
...
     * DeleteLocalRef reclaims the virtual machine resource needed for
       a local reference.
     * NewLocalRef is new in Java 2 SDK release 1.2.
     * EnsureLocalCapacity reserves space in the current thread for a
       fixed number of local references to be created.
       EnsureLocalCapacity is new in Java 2 SDK release 1.2.
     * PushLocalFrame and PopLocalFrame create a nested scope for local
       references. Both functions are new in Java 2 SDK release 1.2.

So, implementing EnsureLocalCapacity would be a nice starting point.
SableVM's stack layout has be designed so that Push/PopLocalFrame
could be easily implemented too; so this would be a nice second step.

Overall, implementing these 3 JNI functions would be better, IMHO,
than wasting precious memory resources on every native call to support
broken applications.  I do NOT think that automatically calling
EnsureLocalCapacity when an application goes over 16 local refs is a
good idea, either, as it would be hiding the bugs (memory leak).
Remember that keeping useless local refs around is very costly in
memory, as it prevents the garbage collector from collecting
*pointed-to* objects!

Etienne

-- 
Etienne M. Gagnon, Ph.D.             http://www.info.uqam.ca/~egagnon/
SableVM:                                       http://www.sablevm.org/
SableCC:                                       http://www.sablecc.org/

Re: [Sablevm-developer] Incrasing SVM_FRAME_NATIVE_REFS_MIN

From: Grzegorz B. P. <ga...@de...> - 2004-03-04 03:10:29

LOL!

I've just saw this email. Good to know our thoughts loiter the same
paths ;-) What can I say, I pretty much agree with the below. But
I can't get it all done right now, and even if - I think
thread.interrupt and friends are of higher priority (due to gtk
peers/awt stuff) especially that they can't be worked around.

And honestly, I'd be _very_ happy to put my hands in these right now.
But I've been working on release and Debian packages since I came back.
I think they're in quite good shape right now so I am gonna give them
some rest and try to finally nail down these inlined performance
problems.

Wish me luck, I'am gonna need it :-/

				GBP

W li=B6cie z =B6ro, 03-03-2004, godz. 21:05, Etienne Gagnon pisze:=20
> I mean, have a [deep ;-)] look at:
> http://java.sun.com/docs/books/jni/html/functions.html#70207
>=20
> Specifically:
>=20
> Global and Local References
> ...
>      * DeleteLocalRef reclaims the virtual machine resource needed for
>        a local reference.
>      * NewLocalRef is new in Java 2 SDK release 1.2.
>      * EnsureLocalCapacity reserves space in the current thread for a
>        fixed number of local references to be created.
>        EnsureLocalCapacity is new in Java 2 SDK release 1.2.
>      * PushLocalFrame and PopLocalFrame create a nested scope for local
>        references. Both functions are new in Java 2 SDK release 1.2.
>=20
> So, implementing EnsureLocalCapacity would be a nice starting point.
> SableVM's stack layout has be designed so that Push/PopLocalFrame
> could be easily implemented too; so this would be a nice second step.
>=20
> Overall, implementing these 3 JNI functions would be better, IMHO,
> than wasting precious memory resources on every native call to support
> broken applications.  I do NOT think that automatically calling
> EnsureLocalCapacity when an application goes over 16 local refs is a
> good idea, either, as it would be hiding the bugs (memory leak).
> Remember that keeping useless local refs around is very costly in
> memory, as it prevents the garbage collector from collecting
> *pointed-to* objects!
>=20
> Etienne
--=20
Grzegorz B. Prokopski <ga...@de...>
Debian GNU/Linux      http://www.debian.org
SableVM - LGPLed JVM  http://www.sablevm.org
Why SableVM ?!?       http://devel.sablevm.org/wiki/WhySableVM

Re: [Sablevm-developer] Incrasing SVM_FRAME_NATIVE_REFS_MIN

From: Archie C. <ar...@de...> - 2004-03-04 16:09:34

Grzegorz B. Prokopski wrote:
> I've just saw this email. Good to know our thoughts loiter the same
> paths ;-) What can I say, I pretty much agree with the below. But
> I can't get it all done right now, and even if - I think
> thread.interrupt and friends are of higher priority (due to gtk
> peers/awt stuff) especially that they can't be worked around.

FYI,

JC (which kindof started out based on SableVM) implementes extendable
local native reference frames and Thread.interrupt(); feel free to steal
back some code. I'd be happy to point anyone in the right direction.

-Archie

__________________________________________________________________________
Archie Cobbs      *        CTO, Awarix        *      http://www.awarix.com