From: <as...@pr...> - 2003-12-31 13:42:11
|
I'd like to load classes (from byte arrays, in case this matters) into a running ABL and have JCLASS & al. find it. Since I can only load classes using an instance of a subclass (something like JavaClassLoader) of ClassLoader, and not the default class loader, Class.forName() won't find it unless it's given the right class loader instance. Should I extend the class finding mechanism in Java.java to deal with this, or is there a simpler alternative? Andras |
From: Peter G. <pe...@ar...> - 2003-12-31 16:18:23
|
On Wed, 31 Dec 2003 at 14:42:01 +0100, Andr=E1s_Simon wrote: > I'd like to load classes (from byte arrays, in case this matters) into > a running ABL and have JCLASS & al. find it. Since I can only load > classes using an instance of a subclass (something like > JavaClassLoader) of ClassLoader, and not the default class loader, > Class.forName() won't find it unless it's given the right class loader > instance. Should I extend the class finding mechanism in Java.java to > deal with this, or is there a simpler alternative? Well, since you ask... ;) It seems like it would be easy enough to replace the calls to Class.forName() in Java.java with calls to a wrapper function that would make a new JavaClassLoader and load the class that way. Then it would be trivial to add support for in-memory byte arrays (if that's what you're talking about) to JavaClassLoader.java, and you could extend the code in Java.java to do the right thing. There's another unimplemented feature lurking here. It would be nice to have a global Lisp variable *DEFAULT-CLASSPATH-DEFAULTS*, which would be initialized at startup to the Java CLASSPATH in the form of a Lisp list (of namestrings, I suppose, until pathnames work a bit better), somewhat in the same spirit as *DEFAULT-PATHNAME-DEFAULTS*. If JCLASS and friends used JavaClassLoader.java to load classes, that code could look at *DEFAULT-CLASSPATH-DEFAULTS* to find the .class file, which means the user could (in effect) modify the CLASSPATH of a running ABL, which would probably be quite useful. So I guess what I'm trying to say here is, there's no simpler alternative that I'm aware of, but extending the class finding mechanism doesn't seem too hard, and the *DEFAULT-CLASSPATH-DEFAULTS* thing might fall out as a useful bonus feature. -Peter |
From: <as...@pr...> - 2003-12-31 17:19:24
|
On Wed, 31 Dec 2003, Peter Graves wrote: > It seems like it would be easy enough to replace the calls to > Class.forName() in Java.java with calls to a wrapper function that > would make a new JavaClassLoader and load the class that way. Then it > would be trivial to add support for in-memory byte arrays (if that's > what you're talking about) to JavaClassLoader.java, and you could > extend the code in Java.java to do the right thing. I thought that one instance of the new class loader (stored in a static variable in Java.java) would suffice. The runtime class generator would use that to load the generated classes, and the wrapper for Class.forName() would use that if it doesn't find the class with the simple (one arg) version of Class.forName(). (But then my understanding of ClassLoader is rather limited.) Is there anything to be gained by creating separate instances of the class loader for every class? Or is it unavoidable? > There's another unimplemented feature lurking here. It would be nice to > have a global Lisp variable *DEFAULT-CLASSPATH-DEFAULTS*, which would > be initialized at startup to the Java CLASSPATH in the form of a Lisp > list (of namestrings, I suppose, until pathnames work a bit better), > somewhat in the same spirit as *DEFAULT-PATHNAME-DEFAULTS*. If JCLASS > and friends used JavaClassLoader.java to load classes, that code could > look at *DEFAULT-CLASSPATH-DEFAULTS* to find the .class file, which > means the user could (in effect) modify the CLASSPATH of a running ABL, > which would probably be quite useful. OK, now I see why you'd want Java (the class) to be responsible for loading classes. For generated classes, loading them right after they're created seems more natural (easier?) for me. Or maybe it's just that first I'd like to get the class generation stuff out of the way before thinking about other matters. Andras |
From: Peter G. <pe...@ar...> - 2003-12-31 17:56:47
|
On Wed, 31 Dec 2003 at 18:19:21 +0100, Andr=E1s_Simon wrote: > I thought that one instance of the new class loader (stored in a > static variable in Java.java) would suffice. The runtime class > generator would use that to load the generated classes, and the > wrapper for Class.forName() would use that if it doesn't find the > class with the simple (one arg) version of Class.forName(). (But then > my understanding of ClassLoader is rather limited.) Is there anything > to be gained by creating separate instances of the class loader for > every class? Or is it unavoidable? My memory of this is a bit hazy, but I seem to recall that classes (as opposed to objects) can only be garbage-collected when the class loader that loaded the class has itself been garbage-collected. That's why LOAD-COMPILED-FUNCTION creates a new JavaClassLoader each time: by doing it that way, the class used for a JVM-compiled function can be gc'ed when it's no longer needed (when the function in question is redefined). If L-C-F used a single static instance of JavaClassLoader, those classes would never be gc'ed, even though they might no longer be needed. This consideration is obviously an important one for the loading of classes that implement Lisp functions, since they tend to get replaced all the time. I suspect the same consideration might apply to JCLASS, but I'm not sure (since I don't know how folks are using JCLASS); right now, those classes are never gc'ed, since they're loaded by Class.forName(), which uses the default class loader. > OK, now I see why you'd want Java (the class) to be responsible for > loading classes. For generated classes, loading them right after > they're created seems more natural (easier?) for me. Or maybe it's > just that first I'd like to get the class generation stuff out of the > way before thinking about other matters. I think it's probably more flexible to use a new class loader for each call to JCLASS. If practice reveals that this causes a performance problem, we can always try out the static variable approach later. -Peter |
From: <as...@pr...> - 2003-12-31 20:09:52
|
On Wed, 31 Dec 2003, Peter Graves wrote: > My memory of this is a bit hazy, but I seem to recall that classes (as > opposed to objects) can only be garbage-collected when the class loader > that loaded the class has itself been garbage-collected. That's why > LOAD-COMPILED-FUNCTION creates a new JavaClassLoader each time: by > doing it that way, the class used for a JVM-compiled function can be > gc'ed when it's no longer needed (when the function in question is > redefined). If L-C-F used a single static instance of JavaClassLoader, > those classes would never be gc'ed, even though they might no longer be > needed. > > This consideration is obviously an important one for the loading of > classes that implement Lisp functions, since they tend to get replaced > all the time. I suspect the same consideration might apply to JCLASS, > but I'm not sure (since I don't know how folks are using JCLASS); right > now, those classes are never gc'ed, since they're loaded by > Class.forName(), which uses the default class loader. Do unneeded classes get gc'ed in a normal Java application? If not, then I wouldn't worry about it here either. > I think it's probably more flexible to use a new class loader for each > call to JCLASS. If practice reveals that this causes a performance > problem, we can always try out the static variable approach later. What I don't understand with this approach is how Java (the class) finds the class definition of a generated class (there's of course no problem with .class files). Unless JCLASS initiates the generation of the class; or the byte array is stored somewhere and JCLASS looks it up so that it can call defineClass() with the byte array. If this is so, then it looks like more bookkeeping than the static approach. But maybe it's worth it, I don't know. Andras |
From: Peter G. <pe...@ar...> - 2003-12-31 22:16:47
|
On Wed, 31 Dec 2003 at 21:09:46 +0100, Andr=E1s_Simon wrote: > Do unneeded classes get gc'ed in a normal Java application? No, not if they are loaded by the default class loader. The default class loader keeps a reference to each class it has loaded, so the class can't be gc'ed until the default class loader goes away, which never happens. > If not, then I wouldn't worry about it here either. Fair enough. > > I think it's probably more flexible to use a new class loader for eac= h > > call to JCLASS. If practice reveals that this causes a performance > > problem, we can always try out the static variable approach later. > > What I don't understand with this approach is how Java (the class) > finds the class definition of a generated class (there's of course no > problem with .class files). Unless JCLASS initiates the generation of > the class; or the byte array is stored somewhere and JCLASS looks it > up so that it can call defineClass() with the byte array. If this is > so, then it looks like more bookkeeping than the static approach. But > maybe it's worth it, I don't know. Now I'm really confused. Even if you keep the class loader around in a static variable, don't you still have to pass the byte array to defineClass()? Where is JCLASS going to get the byte array? I was going on the assumption that you were planning to add an optional or keyword argument to JCLASS so its caller (the application) could pass in the byte array. = But now I'm afraid I just don't understand what you're trying to do. -Peter |
From: <as...@pr...> - 2004-01-01 03:09:38
|
On Wed, 31 Dec 2003, Peter Graves wrote: > Now I'm really confused. Sorry for being unclear! Here's the plan in more detail: JavaClassLoader has a public loadClassFromByteArray(String,byte[]) method, and Java has a public static JavaClassLoader field, say jcl. Now when the class generator has finished filling a byte array, it calls Java.jcl.loadClassFromByteArray with it and the classname. And when JCLASS & friends need to find a class, they call a wrapper around Class.forName, which does something like try { Class.forName(classname); } catch (ClassNotFoundException e) { try { Class.forName(classname, true, jcl); } catch (ClassNotFoundException cnf) { signal(new LispError("class not found: " + cnf.getMessage())); } } > > Even if you keep the class loader around in a static variable, don't > you still have to pass the byte array to defineClass()? Where is JCLASS > going to get the byte array? In this scenario, the class generator passes the byte array to jcl.loadClassFromByteArray() (which in turn calls defineClass() with it), and JCLASS doesn't need it. > > I was going on the assumption that you were planning to add an optional > or keyword argument to JCLASS so its caller (the application) could > pass in the byte array. No, it's the class generator that passes the byte array, and not to JCLASS, but to jcl.loadClassFromByteArray(). (Actually, I'd rather not carry that byte array around.) What the application passes is the description (in Lisp) of the class, and it passes it to the class generator. An optional or keyword argument wouldn't be very practical, because there are quite a few functions in the JAVA package, which do Class.forName implicitly, so they all would have to accept this optional argument. What do you think? Andras ps. I wish you and every reader of this list a very happy 2004! |
From: Peter G. <pe...@ar...> - 2004-01-01 16:43:23
|
On Thu, 1 Jan 2004 at 04:09:35 +0100, Andr=E1s_Simon wrote: > Sorry for being unclear! Here's the plan in more detail: > > JavaClassLoader has a public loadClassFromByteArray(String,byte[]) > method, and Java has a public static JavaClassLoader field, say > jcl. Now when the class generator has finished filling a byte array, > it calls Java.jcl.loadClassFromByteArray with it and the > classname. And when JCLASS & friends need to find a class, they call a > wrapper around Class.forName, which does something like > > > try { > Class.forName(classname); > } > catch (ClassNotFoundException e) { > try { > Class.forName(classname, true, jcl); > } > catch (ClassNotFoundException cnf) { > signal(new LispError("class not found: " + cnf.getMessage())); > } > } OK. > > Even if you keep the class loader around in a static variable, > > don't you still have to pass the byte array to defineClass()? Where > > is JCLASS going to get the byte array? > > In this scenario, the class generator passes the byte array to > jcl.loadClassFromByteArray() (which in turn calls defineClass() with > it), and JCLASS doesn't need it. > > > I was going on the assumption that you were planning to add an > > optional or keyword argument to JCLASS so its caller (the > > application) could pass in the byte array. > > No, it's the class generator that passes the byte array, and not to > JCLASS, but to jcl.loadClassFromByteArray(). (Actually, I'd rather not > carry that byte array around.) What the application passes is the > description (in Lisp) of the class, and it passes it to the class > generator. > > An optional or keyword argument wouldn't be very practical, because > there are quite a few functions in the JAVA package, which do > Class.forName implicitly, so they all would have to accept this > optional argument. > > What do you think? It makes more sense to me now. You're going to write a class generator, that will take a description (in Lisp) of a Java class and generate the class as a byte array. Then the generator will pass the byte array (together with the name of the class) to a persistent instance of a JavaClassLoader, which will return the properly verified and initialized class. And you want JCLASS to be able to find classes that are constructed in this way, so it needs to be made smart enough to consult the persistent JavaClassLoader in case the system class loader has never heard of the class in question. All of that sounds fine. So, in addition to JCLASS, you're probably planning to change the code in Java.forClassRef() that now calls Class.forName() to do the same thing, right? It seems like it would make sense to keep the persistent instance of JavaClassLoader in JavaClassLoader.java and initialize it lazily, more or less like this: private static JavaClassLoader persistentInstance; public static JavaClassLoader getPersistentInstance() { if (persistentInstance =3D=3D null) persistentInstance =3D new JavaClassLoader(); return persistentInstance; } Then the code in Java.java could just call JCL.getPersistentInstance(). (And LOAD-COMPILED-FUNCTION would continue to create a new JCL each time, like it does now.) Please correct me if I'm still missing the point! > ps. I wish you and every reader of this list a very happy 2004! Likewise! -Peter |
From: <as...@pr...> - 2004-01-01 18:04:28
|
On Thu, 1 Jan 2004, Peter Graves wrote: > You're going to write a class generator, that will take a description > (in Lisp) of a Java class and generate the class as a byte array. Then > the generator will pass the byte array (together with the name of the > class) to a persistent instance of a JavaClassLoader, which will return > the properly verified and initialized class. And you want JCLASS to be > able to find classes that are constructed in this way, so it needs to > be made smart enough to consult the persistent JavaClassLoader in case > the system class loader has never heard of the class in question. Exactly! > > All of that sounds fine. Good. > > So, in addition to JCLASS, you're probably planning to change the code > in Java.forClassRef() that now calls Class.forName() to do the same > thing, right? Yes. > > It seems like it would make sense to keep the persistent instance of > JavaClassLoader in JavaClassLoader.java and initialize it lazily, more > or less like this: > > private static JavaClassLoader persistentInstance; > > public static JavaClassLoader getPersistentInstance() > { > if (persistentInstance =3D=3D null) > persistentInstance =3D new JavaClassLoader(); > return persistentInstance; > } > > Then the code in Java.java could just call JCL.getPersistentInstance(). OK, I'll do it this way. > > (And LOAD-COMPILED-FUNCTION would continue to create a new JCL each > time, like it does now.) Yes. BTW, the class generator is already in a usable state, or rather, it was until I did a cvs update, which broke it (probably not seriously) and ilisp (completely). This may have sth to do with the new pathname stuff, but right know this is just guessing. I hope I'll soon find out what's really happening. Andras |
From: Peter G. <pe...@ar...> - 2004-01-01 18:17:20
|
On Thu, 1 Jan 2004 at 19:04:21 +0100, Andr=E1s_Simon wrote: > BTW, the class generator is already in a usable state, or rather, it > was until I did a cvs update, which broke it (probably not seriously) > and ilisp (completely). This may have sth to do with the new pathname > stuff, but right know this is just guessing. I hope I'll soon find out > what's really happening. Well, the new pathname code is quite different from the old pathname code, so it's very possible I broke something. It currently passes 70 out of 90 tests in its category in the ANSI test suite; I'm working on the failures as we speak. If you find something specific that's broken, I can probably fix it quickly. -Peter |
From: <as...@pr...> - 2004-01-01 20:41:52
|
On Thu, 1 Jan 2004, Peter Graves wrote: > Well, the new pathname code is quite different from the old pathname > code, so it's very possible I broke something. It currently passes 70 > out of 90 tests in its category in the ANSI test suite; I'm working on > the failures as we speak. > > If you find something specific that's broken, I can probably fix it > quickly. I still haven't found anything specific, but you've already fixed it :-) Another cvs update brought ilisp back. Thanks! Andras |