openjnlp-devel

[Openjnlp-devel] Lazy jars

[Mario C. <mar...@po...>]

I tried running a JNLP app that used lazy loading of some jars, and it =
simply didn't work because of ClassNotFoundExceptions.  Going through =
the code, I noticed that nothing is done about lazy jars.  My =
understanding of the spec. is that these jars should always be loaded =
over the network, like applet jars.  Is so, the correct way to implement =
it would be to add the remote (http:) URLs of the jars when constructing =
the class loader for the application (eager jars use local file: URLs).  =
I went ahead and made the required changes in FileCacheEntry, and =
included them below.  Maybe it will be useful to some people, maybe it =
will be included in the CVS tree, or maybe someone will set me straight =
on this issue.

Also note that on the Windows 2000 system I'm using to test this, the =
FileCache.toSafeURL() method breaks the local file: URLs, even though it =
seems like the correct thing to do.

Hope this will be useful to someone,

Mario

Index: FileCacheEntry.java

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

RCS file: =
/cvsroot/openjnlp/devel/src/org/nanode/launcher/cache/FileCacheEntry.java=
,v

retrieving revision 1.12

diff -w -r1.12 FileCacheEntry.java

157,159c157

< ArrayList eager =3D new ArrayList();

< File jarFile;

< URL jarURL;

---

> ArrayList urlList =3D new ArrayList();

162,164c160,166

< for (Enumeration enum =3D entryDescriptor.getResources().eagerJars(); =
enum.hasMoreElements();) {

< jarURL =3D ((Reference) enum.nextElement()).getURL();

< jarFile =3D new File(getResourceDir(), FileCache.cacheName(jarURL));

---

> for (Enumeration enum =3D entryDescriptor.getResources().jars(); =
enum.hasMoreElements();) {

> Reference jarReference =3D (Reference) enum.nextElement ();

> URL jarURL =3D jarReference.getURL ();

> if (jarReference.isLazy ()) {

> urlList.add (jarURL);

> } else { // Eager

> File jarFile =3D new File(getResourceDir(), =
FileCache.cacheName(jarURL));

167c169,170

< eager.add(FileCache.toSafeURL(jarFile));

---

> // TODO: This safeURL thing doesn't work under Windows 2000...

> urlList.add(jarFile.toURL () /* MC: WAS: =
FileCache.toSafeURL(jarFile)*/);

171a175

> }

173,174c177,178

< URL[] classpath =3D new URL[eager.size()];

< classpath =3D (URL[]) eager.toArray((Object[]) classpath);

---

> URL[] classpath =3D new URL[urlList.size()];

> classpath =3D (URL[]) urlList.toArray((Object[]) classpath);

Re: [Openjnlp-devel] Lazy jars

[Kevin H. <ke...@na...>]

On Wednesday, May 8, 2002, at 02:34 , Mario Cormier wrote:

> I tried running a JNLP app that used lazy loading of some jars, and it=20=

> simply didn't work because of ClassNotFoundExceptions.=A0 Going =
through=20
> the code, I noticed that nothing is done about lazy jars.=A0 My=20
> understanding of the spec. is that these jars should always be loaded=20=

> over the network, like applet jars.=A0 Is so, the correct way to=20
> implement it would be to add the remote (http:) URLs

You are correct, OpenJNLP does not yet support lazy jars. Unfortunately,=20=

your interpretation of loading lazy jars doesn't seem to agree with the=20=

specification. Lazy jars still are cached locally, but are not retrieved=20=

until needed. The pertinent part of the JNLP specification is Section=20
4.4, "Parts and Lazy Downloads". For this reason I don't think I can use=20=

your suggested changes (as good as they are) but I will use your work as=20=

motivation to get lazy jars implemented.

[...]
> Also note that on the Windows 2000 system I'm using to test this, the=20=

> FileCache.toSafeURL() method breaks the local file: URLs, even though=20=

> it seems like the correct thing to do.

Yeah, what I've discovered is that the "file" URL protocol handler from=20=

Sun is completely broken. You can't use correct URL encoding as per RFC=20=

2396 for file paths. I had to back out the FileCache.toSafeURL() calls.=20=

I ran into some serious problems with that and the way RMI works.=20
Something has to be done about spaces in names in the cache. Blah.

Re: [Openjnlp-devel] Lazy jars

[Mario C. <mar...@po...>]

Kevin,

First of all, the specification states that "a JNLP Client is always allowed
to eagerly download all resources if it chooses".  I think that as a
default, you could use that instead of not doing anything with lazily
downloadable resources.  At least things would work, and comply to the spec
too.

Second, I reread this part of the specification and it still seems to me
that my patch is a valid implementation as per the spec.  I agree with you
that it is desirable to cache the lazily downloaded resources (for
performance but especially for offline launching), but I don't think you are
required to.  My solution was inspired by a footnote in a previous version
of the spec that said "Lazy download is a standard feature of the Java 2
Platform's class loading mechanism."  This pointed directly to the
URLClassLoader, as far as I can tell.  However, this footnote disappeared
from later revisions, so maybe they did imply that caching was if not
mandated, at least strongly suggested, considering the offline scenarios.

In my opinion, lazy resources should be specified as remote URLs initially,
and these remote references be progressively replaced with local URLs as the
resources are downloaded in the background.  However, URLClassLoader does
not support such replacements of jars.  On the other hand, using the remote
URL on the first execution, and the local URL on subsequent executions,
achieves the same results.

Just my current thoughts on the subject, open to discussion... : )

Mario

> > I tried running a JNLP app that used lazy loading of some jars, and it
> > simply didn't work because of ClassNotFoundExceptions. Going through
> > the code, I noticed that nothing is done about lazy jars. My
> > understanding of the spec. is that these jars should always be loaded
> > over the network, like applet jars. Is so, the correct way to
> > implement it would be to add the remote (http:) URLs
>
> You are correct, OpenJNLP does not yet support lazy jars. Unfortunately,
> your interpretation of loading lazy jars doesn't seem to agree with the
> specification. Lazy jars still are cached locally, but are not retrieved
> until needed. The pertinent part of the JNLP specification is Section
> 4.4, "Parts and Lazy Downloads". For this reason I don't think I can use
> your suggested changes (as good as they are) but I will use your work as
> motivation to get lazy jars implemented.
>
> [...]

Re: [Openjnlp-devel] Lazy jars

[Mario C. <mar...@po...>]

Kevin,

I noticed your modifications to handle lazy jars this morning.  I must admit
that was fast work.  And it (mostly) works, too!  : )   The only problem I
saw was intermitent but I managed to pinpoint its source:  non-class
resources loaded from lazy jars.  I think you need to make modifications to
FileCacheClassLoader.findResource() to wait for lazy jars like you did in
findClass().  In my application, the problem showed up in various different
ways that can all be traced back to a missing non-class resource:  icons
missing, property files missing, JAXP
meta-inf/services/javax.xml.parsers.SAXParserFactory file missing (that one
threw me off because it resulted in a ClassNotFoundException on the fallback
value, so I initially thought your implementation was broken)...  All of
these problems are intermitent, because they depend on how fast the
background thread can load the required jars.

I hope this helps, and thanks again for your great work...

Mario

Re: [Openjnlp-devel] Lazy jars

[Kevin H. <ke...@na...>]

On Thursday, May 9, 2002, at 12:05 , Mario Cormier wrote:

> I noticed your modifications to handle lazy jars this morning.  I must 
> admit
> that was fast work.  And it (mostly) works, too!  : )   The only 
> problem I
> saw was intermitent but I managed to pinpoint its source:  non-class
> resources loaded from lazy jars.  I think you need to make 
> modifications to
> FileCacheClassLoader.findResource() to wait for lazy jars like you did 
> in
> findClass().  In my application, the problem showed up in various 
> different

Yeah, I just got the background loading and findClass() checking done. I 
have to work on non-OpenJNLP things during the day and I do need to 
sleep regularly.

I'll get the rest of it done later today/tonight, including the 
nativelib stuff. I've worked on software that does this kind of thing 
before, so that's why I was able to get it in there so quickly. I just 
needed someone to give me a kick in the butt to address the issue.

Re: [Openjnlp-devel] Lazy jars

[Kevin H. <ke...@na...>]

On Thursday, May 9, 2002, at 01:21 , Kevin Herrboldt wrote:
[...]
> I'll get the rest of it done later today/tonight, including the 
> nativelib stuff. I've worked on

This is all done and was committed to cvs, in case it was missed by any 
interested parties.

Re: [Openjnlp-devel] Lazy jars

[Kevin H. <ke...@na...>]

On Wednesday, May 8, 2002, at 04:44 , Mario Cormier wrote:

> Kevin,
>
> First of all, the specification states that "a JNLP Client is always 
> allowed
> to eagerly download all resources if it chooses".  I think that as a
> default, you could use that instead of not doing anything with lazily
> downloadable resources.  At least things would work, and comply to the 
> spec
> too.
>
> Second, I reread this part of the specification and it still seems to me
> that my patch is a valid implementation as per the spec.  I agree with 
> you

I went back and carefully re-read the spec, I now agree with your 
interpretation. Caching is always optional. I don't want to treat lazy 
resources as eager because that could be a horrible performance issue at 
startup time. Your approach with directly accessing lazy resources via 
HTTP would be better. However, I have a better idea...

I've modified the classloader to cache lazy resources in a separate 
thread if any exist. This is done by the constructor, so lazy resources 
are cached right away but still allow the app to start running.

I modified findClass() to wait for lazy resources to be updated before 
failing on not finding a class. Only after all lazy resources are up to 
date and the class is not found will the ClassNotFoundException be 
thrown.

[Openjnlp-devel] Background downloading of newer versions

[Mario C. <mar...@po...>]

Are there any plans to support background downloading of newer versions when
there exists a working version in the cache?  For instance, in section 1.2.2
of the spec it says "a JNLP client can download an updated version in the
background, while the already-downloaded version is being used".  (They seem
to imply that you need to use the version-based protocol to be able to do
that, but I fail to see why you would NEED it.)

I think having support for that would provide much faster startup times when
the network is slow.  I could implement it myself, but I think it might
require small changes all over the place so if it's already in the plans or
if someone is already tackling this issue, I suppose I can wait.  Any
suggestions on how to go about this?

Mario

Re: [Openjnlp-devel] Background downloading of newer versions

[Kevin H. <ke...@na...>]

On Thursday, May 9, 2002, at 01:57 , Mario Cormier wrote:

> Are there any plans to support background downloading of newer versions 
> when
> there exists a working version in the cache?  For instance, in section 
> 1.2.2
> of the spec it says "a JNLP client can download an updated version in 
> the
> background, while the already-downloaded version is being used".  (They 
> seem
> to imply that you need to use the version-based protocol to be able to 
> do
> that, but I fail to see why you would NEED it.)

That's exactly how I read the spec as well, and my admittedly limited 
testing with JWS seems to show the run current/download new behavior is 
only for version-based resources, not basic.

If you think about it, it does make sense. You don't want the background 
download to modify the jars of the version you're running. The way I 
view things a basic (unversioned) resource is just a special version. So 
downloading out-of-date jars is updating jars within that version, not 
getting a different version. You'd have the same scenario if you're 
using foo.jar version 1.2 and it's out of date, but still version 1.2.

> I think having support for that would provide much faster startup times 
> when
> the network is slow.  I could implement it myself, but I think it might
> require small changes all over the place so if it's already in the 
> plans or
> if someone is already tackling this issue, I suppose I can wait.  Any
> suggestions on how to go about this?

I've been giving thoughts as to how to add versioned resources to 
OpenJNLP. I think versioning needs to be implemented before any 
background upgrading can happen.

My thoughts on versioning so far are:

1) Multiple versions of an app can co-exist within the same cache entry. 
This means multiple versions of the same jar within the "Resources" 
directory. I think attaching the version-id to the resource file name 
will accomplish uniqueness. The basic version would simply be the 
resource name.

2) Basic (unversioned) resources are considered a unique version.

3) Updating an out-of-date resource does not change the version.

4) The version of a resource used for an app is determined by the jnlp 
file, which may also be versioned.

5) I think the entry.xml file would get updated to move the <resource> 
and <nativelib> entries inside a <version> tag or somesuch. A possible 
example is:

   <entry vendor="Foo" title="Bar">
     <version id="1.2">
       <resource href="http://www.foo.org/bar.jar" version="1.0" 
modtime="12345" />
     </version>

     <version id="1.1">
       ...
     </version>
   </entry>


I think it needs more thought/discussion before any implementation is 
done. But most importantly I'd like to get a new release of OpenJNLP 
done first. There are already a lot of big fixes/changes "in the can" 
that should be rolled out as an official version.