xmlvm-users

[xmlvm-users] OneJar and XMLVM

[Panayotis K. <pan...@pa...>]

This is a topic I find it really important for the future of XMLVM, since I've seen that there are some development decisions which should be "updated". It has to do with the usage of OneJar, which I believe it is about time to abandon it.
I'll explain why.

First of all, and more importantly, XMLVM jar is HUGE. And it is not only huge, it is huge for NOT a good reason. In general, it is not a bad idea to "statically link" all required objects inside a file. But if this file is in the order of 30MB, I think we are going to have a lot of problems with it, in regards to (a) space - the obvious, (b) time, in order to unzip items inside an already zipped file (c) packager complexity, since for example some classes (like the iphone compatibility library or the chassis & various icons) needed to be at most one level down inside the JAR file  (d) developer complexity, such as the problems we are talking now and probably others...

But this is not all. I was in the process of adding XMLVM to the MacPorts repository, in order to "advertise" it and make it more "accessible" to the people, who would like a tool that "just works" (you know how obsessed I am with this paradigm).
Unfortunately we can't - not as long as all libraries are inside XMLVM.
All distributions love small split packages, especially if there is a well documented, supported and easily provided library for each part. No sane distribution will allow OpenJDK to be "integrated" inside XMLVM, and I couldn't agree more. 
Moreover, since (all?) libraries used in XMLVM are open source and have a rapid development, XMLVM project could benefit of the multi-jar approach. This is the reason all other projects prefer to have (even a private) lib directory and keep all jars there, instead of packaging them inside a single Jar.

Last, but not least is that multi-jar approach could speed up things. xmlvm.jar could be compiled much faster (and on demand), android compatibility Jar could be created much faster etc etc. The whole procedure could be easier and more organized. If a library changes, there is no need to compile everything from scratch. Not to mention that this approach is practically "optimized" nowadays, since all other projects follow it.

The only real benefit is that XMLVM can be distributed just by one file. But this could be the case anyhow. It can be packaged and everything will be fine. Actually things are even better: instead of saying in the documentation to use the command "java -other-jvm-parameters -jar path-of-xmlvm/xmlvm.jar -actual-xmlvm-parameters", which is known to most java developers, but it is not "pretty", by using a specialized bash script things could be as easy (and straight forward) as "xmlvm -actual-xmlvm-parameters".

... and as always, I volunteer to perform this change :)

Re: [xmlvm-users] OneJar and XMLVM

[Jeff P. <jcp...@ro...>]

Maybe something like a .zip or .tar of jars will give a feel of the best of
both approaches.  One file so no accidental mis-matched use, but still
separate when used.

Yes, a .jar is already a .zip, but OS'es do special things when the
extension is .zip.  I believe OSX expands such file automatically when
downloaded & Windows shows as a directory in explorer.



On 9/17/10 3:25 AM, "Panayotis Katsaloulis" <pan...@pa...> wrote:

> This is a topic I find it really important for the future of XMLVM, since I've
> seen that there are some development decisions which should be "updated". It
> has to do with the usage of OneJar, which I believe it is about time to
> abandon it.
> I'll explain why.
> 
> First of all, and more importantly, XMLVM jar is HUGE. And it is not only
> huge, it is huge for NOT a good reason. In general, it is not a bad idea to
> "statically link" all required objects inside a file. But if this file is in
> the order of 30MB, I think we are going to have a lot of problems with it, in
> regards to (a) space - the obvious, (b) time, in order to unzip items inside
> an already zipped file (c) packager complexity, since for example some classes
> (like the iphone compatibility library or the chassis & various icons) needed
> to be at most one level down inside the JAR file  (d) developer complexity,
> such as the problems we are talking now and probably others...
> 
> But this is not all. I was in the process of adding XMLVM to the MacPorts
> repository, in order to "advertise" it and make it more "accessible" to the
> people, who would like a tool that "just works" (you know how obsessed I am
> with this paradigm).
> Unfortunately we can't - not as long as all libraries are inside XMLVM.
> All distributions love small split packages, especially if there is a well
> documented, supported and easily provided library for each part. No sane
> distribution will allow OpenJDK to be "integrated" inside XMLVM, and I
> couldn't agree more.
> Moreover, since (all?) libraries used in XMLVM are open source and have a
> rapid development, XMLVM project could benefit of the multi-jar approach. This
> is the reason all other projects prefer to have (even a private) lib directory
> and keep all jars there, instead of packaging them inside a single Jar.
> 
> Last, but not least is that multi-jar approach could speed up things.
> xmlvm.jar could be compiled much faster (and on demand), android compatibility
> Jar could be created much faster etc etc. The whole procedure could be easier
> and more organized. If a library changes, there is no need to compile
> everything from scratch. Not to mention that this approach is practically
> "optimized" nowadays, since all other projects follow it.
> 
> The only real benefit is that XMLVM can be distributed just by one file. But
> this could be the case anyhow. It can be packaged and everything will be fine.
> Actually things are even better: instead of saying in the documentation to use
> the command "java -other-jvm-parameters -jar path-of-xmlvm/xmlvm.jar
> -actual-xmlvm-parameters", which is known to most java developers, but it is
> not "pretty", by using a specialized bash script things could be as easy (and
> straight forward) as "xmlvm -actual-xmlvm-parameters".
> 
> ... and as always, I volunteer to perform this change :)
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> xmlvm-users mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlvm-users

Re: [xmlvm-users] OneJar and XMLVM

[Panayotis K. <pan...@pa...>]

On Sep 17, 2010, at 5:22 PM, Jeff Palmer wrote:

> Maybe something like a .zip or .tar of jars will give a feel of the best of
> both approaches.  One file so no accidental mis-matched use, but still
> separate when used.
> 
> Yes, a .jar is already a .zip, but OS'es do special things when the
> extension is .zip.  I believe OSX expands such file automatically when
> downloaded & Windows shows as a directory in explorer.
> 

I am not talking about distribution - distribution can be anything you like - from a deb package to a tag.bz2 file.
Or even a portlist entry in MacPorts.

I am mostly talking about how XMLVM tool-chain will "live" in our hard-drive. 
In a few words, I just believe that the monolithic approach should come to an end :)

Re: [xmlvm-users] OneJar and XMLVM

[Joshua M. <xm...@me...>]

Panayotis,

The one jar distribution has a single big advantage:  It has a known,
working, consistent set of files/code/classes that an provably work
together.  If we wanted to I could test it and say here, it works, use it.
It is a much more difficult process (and usually combinatorially impossible)
to do that with every version of every library that someone might attempt to
run XMLVM against.

As a user, I care mostly about one thing: that java -jar xmlvm.jar always
works with minimal hassle.  The one jar approach helps with that in certain
cases.

To address some of your other points:

Size: In 1992 I downloaded 12 megs over a 14400 baud modem.  It took most of
the day.  These days I can download 200 mb during a coffee break.  Its just
not that much of an issue.

Command line: I believe with most JRE implementations, you can get
./xmlvm.jar --arguments to work somehow.

Complication time packaging time for xmlvm.jar: meaningless for users, only
XMLVM dev types do that.  XMLVM run time might be a consideration, though in
my experience it is fast enough that it doesn't feel like a bottle neck in
my work flow.

Thats my 2 cents --

I like hearing your ideas they are always interesting.  Unfortunately, I
am arguing more against this one than for it.

-- Joshua Melcon


On Fri, Sep 17, 2010 at 7:28 AM, Panayotis Katsaloulis <
pan...@pa...> wrote:

>
> On Sep 17, 2010, at 5:22 PM, Jeff Palmer wrote:
>
> > Maybe something like a .zip or .tar of jars will give a feel of the best
> of
> > both approaches.  One file so no accidental mis-matched use, but still
> > separate when used.
> >
> > Yes, a .jar is already a .zip, but OS'es do special things when the
> > extension is .zip.  I believe OSX expands such file automatically when
> > downloaded & Windows shows as a directory in explorer.
> >
>
> I am not talking about distribution - distribution can be anything you like
> - from a deb package to a tag.bz2 file.
> Or even a portlist entry in MacPorts.
>
> I am mostly talking about how XMLVM tool-chain will "live" in our
> hard-drive.
> In a few words, I just believe that the monolithic approach should come to
> an end :)
>
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> xmlvm-users mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlvm-users
>

Re: [xmlvm-users] OneJar and XMLVM

[Panayotis K. <pan...@pa...>]

On 17 Σεπ 2010, at 7:20 μ.μ., Joshua Melcon wrote:

> Panayotis,
> 
> The one jar distribution has a single big advantage:  It has a known, working, consistent set of files/code/classes that an provably work together.  If we wanted to I could test it and say here, it works, use it.   It is a much more difficult process (and usually combinatorially impossible) to do that with every version of every library that someone might attempt to run XMLVM against. 

I agree with that.
As I said in the looong message, we can really have for our own use a "private" lib folder with all recommended lbrary versions - and leave compatibility chaos to the system packages.
What I don't like is to have embedded OpenJDK (and much more) inside the JAR, which needed to be double-unzipped every time.
What I am proposing actually is what every other project is doing: go from a monolithic to a modular system - even if the modules are "private", and "independent" from the "system libraries" of the same name.


> Size: In 1992 I downloaded 12 megs over a 14400 baud modem.  It took most of the day.  These days I can download 200 mb during a coffee break.  Its just not that much of an issue.

Size, probably not. But speed is. And it is here every time you compile a program. Don't tell me you haven't experienced it?

> Complication time packaging time for xmlvm.jar: meaningless for users, only XMLVM dev types do that.  XMLVM run time might be a consideration, though in my experience it is fast enough that it doesn't feel like a bottle neck in my work flow.

See above ;)

Re: [xmlvm-users] OneJar and XMLVM

[Arno P. <ar...@pu...>]

fact is that we will soon add quite a bit more to the OneJar. The source 
code of the Boehm GC and (more importantly) the cross-compiled version 
of the OpenJDK. Especially the latter will increase the size of the 
OneJar significantly.

In principle I like the OneJar approach for its simplicity: everything 
is bundled in one jar. However, we should make sure that the 
jar-inside-jar for huge files does not have any significant performance 
impact. If that should be the case, we definitely need to abandon the 
OneJar approach.

Anyone else want to chip in?

Arno


On 9/17/10 10:21 AM, Panayotis Katsaloulis wrote:
> On 17 Σεπ 2010, at 7:20 μ.μ., Joshua Melcon wrote:
>
>> Panayotis,
>>
>> The one jar distribution has a single big advantage:  It has a known, working, consistent set of files/code/classes that an provably work together.  If we wanted to I could test it and say here, it works, use it.   It is a much more difficult process (and usually combinatorially impossible) to do that with every version of every library that someone might attempt to run XMLVM against.
>
> I agree with that.
> As I said in the looong message, we can really have for our own use a "private" lib folder with all recommended lbrary versions - and leave compatibility chaos to the system packages.
> What I don't like is to have embedded OpenJDK (and much more) inside the JAR, which needed to be double-unzipped every time.
> What I am proposing actually is what every other project is doing: go from a monolithic to a modular system - even if the modules are "private", and "independent" from the "system libraries" of the same name.
>
>
>> Size: In 1992 I downloaded 12 megs over a 14400 baud modem.  It took most of the day.  These days I can download 200 mb during a coffee break.  Its just not that much of an issue.
>
> Size, probably not. But speed is. And it is here every time you compile a program. Don't tell me you haven't experienced it?
>
>> Complication time packaging time for xmlvm.jar: meaningless for users, only XMLVM dev types do that.  XMLVM run time might be a consideration, though in my experience it is fast enough that it doesn't feel like a bottle neck in my work flow.
>
> See above ;)
>
>
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> xmlvm-users mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlvm-users

Re: [xmlvm-users] OneJar and XMLVM

[Lennie De V. <len...@gm...>]

The one JAR approach is nice if you got a small project but XMLVM is
getting to a size where its not... by splitting everything into
seperate JARs make it more modular.
You can have a set of developers working on the Android-iPhone
specific libraried without inclusing the core lib or other libs.

On Fri, Sep 17, 2010 at 8:03 PM, Arno Puder <ar...@pu...> wrote:
>
> fact is that we will soon add quite a bit more to the OneJar. The source
> code of the Boehm GC and (more importantly) the cross-compiled version
> of the OpenJDK. Especially the latter will increase the size of the
> OneJar significantly.
>
> In principle I like the OneJar approach for its simplicity: everything
> is bundled in one jar. However, we should make sure that the
> jar-inside-jar for huge files does not have any significant performance
> impact. If that should be the case, we definitely need to abandon the
> OneJar approach.
>
> Anyone else want to chip in?
>
> Arno
>
>
> On 9/17/10 10:21 AM, Panayotis Katsaloulis wrote:
>> On 17 Σεπ 2010, at 7:20 μ.μ., Joshua Melcon wrote:
>>
>>> Panayotis,
>>>
>>> The one jar distribution has a single big advantage:  It has a known, working, consistent set of files/code/classes that an provably work together.  If we wanted to I could test it and say here, it works, use it.   It is a much more difficult process (and usually combinatorially impossible) to do that with every version of every library that someone might attempt to run XMLVM against.
>>
>> I agree with that.
>> As I said in the looong message, we can really have for our own use a "private" lib folder with all recommended lbrary versions - and leave compatibility chaos to the system packages.
>> What I don't like is to have embedded OpenJDK (and much more) inside the JAR, which needed to be double-unzipped every time.
>> What I am proposing actually is what every other project is doing: go from a monolithic to a modular system - even if the modules are "private", and "independent" from the "system libraries" of the same name.
>>
>>
>>> Size: In 1992 I downloaded 12 megs over a 14400 baud modem.  It took most of the day.  These days I can download 200 mb during a coffee break.  Its just not that much of an issue.
>>
>> Size, probably not. But speed is. And it is here every time you compile a program. Don't tell me you haven't experienced it?
>>
>>> Complication time packaging time for xmlvm.jar: meaningless for users, only XMLVM dev types do that.  XMLVM run time might be a consideration, though in my experience it is fast enough that it doesn't feel like a bottle neck in my work flow.
>>
>> See above ;)
>>
>>
>> ------------------------------------------------------------------------------
>> Start uncovering the many advantages of virtual appliances
>> and start using them to simplify application deployment and
>> accelerate your shift to cloud computing.
>> http://p.sf.net/sfu/novell-sfdev2dev
>> _______________________________________________
>> xmlvm-users mailing list
>> xml...@li...
>> https://lists.sourceforge.net/lists/listinfo/xmlvm-users
>
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> xmlvm-users mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlvm-users
>



-- 
Lennie De Villiers Blog: http://lenniedevilliers.blogspot.com/

Re: [xmlvm-users] OneJar and XMLVM

[Panayotis K. <pan...@pa...>]

On 17 Σεπ 2010, at 9:03 μ.μ., Arno Puder wrote:

> 
> fact is that we will soon add quite a bit more to the OneJar. The source 
> code of the Boehm GC and (more importantly) the cross-compiled version 
> of the OpenJDK. Especially the latter will increase the size of the 
> OneJar significantly.
> 
> In principle I like the OneJar approach for its simplicity: everything 
> is bundled in one jar. However, we should make sure that the 
> jar-inside-jar for huge files does not have any significant performance 
> impact. If that should be the case, we definitely need to abandon the 
> OneJar approach.


I totally agree. But I do think that the one jar approach makes system slow. For example, if you run xmlvm with no command line arguments, there is a significant time before displaying the help text, in order to unzip the contents of the file. This is an obvious problem but I also thing there are other performance issues elsewhere.

One thing I thought and didn't discuss it is, if we want to develop a java-based iphone application, only the xml2objc jar should could be added to the project and not everything. Similar with Android projects… :) This will decrease developer frustration - I think.

Re: [xmlvm-users] OneJar and XMLVM

[Arno P. <ar...@pu...>]

during the build process, the ant-script builds up a verbatim version of 
the content of the OneJar (under xmlvm/build). Perhaps we can do a 
hybrid approach for now: it would be easy enough to add an "ant install" 
target that simply copies the content of xmlvm/build somewhere. Add a 
little shell script with the proper -classpath arguments and we have it 
both ways.

Arno


On 9/17/10 12:24 PM, Panayotis Katsaloulis wrote:
> On 17 Σεπ 2010, at 9:03 μ.μ., Arno Puder wrote:
>
>>
>> fact is that we will soon add quite a bit more to the OneJar. The source
>> code of the Boehm GC and (more importantly) the cross-compiled version
>> of the OpenJDK. Especially the latter will increase the size of the
>> OneJar significantly.
>>
>> In principle I like the OneJar approach for its simplicity: everything
>> is bundled in one jar. However, we should make sure that the
>> jar-inside-jar for huge files does not have any significant performance
>> impact. If that should be the case, we definitely need to abandon the
>> OneJar approach.
>
>
> I totally agree. But I do think that the one jar approach makes system slow. For example, if you run xmlvm with no command line arguments, there is a significant time before displaying the help text, in order to unzip the contents of the file. This is an obvious problem but I also thing there are other performance issues elsewhere.
>
> One thing I thought and didn't discuss it is, if we want to develop a java-based iphone application, only the xml2objc jar should could be added to the project and not everything. Similar with Android projects… :) This will decrease developer frustration - I think.
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> xmlvm-users mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlvm-users

Re: [xmlvm-users] OneJar and XMLVM

[Lennie De V. <len...@gm...>]

This is one of the frustrations I'm starting to have to write
Android->iPhone apps.

On Fri, Sep 17, 2010 at 9:24 PM, Panayotis Katsaloulis
<pan...@pa...> wrote:
> On 17 Σεπ 2010, at 9:03 μ.μ., Arno Puder wrote:
>
>>
>> fact is that we will soon add quite a bit more to the OneJar. The source
>> code of the Boehm GC and (more importantly) the cross-compiled version
>> of the OpenJDK. Especially the latter will increase the size of the
>> OneJar significantly.
>>
>> In principle I like the OneJar approach for its simplicity: everything
>> is bundled in one jar. However, we should make sure that the
>> jar-inside-jar for huge files does not have any significant performance
>> impact. If that should be the case, we definitely need to abandon the
>> OneJar approach.
>
>
> I totally agree. But I do think that the one jar approach makes system slow. For example, if you run xmlvm with no command line arguments, there is a significant time before displaying the help text, in order to unzip the contents of the file. This is an obvious problem but I also thing there are other performance issues elsewhere.
>
> One thing I thought and didn't discuss it is, if we want to develop a java-based iphone application, only the xml2objc jar should could be added to the project and not everything. Similar with Android projects... :) This will decrease developer frustration - I think.
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> xmlvm-users mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlvm-users
>



-- 
Lennie De Villiers Blog: http://lenniedevilliers.blogspot.com/

Re: [xmlvm-users] OneJar and XMLVM

[Sascha H. <sa...@xm...>]

Let me chime in as well, as I did quite a lot of work recently to integrate
the OneJAR  mechanism into our processing pipeline ...

I agree with Joshua on this one. While I was skeptical when we introduced
the one-jar, I became a fan of it. There is a huge benefit of having a
single consistent JAR that just works and you don't have to bother with
anything else.

I invested quite a lot of time to improve the internal structures of XMLVM,
mainly the UniversalFile API, that makes it now very easy to handle the
situation where resources are located either inside or outside the One-JAR.

You said that things grow big. That is natural and as Joshua pointed out,
this is not a problem nowadays. Of course, if the One-Jar would become
hundreds of MBs big, that would be too much. But right now it doesn't feel
too big to download.
One more note on size: Of course things will get added, but we will
hopefully be able to significantly reduce the size of it soon. The OpenJDK
libs included contain too much stuff we will never need. Once we figured out
what is "junk", we will cut it out and we should be able to get rid of many
MBs easily.

The other argument you made is speed. Panayotis, you wrote a long e-mail
(which you ponted out yourself), but you failed to provide data: What I need
to to know are hard facts. Provide data about what takes how long and why
this is a problem, and how fast it would be in another case. THEN we we have
foundation to talk about and THEN we can maybe find other solutions as well.
For example: I don't know how the OneJAR mechanism exactly does its work,
but I agree with you in that it seams silly to extract parts of the OneJAR
that are not needed. E.g. when you just want to cross-compile a bunch of
files, you don't need to extract certain libs every time. However, with ZIP
files you can extract certain parts only, so it would make sense to have it
extract only the needed parts.
Another way to solve this is to extract things on first run and to cache it
on the disk to improve speed when executed again.

But again, for me what matters are numbers, and not assumptions. If you can
provide measurable numbers about *what's* slow, *why* it is bad and *how
fast* it could be, *then* we can talk concrete steps in improving the
situation.

// Sascha

2010/9/18 Lennie De Villiers <len...@gm...>

> This is one of the frustrations I'm starting to have to write
> Android->iPhone apps.
>
> On Fri, Sep 17, 2010 at 9:24 PM, Panayotis Katsaloulis
> <pan...@pa...> wrote:
> > On 17 Σεπ 2010, at 9:03 μ.μ., Arno Puder wrote:
> >
> >>
> >> fact is that we will soon add quite a bit more to the OneJar. The source
> >> code of the Boehm GC and (more importantly) the cross-compiled version
> >> of the OpenJDK. Especially the latter will increase the size of the
> >> OneJar significantly.
> >>
> >> In principle I like the OneJar approach for its simplicity: everything
> >> is bundled in one jar. However, we should make sure that the
> >> jar-inside-jar for huge files does not have any significant performance
> >> impact. If that should be the case, we definitely need to abandon the
> >> OneJar approach.
> >
> >
> > I totally agree. But I do think that the one jar approach makes system
> slow. For example, if you run xmlvm with no command line arguments, there is
> a significant time before displaying the help text, in order to unzip the
> contents of the file. This is an obvious problem but I also thing there are
> other performance issues elsewhere.
> >
> > One thing I thought and didn't discuss it is, if we want to develop a
> java-based iphone application, only the xml2objc jar should could be added
> to the project and not everything. Similar with Android projects... :) This
> will decrease developer frustration - I think.
> >
> ------------------------------------------------------------------------------
> > Start uncovering the many advantages of virtual appliances
> > and start using them to simplify application deployment and
> > accelerate your shift to cloud computing.
> > http://p.sf.net/sfu/novell-sfdev2dev
> > _______________________________________________
> > xmlvm-users mailing list
> > xml...@li...
> > https://lists.sourceforge.net/lists/listinfo/xmlvm-users
> >
>
>
>
> --
> Lennie De Villiers Blog: http://lenniedevilliers.blogspot.com/
>
>
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> xmlvm-users mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlvm-users
>

Re: [xmlvm-users] OneJar and XMLVM

[Panayotis K. <pan...@pa...>]

On 18 Σεπ 2010, at 2:25 μ.μ., Sascha Haeberling wrote:
> I invested quite a lot of time to improve the internal structures of XMLVM, mainly the UniversalFile API, that makes it now very easy to handle the situation where resources are located either inside or outside the One-JAR.

Exactly that's my point.
I have seen in the source code that you have started to use  UniversalFile which handles this situation, which from my point of view is a bad approach.
And before all this effort grows and be too big and/or important to abandon it, I wanted to present these problems and why (since some time ago) I am against the one-jar approach.


> You said that things grow big. That is natural and as Joshua pointed out, this is not a problem nowadays. Of course, if the One-Jar would become hundreds of MBs big, that would be too much. But right now it doesn't feel too big to download.
> One more note on size: Of course things will get added, but we will hopefully be able to significantly reduce the size of it soon. The OpenJDK libs included contain too much stuff we will never need. Once we figured out what is "junk", we will cut it out and we should be able to get rid of many MBs easily.

Still, the performance loss in terms of speed will always be there.
And as you said exactly, there is a lot of developer effort to support something that (in my point of view) is not a good idea any more. Space is one issue only. Anyway, even space is important, I don't like bloated software just because we can do it. I prefer more elegant solutions, without sacrificing ease of use.


> The other argument you made is speed. Panayotis, you wrote a long e-mail (which you ponted out yourself), but you failed to provide data: What I need to to know are hard facts. Provide data about what takes how long and why this is a problem, and how fast it would be in another case. THEN we we have foundation to talk about and THEN we can maybe find other solutions as well.
> For example: I don't know how the OneJAR mechanism exactly does its work, but I agree with you in that it seams silly to extract parts of the OneJAR that are not needed. E.g. when you just want to cross-compile a bunch of files, you don't need to extract certain libs every time. However, with ZIP files you can extract certain parts only, so it would make sense to have it extract only the needed parts.
> Another way to solve this is to extract things on first run and to cache it on the disk to improve speed when executed again.
> But again, for me what matters are numbers, and not assumptions. If you can provide measurable numbers about what's slow, why it is bad and how fast it could be, then we can talk concrete steps in improving the situation.


I said in a previous post how to get these numbers you are talking about, but, anyway, here they are.
If you want numbers, numbers you will have:

cd build/base/main ; time java -cp main.jar org.xmlvm.proc.NewMain 
0.24 real         0.25 user         0.04 sys

cd build/bin/ ; time java org.xmlvm.proc.NewMain
0.23 real         0.24 user         0.04 sys

cd dist ; time java -jar xmlvm.jar
2.16 real         2.37 user         0.25 sys


It's as you can see, an order of magnitude slower. Are these enough hard facts for you? If you run these scripts hundred times per day, in production, you will have the same feeling as I do. 

What actually surprises me is that the speed of using jars and files on disk takes exactly the  same time. It makes sense of course, since the time we loose to unzip something in memory will equal the time to access different files on disk.

Re: [xmlvm-users] OneJar and XMLVM

[Sascha H. <sa...@xm...>]

2010/9/18 Panayotis Katsaloulis <pan...@pa...>

> On 18 Σεπ 2010, at 2:25 μ.μ., Sascha Haeberling wrote:
> > I invested quite a lot of time to improve the internal structures of
> XMLVM, mainly the UniversalFile API, that makes it now very easy to handle
> the situation where resources are located either inside or outside the
> One-JAR.
>
> Exactly that's my point.
> I have seen in the source code that you have started to use  UniversalFile
> which handles this situation, which from my point of view is a bad approach.
> And before all this effort grows and be too big and/or important to abandon
> it, I wanted to present these problems and why (since some time ago) I am
> against the one-jar approach.
>

I disagree that this is a bad approach. The API makes it quite simple and
totally transparent to handle both situations well. I am not planning to
abandon it at all. It is now a well documented powerful API that is a core
part of XMLVM.


>
>
> > You said that things grow big. That is natural and as Joshua pointed out,
> this is not a problem nowadays. Of course, if the One-Jar would become
> hundreds of MBs big, that would be too much. But right now it doesn't feel
> too big to download.
> > One more note on size: Of course things will get added, but we will
> hopefully be able to significantly reduce the size of it soon. The OpenJDK
> libs included contain too much stuff we will never need. Once we figured out
> what is "junk", we will cut it out and we should be able to get rid of many
> MBs easily.
>
> Still, the performance loss in terms of speed will always be there.
> And as you said exactly, there is a lot of developer effort to support
> something that (in my point of view) is not a good idea any more. Space is
> one issue only. Anyway, even space is important, I don't like bloated
> software just because we can do it. I prefer more elegant solutions, without
> sacrificing ease of use.
>

Of course software shouldn't be bloated. But if many of the bytes that are
inside the JAR are needed, the other way would be to download these
dependencies separately. So having the One-JAR is much nicer.


>
>
> > The other argument you made is speed. Panayotis, you wrote a long e-mail
> (which you ponted out yourself), but you failed to provide data: What I need
> to to know are hard facts. Provide data about what takes how long and why
> this is a problem, and how fast it would be in another case. THEN we we have
> foundation to talk about and THEN we can maybe find other solutions as well.
> > For example: I don't know how the OneJAR mechanism exactly does its work,
> but I agree with you in that it seams silly to extract parts of the OneJAR
> that are not needed. E.g. when you just want to cross-compile a bunch of
> files, you don't need to extract certain libs every time. However, with ZIP
> files you can extract certain parts only, so it would make sense to have it
> extract only the needed parts.
> > Another way to solve this is to extract things on first run and to cache
> it on the disk to improve speed when executed again.
> > But again, for me what matters are numbers, and not assumptions. If you
> can provide measurable numbers about what's slow, why it is bad and how fast
> it could be, then we can talk concrete steps in improving the situation.
>
>
> I said in a previous post how to get these numbers you are talking about,
> but, anyway, here they are.
> If you want numbers, numbers you will have:
>
> cd build/base/main ; time java -cp main.jar org.xmlvm.proc.NewMain
> 0.24 real         0.25 user         0.04 sys
>
> cd build/bin/ ; time java org.xmlvm.proc.NewMain
> 0.23 real         0.24 user         0.04 sys
>
> cd dist ; time java -jar xmlvm.jar
> 2.16 real         2.37 user         0.25 sys
>
>
> It's as you can see, an order of magnitude slower. Are these enough hard
> facts for you? If you run these scripts hundred times per day, in
> production, you will have the same feeling as I do.
>

Actually no. Of course just showing the help dialog is the worst situation.
But show me the numbers when compiling a whole project.

I don't think that extracting data from the OneJAR is a huge part in the
whole process. Believe me, I care about speed just as much as you do. Last
week I made some significant changes that speed up the compilation
significantly. I was able to save minutes. If the OneJAR only adds 2 seconds
to a cross-compilation, I would see it as rather insignificant. But of
course, if there are clever ways to make the OneJAR faster, I am always for
it,but I wouldn't abandon it.



>
> What actually surprises me is that the speed of using jars and files on
> disk takes exactly the  same time. It makes sense of course, since the time
> we loose to unzip something in memory will equal the time to access
> different files on disk.
>
>
>
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> xmlvm-users mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlvm-users
>

Re: [xmlvm-users] OneJar and XMLVM

[Panayotis K. <pan...@pa...>]

On 18 Σεπ 2010, at 3:01 μ.μ., Sascha Haeberling wrote:

> 
> Actually no. Of course just showing the help dialog is the worst situation. But show me the numbers when compiling a whole project.

No it's not the worst case scenario, since all files need to be extracted all the time. It is not the best, it's just an indication that it slows down things. Probably not with

> 
> I don't think that extracting data from the OneJAR is a huge part in the whole process. Believe me, I care about speed just as much as you do. Last week I made some significant changes that speed up the compilation significantly. I was able to save minutes. If the OneJAR only adds 2 seconds to a cross-compilation, I would see it as rather insignificant. But of course, if there are clever ways to make the OneJAR faster, I am always for it,but I wouldn't abandon it.

It still needs a lot of improvements and I have made some myself in the past. Still, as Josua said, this is a benefit only for the xmlvm developer, not user (the iphone developer).

For an iPhone developer who compiles and saves a project 100s times, even a 2 second penalty is too much.

Now, for an actual case scenario, in order to do it it needs to make changes to the xmlvm to perform it.
If I make these changes, I prefer to submit it as a patch, instead of discussing here ;)

Re: [xmlvm-users] OneJar and XMLVM

[Sascha H. <sa...@gm...>]

2010/9/18 Panayotis Katsaloulis <pan...@pa...>

> On 18 Σεπ 2010, at 3:01 μ.μ., Sascha Haeberling wrote:
>
> >
> > Actually no. Of course just showing the help dialog is the worst
> situation. But show me the numbers when compiling a whole project.
>
> No it's not the worst case scenario, since all files need to be extracted
> all the time. It is not the best, it's just an indication that it slows down
> things. Probably not with
>

I call it worst case, because the work XMLVM needs to do is minimal (just
printing some text), but still everything needs to be launched and
extracted. When XMLVM does "real" work, this boot-up time will be a big part
anymore.


>
> >
> > I don't think that extracting data from the OneJAR is a huge part in the
> whole process. Believe me, I care about speed just as much as you do. Last
> week I made some significant changes that speed up the compilation
> significantly. I was able to save minutes. If the OneJAR only adds 2 seconds
> to a cross-compilation, I would see it as rather insignificant. But of
> course, if there are clever ways to make the OneJAR faster, I am always for
> it,but I wouldn't abandon it.
>
> It still needs a lot of improvements and I have made some myself in the
> past. Still, as Josua said, this is a benefit only for the xmlvm developer,
> not user (the iphone developer).
>
> For an iPhone developer who compiles and saves a project 100s times, even a
> 2 second penalty is too much.
>
> Now, for an actual case scenario, in order to do it it needs to make
> changes to the xmlvm to perform it.
> If I make these changes, I prefer to submit it as a patch, instead of
> discussing here ;)
>

Well now you are digressing, this is not productive. We all use XMLVM
everyday on tasks and we see how long things take. As I said, I did some
benchmarking last week when I implemented quite a few speed improvements.
Most of these task would run for many seconds anyway, so 2 seconds don't
seem much. But please correct me, if I am wrong.

As a concrete example: I was corss-compiling half of the OpenJDK library,
which originally took about 5 minutes to do. After the improvements
I've submitted it came down to 1.5 minutes. 2 seconds are not
even noticeable in such a scenario.

What I want to say is this: If most usual "real" tasks have a duration of
many seconds, then making a change as drastic as "getting rid of OneJAR"
only to save a minimal barely noticeable speed improvements seems out of
proportion.

// Sascha


>
>
>
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> xmlvm-users mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlvm-users
>