Re: [xmlvm-users] Intermediate conversion to xmlvm format

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Thanks for all of the replies on this.  I have succeeded in knocking
rebuild times down to under 10 seconds in some cases, 20 seconds in most
cases and 45 seconds in almost all cases.  The strategy I used was to:

1. Compile everything using the "vtable" target, and store in a cached
directory.
2. Compile everything to c (using whatever target I plan to use... in my
case I'm generating the iphone xcode project).

3. On rebuilds, I compare modification times of java source files with the
corresponding c header files in the destination build directory.  I only
recompile the changed files with javac.
4. I build a dependency tree starting with the changed files to produce a
set of "dirty" files (e.g. files that might have to be changed due to the
changes to the changed files).
5. I convert the set of dirty .class files to c files using XMLVM (using
the "c" target).  This doesn't pull in any of the dependencies, but it does
generate warnings when classes vtables can't be found.
6. Remove the dependency on the constant pool for the converted c files.
 This was fairly easy to do via regex.
7. Fix vtable errors in the converted c files.  This involves some crude
static analysis of the c source files (and I can make use of the cached c
source from the initial builds).
8. Copy the resulting c source files into the final build directory.
9. Run the project in Xcode.  It will only need to recompile the changed
source files, so this is very fast.

If I'm making changes to a java class that uses other classes, but isn't
itself used by any other classes, then the set of dirty files will only be
that one file.  If I'm working on a class that is used by a lot of other
classes, of course this will result in more classes requiring recompile.
 But unless I'm making changes to java.lang.Object (which I shouldn't be)
 the dirty set is still usually pretty reasonable.

Right now the source code is a bit of a mess because it was the result of a
lot of experimentation, and it is geared toward my particular use case but
I will be placing it on Github once I've cleaned it up a bit.

Steve

On Mon, Mar 10, 2014 at 9:11 PM, Arno Puder <ar...@pu...> wrote:

>
> some quick comments from my side: XMLVM performs a static analysis of
> the Java classes needed for an application. This also determines which
> methods will need to be called via a vtable (for the C backend). If the
> application changes, this might have impact on the way the base class is
> generated (e.g., if the way a base class method is called changes). For
> the same reason the itable (interface jump table) and constant pool can
> potentially change between builds.
>
> It would be possible to significantly speed up compilation times if one
> were to generate a library that contains a cross-compiled version of all
> JRE classes. This would also require to disable the
> vtable/itable/constant pool optimizations. No static analysis would be
> necessary and the only classes to be cross-compiled would be the
> application itself. While this is possible, this is also *a lot* of
> work. This changes the whole build process and also requires changes to
> the generated Xcode project.
>
> Paul Poley suggested a way of using rsync to copy only files that have
> changed between builds. This will not speed up XMLVM but at least the
> following Xcode build. You can find his suggestion somewhere in the
> archives of this list.
>
> Another quick comment about the --target=xmlvm: this was never fully
> implemented. You can generate .xmlvm but there is no frontend (or
> InputProcess in XMLVM terms) to read these .xmlvm files.
>
> Sorry to be the bearer of bad news but there is no simple solution.
>
> Arno
>
>
> On 3/10/14, 6:38 PM, Steve Hannah wrote:
> > Thanks for the reply.  I don't think static libraries on their own will
> > get me where I need to go.  Have you created static libraries with the c
> > output of xmlvm?  As far as I can tell there are a few aspects that
> > won't work nicely with this .  eg constant pools.
> >
> > Currently the compile time from java source to running on the iOS
> > simulator can be upwards of 10 minutes.   Even if a few java files are
> > changed most of the c files need to be regenerated.  My goal is to get
> > recompilation to under 10 seconds.  I've looked at the problem for a
> > while and believe I can do this.  The xmlvm format was my first choice
> > for an intermediate format because of the ease of transforming XML.  But
> > I can work with java source files also if this won't work.
> >
> > Static libraries may possible as a byproduct of what I will be doing but
> > I don't think they are a solution in themselves.
> >
> > Steve
> >
> > On Mar 10, 2014 6:20 PM, "Panayotis Katsaloulis"
> > <pan...@pa... <mailto:pan...@pa...>> wrote:
> >
> >     I believe the answer to your problem should be to create a static
> >     library instead.
> >     This article might be of help - at least it is the only one that I
> >     followed for my own projects and worked.
> >
> >
> http://www.raywenderlich.com/41377/creating-a-static-library-in-ios-tutorial
> >
> >
> >
> >
> >
> >     On March 11, 2014 at 2:17:21 AM, Steve Hannah (st...@we...
> >     <mailto:st...@we...>) wrote:
> >      > I just want to check if this is supposed to be possible.
> >      >
> >      > Suppose I have a folder with .class files named "java-src".
> >      >
> >      > I run
> >      > xmlvm --in=java-src --out=xmlvm-src --target=xmlvm
> >      >
> >      > Then at a later time
> >      >
> >      > xmlvm --in=xmlvm-src --out=c-src --target=posix
> >      > or
> >      > xmlvm --in=xmlvm-src --out=c-src --target=c
> >      >
> >      > etc..
> >      >
> >      > Is this supposed to be possible?
> >      >
> >      > I'm working on improving build times by introducing some caching
> and
> >      > eliminating inter-class dependencies. My first idea was to
> >     generate .xmlvm
> >      > files as an intermediate representation to treat as a cache.
> >     However I am
> >      > having difficulty generating anything from .xmlvm files. If this
> is
> >      > supposed to be possible, I can provide stack traces etc... If
> >     not, I'll
> >      > proceed to attempt different strategies.
> >      >
> >      > Best regards
> >      >
> >      > Steve
> >      >
> >
> ------------------------------------------------------------------------------
> >      > Learn Graph Databases - Download FREE O'Reilly Book
> >      > "Graph Databases" is the definitive new guide to graph databases
> >     and their
> >      > applications. Written by three acclaimed leaders in the field,
> >      > this first edition is now available. Download your free book
> today!
> >      >
> >
> http://p.sf.net/sfu/13534_NeoTech_______________________________________________
> >      > xmlvm-users mailing list
> >      > xml...@li...
> >     <mailto:xml...@li...>
> >      > https://lists.sourceforge.net/lists/listinfo/xmlvm-users
> >      >
> >
> >     --
> >     Panayotis
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> >     Learn Graph Databases - Download FREE O'Reilly Book
> >     "Graph Databases" is the definitive new guide to graph databases and
> >     their
> >     applications. Written by three acclaimed leaders in the field,
> >     this first edition is now available. Download your free book today!
> >     http://p.sf.net/sfu/13534_NeoTech
> >     _______________________________________________
> >     xmlvm-users mailing list
> >     xml...@li...
> >     <mailto:xml...@li...>
> >     https://lists.sourceforge.net/lists/listinfo/xmlvm-users
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > Learn Graph Databases - Download FREE O'Reilly Book
> > "Graph Databases" is the definitive new guide to graph databases and
> their
> > applications. Written by three acclaimed leaders in the field,
> > this first edition is now available. Download your free book today!
> > http://p.sf.net/sfu/13534_NeoTech
> >
> >
> >
> > _______________________________________________
> > xmlvm-users mailing list
> > xml...@li...
> > https://lists.sourceforge.net/lists/listinfo/xmlvm-users
> >
>
>
> ------------------------------------------------------------------------------
> Learn Graph Databases - Download FREE O'Reilly Book
> "Graph Databases" is the definitive new guide to graph databases and their
> applications. Written by three acclaimed leaders in the field,
> this first edition is now available. Download your free book today!
> http://p.sf.net/sfu/13534_NeoTech
> _______________________________________________
> xmlvm-users mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlvm-users
>

-- 
Steve Hannah
Web Lite Solutions Corp.