Menu

#28 Speed and memory footprint improvement -> No More OOM

closed-fixed
5
2010-06-03
2009-02-11
ezhikov
No

I've started to use onejar in pretty big project and then my single-jar-app became ~60mb onejar started to throw OutOfmemory exception. -Xmx helps but this way seems to be a bit hacky ... OOMs were thrown out of alreadyCached method which is really strange ...
I've looked through JarClassLoader.java and found non-optimal memory usage. Any bytecode in this class is copied into ByteArrayOutputStream, but each time byte[] is necessay method toByteArray() is called ... ByteArrayOutputStream.toByteArray() ALWAYS allocates new buffer .... I've just replaced ByteArrayOutputStream with byte[] .... no more OOMs

Discussion

  • ezhikov

    ezhikov - 2009-02-11

    patch

     
  • P. Simon Tuffs

    P. Simon Tuffs - 2010-05-31

    This looks like a worthwhile stop-gap for One-Jar until I can upgrade the classloader to lazy-load the bytecode from the Jar files. This would slow things down substantially, but classloading is a one-time operation and I don't think it would be too bad. I plan to look ath lazy-loading by One-Jar 1.0, maybe even after 0.97, but I may pull it into 0.97 if this method doesn't scale on large projects.

     
  • P. Simon Tuffs

    P. Simon Tuffs - 2010-05-31
    • assigned_to: nobody --> simontuffs
    • status: open --> open-accepted
     
  • P. Simon Tuffs

    P. Simon Tuffs - 2010-05-31
    • labels: --> Classloading
     
  • P. Simon Tuffs

    P. Simon Tuffs - 2010-06-03
    • status: open-accepted --> closed-fixed
     
  • P. Simon Tuffs

    P. Simon Tuffs - 2010-06-03

    I tried to reproduce this problem using JDK 1.6, and found that OOM problems did not seem to be significant on my test. My test is checked into the tests/test-large-jars directory, and consists of loading the entire JRE into the lib directory and running with a default of 64MB memory. Pure coincidence, the lib directories are just under that in-memory. By modifying the JarClassLoader alreadyCached method to avoid the .toByteArray() call unless the resource exists I expected to see better improvement. However, free memory increased from 43KB to about 120KB, not significant. If you still see this problem with RC3, please let me know.

     

Log in to post a comment.