[Bigdata-developers] JVM crash during CI builds

[Bryan T. <br...@sy...>]

All,

I have noticed a number of JVM crashes in recent CI builds.  I've inlined my notes on this below.  I do not have any conclusions yet as to why the builds are failing, that is, whether this has something to do with recent changes in the code base (I have been modifying the cache implementations and added a dependency on high-scale-lib, BrianM has been modifying the test setup).  It is trying to allocate a 500M chunk when it dies.  This is "semi" repeatable and may be related to [1], but the crash occurs with JDK 1.6.0_18.

Please let me know if anyone has any ideas about this, otherwise I will file a bug report against the JVM.

Thanks,
Bryan

[1] http://bugs.sun.com/bugdatabase/view_bug.do;jsessionid=a7084b4c046331411163d306bd6?bug_id=6711109

Here is a trace of the JVM crash while running the service test suite.  I am not sure if this is related to the change in the cache implementations, a spurious failure, or something related to the changes in the test suite setup.

    [junit] Running com.bigdata.cache.TestAll
    [junit] Tests run: 55, Failures: 1, Errors: 9, Time elapsed: 156.78 sec
    [junit] Test com.bigdata.cache.TestAll FAILED
    [junit] dir attribute ignored if running in the same VM
    [junit] Running com.bigdata.io.TestAll
    [junit] Tests run: 86, Failures: 0, Errors: 0, Time elapsed: 55.461 sec
    [junit] dir attribute ignored if running in the same VM
    [junit] Running com.bigdata.net.TestAll
    [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.036 sec
    [junit] dir attribute ignored if running in the same VM
    [junit] Running com.bigdata.config.TestAll
    [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.429 sec
    [junit] dir attribute ignored if running in the same VM
    [junit] Running com.bigdata.util.TestAll
    [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 7.246 sec
    [junit] dir attribute ignored if running in the same VM
    [junit] Running com.bigdata.util.concurrent.TestAll
    [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 0.325 sec
    [junit] dir attribute ignored if running in the same VM
    [junit] Running com.bigdata.striterator.TestAll
    [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 2.679 sec
    [junit] dir attribute ignored if running in the same VM
    [junit] Running com.bigdata.counters.TestAll
    [junit] Tests run: 19, Failures: 0, Errors: 0, Time elapsed: 0.376 sec
    [junit] dir attribute ignored if running in the same VM
    [junit] Running com.bigdata.rawstore.TestAll
    [junit] Tests run: 35, Failures: 0, Errors: 0, Time elapsed: 8.258 sec
    [junit] dir attribute ignored if running in the same VM
    [junit] Running com.bigdata.btree.TestAll
    [junit] Tests run: 431, Failures: 1, Errors: 0, Time elapsed: 957.023 sec
    [junit] Test com.bigdata.btree.TestAll FAILED
    [junit] dir attribute ignored if running in the same VM
    [junit] Running com.bigdata.concurrent.TestAll
    [junit] Tests run: 57, Failures: 0, Errors: 0, Time elapsed: 126.287 sec
    [junit] dir attribute ignored if running in the same VM
    [junit] Running com.bigdata.journal.TestAll
    [junit] Tests run: 623, Failures: 17, Errors: 0, Time elapsed: 285.11 sec
    [junit] Test com.bigdata.journal.TestAll FAILED
    [junit] dir attribute ignored if running in the same VM
    [junit] Running com.bigdata.resources.TestAll
    [junit] Tests run: 24, Failures: 2, Errors: 1, Time elapsed: 36.344 sec
    [junit] Test com.bigdata.resources.TestAll FAILED
    [junit] dir attribute ignored if running in the same VM
    [junit] Running com.bigdata.mdi.TestAll
    [junit] Tests run: 0, Failures: 0, Errors: 0, Time elapsed: 0.005 sec
    [junit] dir attribute ignored if running in the same VM
    [junit] Running com.bigdata.service.TestAll
#
# A fatal error has been detected by the Java Runtime Environment:
#
# java.lang.OutOfMemoryError: requested 536870920 bytes for Chunk::new. Out of swap space?
#
#  Internal Error (allocation.cpp:215), pid=29821, tid=1407789968
#  Error: Chunk::new
#
# JRE version: 6.0_18-b07
# Java VM: Java HotSpot(TM) Server VM (16.0-b13 mixed mode linux-x86 )
# An error report file with more information is saved as:
# /var/lib/hudson/jobs/BIGDATA/workspace/trunk/hs_err_pid29821.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#