[Archive-access-discuss] [wayback] AutoARCIndexThread Crashing

[Ignacio G. <igc...@gm...>]

Hello,

I have the following issue with wayback.
I am trying to add several arcs to wayback, but for some reason some of the
them make the ArcIndexer procedure crash and I get the error: 'Exception in
thread "AutoARCIndexThread" java.lang.OutOfMemoryError: Java heap space'
The collection I was trying to add consisted of 11 arc files with a total
size of ~4.5 Gb
I also tried adding them one by one, and I found out that only some of the
arc files caused wayback to fail.

My first thought was that maybe the arcs were corrupted and Wayback was not
able to index them because of that, but if I use the 'index-client'
application or Heritrix's 'arcreader' script, the contents of the mentioned
arcs are listed properly and no error messages appear.

I have tried with version 0.8 and also with two 0.9 countinous releases and
I've had the same results in all cases...

Does anyone know what the issue may be, or if there is any certain way to
test if an arc file is corrupted and will crash the AutoARCIndexThread?

Thank you.