Re: [Archive-access-discuss] Workaround for locationDBResourceStore bug in 1.7.1-SNAPSHOT

[Søren V. C. <sv...@kb...>]

Hi all.

I have found the problem. It was in  the wayback-core module in the class org.archive.wayback.resourcestore.resourcefile.ResourceFactory.getResource(File
file, long offset)

The method-call “ARCReaderFactory.get(path.getName(), is, false);”
assumes, that the file is a gzipped ARC-file, even though the getResource method should work for both compressed
and uncompressed arc-files

The solution is to replace this call with ARCReaderFactory.get(file, offset).
This makes the method work for both compressed and uncompressed arc-files.


/Søren V. Carlsen  (Royal Library, Copenhagen)

Fra: Søren Vejrup Carlsen [mailto:sv...@kb...]
Sendt: 1. februar 2013 12:32
Til: arc...@li...
Emne: [Archive-access-discuss] Workaround for locationDBResourceStore bug in 1.7.1-SNAPSHOT

Hi all.
I have installed wayback 1.7.1-SNAPSHOT, built myself directly from the pom.xml after downloading the code from https://github.com/internetarchive/wayback

I’m using the locationDBResourceStore that the CDXCollection.xml uses, and it can find the correct files from the CDX.
However, it fails to extract the record, as it somehow assumes that all files are GZIPPED, and when it is now, it fails miserably with the following log-entries:

Jan 31, 2013 6:49:18 PM org.archive.wayback.resourcestore.resourcefile.ResourceFactory getResource
INFO: Fetching: /home/prod/wayback/arcs/83807-92-0000-1.arc : 39136770
Jan 31, 2013 6:49:18 PM org.archive.wayback.resourcestore.resourcefile.ResourceFactory getResource
WARNING: ResourceNotAvailable for /home/prod/wayback/arcs/83807-92-0000-1.arc Not in GZIP format
Jan 31, 2013 6:49:18 PM org.archive.wayback.resourcestore.LocationDBResourceStore retrieveResource
INFO: Unable to retrieve /home/prod/wayback/arcs/83807-92-0000-1.arc - java.util.zip.ZipException: Not in GZIP format
Jan 31, 2013 6:49:18 PM org.archive.wayback.webapp.AccessPoint handleReplay
WARNING: (1)LOADFAIL: /home/prod/wayback/arcs/83807-92-0000-1.arc - java.util.zip.ZipException: Not in GZIP format /20100107153228/http://www2.kb.dk/elib/mss/skatte/aeldre_danske/ln185.htm
Can anyone help me here?

/Søren

---------------------------------------------------------------------------
Søren Vejrup Carlsen, Department of Digital Preservation, Royal Library, Copenhagen, Denmark
tlf: (+45) 33 47 48 41
email: sv...@kb...<mailto:sv...@kb...>
----------------------------------------------------------------------------
Non omnia possumus omnes
--- Macrobius, Saturnalia, VI, 1, 35 -------