#1758 Embedded database extremely slow because of inefficient length measurement

[Martin Desruisseaux]

A call to getConnection() in an embedded database of 32 MB takes 30 seconds. Profiler shows that HSQLSB spends 95% of its time in the org.hsqldb.persist.RAFileInJar constructor, more specifically in the getLength() method. That methods implement an extremely inefficient way to get the length of a JAR entry, reading bytes one by one from an InputStream and counting them. A much more efficient way is to skip a large number of bytes and check how many bytes have been effectively skipped. OpenJDK implements that operation very efficiently, by moving a cursor without actually reading data.

Implementation proposal below. Note: this proposal opportunistically handles exceptions in a more conservative way (Throwable should generally not be caught) and reports the reason why the operation failed.

private long getLength() throws IOException {
    long count = -1;
    try (InputStream fis = open()) {
        do {
            count++;  // Account for the `in.read()` call.
            count += fis.skip(Integer.MAX_VALUE);
        } while (fis.read() >= 0);
    }
    return count;
}

private InputStream open() throws IOException {
    Throwable error = null;
    InputStream fis = null;
    try  {
        fis = getClass().getResourceAsStream(fileName);
        if (fis == null) {
            ClassLoader cl = Thread.currentThread().getContextClassLoader();
            if (cl != null) {
                fis = cl.getResourceAsStream(fileName);
            }
        }
    } catch (RuntimeException e) {
        error = e;
    }
    if (fis == null) {
        throw (IOException) new FileNotFoundException(fileName).initCause(error);
    }
    return fis;
}

private void resetStream() throws IOException {
    if (file != null) {
        file.close();
    }
    file = new DataInputStream(open());
}

[Martin Desruisseaux]

I forgot to add: with this change, the execution time of getLength() on my machine goes down from 30 seconds to 0.02 seconds.

[Fred Toussi]

Thanks. I will check and apply this optimisation.