#138 Cache/memory paging handling problem for free space

[Bryan Cheng]

Jackcess version: 2.1.5

When there have paged size over the limit (~40MB) of the Jackcess API, the ArrayIndexOutOfBoundsException will be thrown.

To replicate the issue:

1. Create a MS Access file (MDB) vis MS Access or by program.
2. Import some rubbish data to database.i.e. for my test case around 150MB.
3. Delete all rubbish database data from MS Access. The mdb data file will keep the size remain unchanged.
4. Use ImportUtil.importResultSet(ResultSet, Database, tableName, SimpleImportFilter.INSTANCE, true); for input normal records.

The real scenario generate the above problem:

1. Import difference data size (e.g. 20MB, 30MB) to .mdb table. The actual free size of .mdb file will pre-allocate concern amount of disk space (e.g. 25MB) each time.
2. When the pre-allocate disk space for import data is over around 40MB, the paging mechnism will be overflown.

To detect the free paged size of .mdb file, you can clone a copy of .mdb file, and use"Compact and Repair Database" function on MS Access. The difference of file size between current and trimmed .mdb file will be the free paged size.

The stacktrace provided are indicated as follows:
Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
at sun.nio.ch.NativeThreadSet.remove(Unknown Source)
at sun.nio.ch.FileChannelImpl.size(Unknown Source)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:327)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)
at com.healthmarketscience.jackcess.impl.PageChannel.allocateNewPage(PageChannel.java:352)
at com.healthmarketscience.jackcess.impl.TempPageHolder.setNewPage(TempPageHolder.java:104)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.createNewUsageMapPage(UsageMap.java:748)
at com.healthmarketscience.jackcess.impl.UsageMap$ReferenceHandler.addOrRemovePageNumber(UsageMap.java:732)
at com.healthmarketscience.jackcess.impl.UsageMap.removePageNumber(UsageMap.java:322)

[James Ahlborn]

Fixed in trunk, will be in the 2.1.6 release.

[Bryan Cheng]

Any target release date for this bug?