[sleuthkit-users] Slow Add Image Process Cause

[Luís F. N. <lfc...@gm...>]

Hi,

We have investigated a bit why the add image process is too slow in some
cases. The add image process time seems to be quadratic with the number of
files in the image.

We detected that the function TskDbSqlite::findParObjId(), in
db_sqlite.cpp, is not finding the parent_meta_addr -> parent_file_id
mapping in the local cache for a lot of files, causing it to search for the
mapping in the database (not sure if it is an non-indexed search?)

For testing purposes, we added a "return 1;" line right after the cache
look up, disabling the database look up, and this resulted in great speed
ups:

number of files / default load_db time / patched load_db time
~80.000 / 20min / 2min
~300.000 / 3h / 7min
~700.000 / 48h / 27min

We wonder if it is possible to store all par_meta_addr -> par_id mappings
into local cache (better) or doing an improved (indexed?) search for the
mapping in the database. We think that someone with more knowledge of
load_db code could help a lot here.