[sleuthkit-users] Slow Ingest

[MATT P. <mat...@ad...>]

I'm working a case and again have issues with performance using Autopsy.  I have setup a dedicated server for running Autopsy.  In two days of ingest I'm at 45%.  In 8 hours it has only progressed 7%. I was hoping someone can spot where my bottle neck is?

The system has dual Xeon x5550 2.66 quad core processors.  24 GB RAM.  Windows 2012 R2 x64.   The case drive is an OCZ Revo 350 PCIe SSD.  Autopsy is loaded on a Raid0 15k SAS volume.

Autopsy Load.
Product Version: Autopsy 4.1.1 (RELEASE) Sleuth Kit Version: 4.2.0 Netbeans RCP Build: 201510222201 Java: 1.8.0_92; Java HotSpot(TM) 64-Bit Server VM 25.92-b14 System: Windows Server 2012 R2 version 6.3 running on amd64; Cp1252; en_US (autopsy)

The image is from a Windows 7 workstation.  FTKimager took the disk image in E01 format.  I have the NSRL known good hash database loaded.  I've set number of threads to 12 as suggested by the Options dialog.  I'm running the default ingest process with no 3rd party modules.

Performance Diagnostics
[cid:image001.png@01D20EA9.99419680]

Ingest Progress Snapshot

1              IDLE                                       Wed Sep 14 01:10:44 CDT 2016   15:49:08.024       0
2              Keyword Search               2016-08-31-1-1.E01          image1.emf        Wed Sep 14 16:59:26 CDT 2016   0:00:25.759                2
3              IDLE                                       Wed Sep 14 16:59:26 CDT 2016   0:00:25.758         0
4              Keyword Search               2016-08-31-1-1.E01          image1.emf        Wed Sep 14 16:59:26 CDT 2016   0:00:25.798                2
5              IDLE                                       Wed Sep 14 16:59:26 CDT 2016   0:00:25.764         0
6              Keyword Search               2016-08-31-1-1.E01          image1.emf        Wed Sep 14 16:59:26 CDT 2016   0:00:25.759                2
7              Keyword Search               2016-08-31-1-1.E01          image1.emf        Wed Sep 14 16:59:26 CDT 2016   0:00:26.155                2
8              Keyword Search               2016-08-31-1-1.E01          image1.emf        Wed Sep 14 16:59:26 CDT 2016   0:00:26.132                2
9              IDLE                                       Wed Sep 14 16:59:26 CDT 2016   0:00:25.742         0
10           IDLE                                       Wed Sep 14 16:59:26 CDT 2016   0:00:25.812         0
11           Keyword Search               2016-08-31-1-1.E01          image1.emf        Wed Sep 14 16:59:26 CDT 2016   0:00:26.120                2
12           IDLE                                       Wed Sep 14 16:59:26 CDT 2016   0:00:25.811         0
13           Keyword Search               2016-08-31-1-1.E01          image1.emf        Wed Sep 14 16:59:26 CDT 2016   0:00:26.155                2

2              2016-08-31-1-1.E01          15:23:00               193034  2.0938940654524942       84           2              36           34           0

Keyword Search               155:52:59.626 (36%)
File Type Identification  139:55:17.562 (32%)
Hash Lookup      48:49:44.445 (11%)
Embedded File Extractor              46:01:20.323 (10%)
Email Parser       28:37:58.461 (6%)
Extension Mismatch Detector    4:51:51.763 (1%)
Exif Parser           1:41:39.597 (0%)
PhotoRec Carver              0:00:04.762 (0%)
Interesting Files Identifier            0:00:00.105 (0%)

I get a bunch of errors but that is pretty typical.

[cid:image002.png@01D20EAA.066E5950]