Re: [sleuthkit-users] GC overhead limit exceeded on pst ingest.

[MATT P. <mat...@ad...>]

I ran a scanpst on the file causing errors and changed the behavior of autopsy.  Now I get a Java Heap Space error.

2016-02-19 13:59:23.19 org.sleuthkit.autopsy.thunderbirdparser.PstParser extractAttachments
WARNING: Failed to extract attachment from pst file.
java.io.FileNotFoundException: B:\test\person test\ModuleOutput\Email Parser\3077828-RE: Address: Dell India Private Limited (The filename, directory name, or volume label syntax is incorrect):
                java.io.FileOutputStream.open0(Native Method)
                java.io.FileOutputStream.open(Unknown Source)
                java.io.FileOutputStream.<init>(Unknown Source)
                java.io.FileOutputStream.<init>(Unknown Source)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.saveAttachmentToDisk(PstParser.java:254)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractAttachments(PstParser.java:220)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractEmailMessage(PstParser.java:189)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:149)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.parse(PstParser.java:86)
                org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.processPst(ThunderbirdMboxFileIngestModule.java:142)
                org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.process(ThunderbirdMboxFileIngestModule.java:105)
                org.sleuthkit.autopsy.ingest.FileIngestPipeline$PipelineModule.process(FileIngestPipeline.java:222)
                org.sleuthkit.autopsy.ingest.FileIngestPipeline.process(FileIngestPipeline.java:122)
                org.sleuthkit.autopsy.ingest.DataSourceIngestJob.process(DataSourceIngestJob.java:703)
                org.sleuthkit.autopsy.ingest.FileIngestTask.execute(FileIngestTask.java:44)
                org.sleuthkit.autopsy.ingest.IngestManager$ExecuteIngestJobsTask.run(IngestManager.java:989)
                java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
                java.util.concurrent.FutureTask.run(Unknown Source)
                java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
                java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
                java.lang.Thread.run(Unknown Source)

2016-02-19 13:59:24.272 org.sleuthkit.autopsy.thunderbirdparser.PstParser extractAttachments
WARNING: Failed to extract attachment from pst file.
java.io.FileNotFoundException: B:\test\person test\ModuleOutput\Email Parser\3259204-RE: Your Approval Required [Requisition : 3299154 - PR       - 1] (The filename, directory name, or volume label syntax is incorrect):
                java.io.FileOutputStream.open0(Native Method)
                java.io.FileOutputStream.open(Unknown Source)
                java.io.FileOutputStream.<init>(Unknown Source)
                java.io.FileOutputStream.<init>(Unknown Source)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.saveAttachmentToDisk(PstParser.java:254)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractAttachments(PstParser.java:220)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractEmailMessage(PstParser.java:189)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:149)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.parse(PstParser.java:86)
                org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.processPst(ThunderbirdMboxFileIngestModule.java:142)
                org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.process(ThunderbirdMboxFileIngestModule.java:105)
                org.sleuthkit.autopsy.ingest.FileIngestPipeline$PipelineModule.process(FileIngestPipeline.java:222)
                org.sleuthkit.autopsy.ingest.FileIngestPipeline.process(FileIngestPipeline.java:122)
                org.sleuthkit.autopsy.ingest.DataSourceIngestJob.process(DataSourceIngestJob.java:703)
                org.sleuthkit.autopsy.ingest.FileIngestTask.execute(FileIngestTask.java:44)
                org.sleuthkit.autopsy.ingest.IngestManager$ExecuteIngestJobsTask.run(IngestManager.java:989)
                java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
                java.util.concurrent.FutureTask.run(Unknown Source)
                java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
                java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
                java.lang.Thread.run(Unknown Source)

2016-02-19 13:59:35.273 org.sleuthkit.autopsy.thunderbirdparser.PstParser extractAttachments
WARNING: Failed to extract attachment from pst file.
java.io.IOException: attachmentStream invalid (read() fails). File Scorecard 2011.txt skipped:
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.saveAttachmentToDisk(PstParser.java:261)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractAttachments(PstParser.java:220)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractEmailMessage(PstParser.java:189)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:149)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.parse(PstParser.java:86)
                org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.processPst(ThunderbirdMboxFileIngestModule.java:142)
                org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.process(ThunderbirdMboxFileIngestModule.java:105)
                org.sleuthkit.autopsy.ingest.FileIngestPipeline$PipelineModule.process(FileIngestPipeline.java:222)
                org.sleuthkit.autopsy.ingest.FileIngestPipeline.process(FileIngestPipeline.java:122)
                org.sleuthkit.autopsy.ingest.DataSourceIngestJob.process(DataSourceIngestJob.java:703)
                org.sleuthkit.autopsy.ingest.FileIngestTask.execute(FileIngestTask.java:44)
                org.sleuthkit.autopsy.ingest.IngestManager$ExecuteIngestJobsTask.run(IngestManager.java:989)
                java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
                java.util.concurrent.FutureTask.run(Unknown Source)
                java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
                java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
                java.lang.Thread.run(Unknown Source)

2016-02-19 13:59:49.05 org.sleuthkit.autopsy.thunderbirdparser.PstParser extractAttachments
WARNING: Failed to extract attachment from pst file.
java.io.FileNotFoundException: B:\test\person test\ModuleOutput\Email Parser\2854020-C:\Users\person\AppData\Local\Temp\Inv_336802_from_Adobe_Systems_9452.pdf (The filename, directory name, or volume label syntax is incorrect):
                java.io.FileOutputStream.open0(Native Method)
                java.io.FileOutputStream.open(Unknown Source)
                java.io.FileOutputStream.<init>(Unknown Source)
                java.io.FileOutputStream.<init>(Unknown Source)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.saveAttachmentToDisk(PstParser.java:254)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractAttachments(PstParser.java:220)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractEmailMessage(PstParser.java:189)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:149)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.parse(PstParser.java:86)
                org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.processPst(ThunderbirdMboxFileIngestModule.java:142)
                org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.process(ThunderbirdMboxFileIngestModule.java:105)
                org.sleuthkit.autopsy.ingest.FileIngestPipeline$PipelineModule.process(FileIngestPipeline.java:222)
                org.sleuthkit.autopsy.ingest.FileIngestPipeline.process(FileIngestPipeline.java:122)
                org.sleuthkit.autopsy.ingest.DataSourceIngestJob.process(DataSourceIngestJob.java:703)
                org.sleuthkit.autopsy.ingest.FileIngestTask.execute(FileIngestTask.java:44)
                org.sleuthkit.autopsy.ingest.IngestManager$ExecuteIngestJobsTask.run(IngestManager.java:989)
                java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
                java.util.concurrent.FutureTask.run(Unknown Source)
                java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
                java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
                java.lang.Thread.run(Unknown Source)

2016-02-19 14:00:03.618 org.sleuthkit.autopsy.ingest.DataSourceIngestJob logIngestModuleErrors
SEVERE: Email Parser experienced an error analyzing LogicalFileSet2 (jobId=4)
java.lang.OutOfMemoryError: Java heap space:
                java.lang.StringCoding$StringDecoder.decode(Unknown Source)
                java.lang.StringCoding.decode(Unknown Source)
                java.lang.StringCoding.decode(Unknown Source)
                java.lang.String.<init>(Unknown Source)
                java.lang.String.<init>(Unknown Source)
                com.pff.LZFu.decode(LZFu.java:115)
                com.pff.PSTMessage.getRTFBody(PSTMessage.java:79)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractEmailMessage(PstParser.java:179)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:149)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.parse(PstParser.java:86)
                org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.processPst(ThunderbirdMboxFileIngestModule.java:142)
                org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.process(ThunderbirdMboxFileIngestModule.java:105)
                org.sleuthkit.autopsy.ingest.FileIngestPipeline$PipelineModule.process(FileIngestPipeline.java:222)
                org.sleuthkit.autopsy.ingest.FileIngestPipeline.process(FileIngestPipeline.java:122)
                org.sleuthkit.autopsy.ingest.DataSourceIngestJob.process(DataSourceIngestJob.java:703)
                org.sleuthkit.autopsy.ingest.FileIngestTask.execute(FileIngestTask.java:44)
                org.sleuthkit.autopsy.ingest.IngestManager$ExecuteIngestJobsTask.run(IngestManager.java:989)
                java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
                java.util.concurrent.FutureTask.run(Unknown Source)
                java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
                java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
                java.lang.Thread.run(Unknown Source)

2016-02-19 14:00:03.618 org.sleuthkit.autopsy.ingest.DataSourceIngestJob finishFirstStage
INFO: Finished first stage analysis for LogicalFileSet2 (jobId=4)
2016-02-19 14:00:03.619 org.sleuthkit.autopsy.ingest.DataSourceIngestJob finish
INFO: Finished analysis for LogicalFileSet2 (jobId=4)
2016-02-19 14:00:03.619 org.sleuthkit.autopsy.ingest.IngestManager finishIngestJob
INFO: Ingest job 4 completed

From: Simson Garfinkel [mailto:si...@gm...]
Sent: Friday, February 19, 2016 2:50 PM
To: MATT PIERCE <mat...@ad...>
Subject: Re: [sleuthkit-users] GC overhead limit exceeded on pst ingest.

But autopsy does not display the mail headers. So you are missing valuable and important metadata

----
Sent from my phone.

On Feb 19, 2016, at 3:37 PM, MATT PIERCE <mat...@ad...<mailto:mat...@ad...>> wrote:
I'm using Autopsy to scan psts to find wich might contain relevant.



Sent from my Verizon Wireless 4G LTE smartphone


-------- Original message --------
From: Simson Garfinkel <si...@gm...<mailto:si...@gm...>>
Date: 2/19/2016 2:35 PM (GMT-06:00)
To: MATT PIERCE <mat...@ad...<mailto:mat...@ad...>>
Cc: sle...@li...<mailto:sle...@li...>
Subject: Re: [sleuthkit-users] GC overhead limit exceeded on pst ingest.

Matt,

How are you handling the mail headers? Are you ignoring them in your investigation?

Sent from my iPhone

On Feb 19, 2016, at 1:42 PM, MATT PIERCE <mat...@ad...<mailto:mat...@ad...>> wrote:
I’m performing a logical file ingest of a pst file.  I would like to search through the contents of this mail file for particular keywords.  Unfortunately this pst file is generating the error GC overhead limit exceeded.  An OST file from the same case is working fine.  Is there anything I can do to complete the ingest?  I found a few 9other users have experienced this error but no suggestions concerning what to do.

Forum Post.
http://forum.sleuthkit.org/viewtopic.php?f=6&t=2337&p=2478&hilit=gc+overhead#p2478

Version info.
Product Version: Autopsy 4.0.0 (RELEASE)
Sleuth Kit Version: 4.2.0
Netbeans RCP Build: 201411181905
Java: 1.8.0_66; Java HotSpot(TM) 64-Bit Server VM 25.66-b17
System: Windows 7 version 6.1 running on amd64; Cp1252; en_US (autopsy)

Here is what I believe to be the relevant log entry

2016-02-19 12:16:51.052 org.sleuthkit.autopsy.ingest.DataSourceIngestJob logIngestModuleErrors
SEVERE: Email Parser experienced an error analyzing LogicalFileSet1 (jobId=2)
java.lang.OutOfMemoryError: Java heap space:
                com.pff.LZFu.decode(LZFu.java:60)
                com.pff.PSTMessage.getRTFBody(PSTMessage.java:79)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractEmailMessage(PstParser.java:179)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:149)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
                org.sleuthkit.autopsy.thunderbirdparser.PstParser.parse(PstParser.java:86)
                org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.processPst(ThunderbirdMboxFileIngestModule.java:142)
                org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.process(ThunderbirdMboxFileIngestModule.java:105)
                org.sleuthkit.autopsy.ingest.FileIngestPipeline$PipelineModule.process(FileIngestPipeline.java:222)
                org.sleuthkit.autopsy.ingest.FileIngestPipeline.process(FileIngestPipeline.java:122)
                org.sleuthkit.autopsy.ingest.DataSourceIngestJob.process(DataSourceIngestJob.java:703)
                org.sleuthkit.autopsy.ingest.FileIngestTask.execute(FileIngestTask.java:44)
                org.sleuthkit.autopsy.ingest.IngestManager$ExecuteIngestJobsTask.run(IngestManager.java:989)
                java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
                java.util.concurrent.FutureTask.run(Unknown Source)
                java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
                java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
                java.lang.Thread.run(Unknown Source)
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
sleuthkit-users mailing list
https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
http://www.sleuthkit.org