Re: [sleuthkit-users] GC overhead limit exceeded on pst ingest.

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

The java heap error references the jvm running out of memory. The best way
to test if it is the jvm or the pst file would be to load the pst file in
outlook or a pst viewer and see if it loads completely or errors out.

Im not sure how autopsy, the pst ingest module, or the jvm interaction are
configured in terms of memory management, but how much memory does your
system have?

Pasquale
On Feb 19, 2016 5:23 PM, "MATT PIERCE" <mat...@ad...> wrote:

> I ran a scanpst on the file causing errors and changed the behavior of
> autopsy.  Now I get a Java Heap Space error.
>
>
>
> 2016-02-19 13:59:23.19 org.sleuthkit.autopsy.thunderbirdparser.PstParser
> extractAttachments
>
> WARNING: Failed to extract attachment from pst file.
>
> java.io.FileNotFoundException: B:\test\person test\ModuleOutput\Email
> Parser\3077828-RE: Address: Dell India Private Limited (The filename,
> directory name, or volume label syntax is incorrect):
>
>                 java.io.FileOutputStream.open0(Native Method)
>
>                 java.io.FileOutputStream.open(Unknown Source)
>
>                 java.io.FileOutputStream.<init>(Unknown Source)
>
>                 java.io.FileOutputStream.<init>(Unknown Source)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.saveAttachmentToDisk(PstParser.java:254)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractAttachments(PstParser.java:220)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractEmailMessage(PstParser.java:189)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:149)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.parse(PstParser.java:86)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.processPst(ThunderbirdMboxFileIngestModule.java:142)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.process(ThunderbirdMboxFileIngestModule.java:105)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestPipeline$PipelineModule.process(FileIngestPipeline.java:222)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestPipeline.process(FileIngestPipeline.java:122)
>
>
> org.sleuthkit.autopsy.ingest.DataSourceIngestJob.process(DataSourceIngestJob.java:703)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestTask.execute(FileIngestTask.java:44)
>
>
> org.sleuthkit.autopsy.ingest.IngestManager$ExecuteIngestJobsTask.run(IngestManager.java:989)
>
>
> java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>
>                 java.util.concurrent.FutureTask.run(Unknown Source)
>
>                 java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)
>
>                 java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
>
>                 java.lang.Thread.run(Unknown Source)
>
>
>
> 2016-02-19 13:59:24.272 org.sleuthkit.autopsy.thunderbirdparser.PstParser
> extractAttachments
>
> WARNING: Failed to extract attachment from pst file.
>
> java.io.FileNotFoundException: B:\test\person test\ModuleOutput\Email
> Parser\3259204-RE: Your Approval Required [Requisition : 3299154 - PR
> - 1] (The filename, directory name, or volume label syntax is incorrect):
>
>                 java.io.FileOutputStream.open0(Native Method)
>
>                 java.io.FileOutputStream.open(Unknown Source)
>
>                 java.io.FileOutputStream.<init>(Unknown Source)
>
>                 java.io.FileOutputStream.<init>(Unknown Source)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.saveAttachmentToDisk(PstParser.java:254)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractAttachments(PstParser.java:220)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractEmailMessage(PstParser.java:189)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:149)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.parse(PstParser.java:86)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.processPst(ThunderbirdMboxFileIngestModule.java:142)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.process(ThunderbirdMboxFileIngestModule.java:105)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestPipeline$PipelineModule.process(FileIngestPipeline.java:222)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestPipeline.process(FileIngestPipeline.java:122)
>
>
> org.sleuthkit.autopsy.ingest.DataSourceIngestJob.process(DataSourceIngestJob.java:703)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestTask.execute(FileIngestTask.java:44)
>
>
> org.sleuthkit.autopsy.ingest.IngestManager$ExecuteIngestJobsTask.run(IngestManager.java:989)
>
>
> java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>
>                 java.util.concurrent.FutureTask.run(Unknown Source)
>
>                 java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)
>
>                 java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
>
>                 java.lang.Thread.run(Unknown Source)
>
>
>
> 2016-02-19 13:59:35.273 org.sleuthkit.autopsy.thunderbirdparser.PstParser
> extractAttachments
>
> WARNING: Failed to extract attachment from pst file.
>
> java.io.IOException: attachmentStream invalid (read() fails). File
> Scorecard 2011.txt skipped:
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.saveAttachmentToDisk(PstParser.java:261)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractAttachments(PstParser.java:220)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractEmailMessage(PstParser.java:189)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:149)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.parse(PstParser.java:86)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.processPst(ThunderbirdMboxFileIngestModule.java:142)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.process(ThunderbirdMboxFileIngestModule.java:105)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestPipeline$PipelineModule.process(FileIngestPipeline.java:222)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestPipeline.process(FileIngestPipeline.java:122)
>
>
> org.sleuthkit.autopsy.ingest.DataSourceIngestJob.process(DataSourceIngestJob.java:703)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestTask.execute(FileIngestTask.java:44)
>
>
> org.sleuthkit.autopsy.ingest.IngestManager$ExecuteIngestJobsTask.run(IngestManager.java:989)
>
>
> java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>
>                 java.util.concurrent.FutureTask.run(Unknown Source)
>
>                 java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)
>
>                 java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
>
>                 java.lang.Thread.run(Unknown Source)
>
>
>
> 2016-02-19 13:59:49.05 org.sleuthkit.autopsy.thunderbirdparser.PstParser
> extractAttachments
>
> WARNING: Failed to extract attachment from pst file.
>
> java.io.FileNotFoundException: B:\test\person test\ModuleOutput\Email
> Parser\2854020-C:\Users\person\AppData\Local\Temp\Inv_336802_from_Adobe_Systems_9452.pdf
> (The filename, directory name, or volume label syntax is incorrect):
>
>                 java.io.FileOutputStream.open0(Native Method)
>
>                 java.io.FileOutputStream.open(Unknown Source)
>
>                 java.io.FileOutputStream.<init>(Unknown Source)
>
>                 java.io.FileOutputStream.<init>(Unknown Source)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.saveAttachmentToDisk(PstParser.java:254)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractAttachments(PstParser.java:220)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractEmailMessage(PstParser.java:189)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:149)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.parse(PstParser.java:86)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.processPst(ThunderbirdMboxFileIngestModule.java:142)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.process(ThunderbirdMboxFileIngestModule.java:105)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestPipeline$PipelineModule.process(FileIngestPipeline.java:222)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestPipeline.process(FileIngestPipeline.java:122)
>
>
> org.sleuthkit.autopsy.ingest.DataSourceIngestJob.process(DataSourceIngestJob.java:703)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestTask.execute(FileIngestTask.java:44)
>
>
> org.sleuthkit.autopsy.ingest.IngestManager$ExecuteIngestJobsTask.run(IngestManager.java:989)
>
>
> java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>
>                 java.util.concurrent.FutureTask.run(Unknown Source)
>
>                 java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)
>
>                 java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
>
>                 java.lang.Thread.run(Unknown Source)
>
>
>
> 2016-02-19 14:00:03.618 org.sleuthkit.autopsy.ingest.DataSourceIngestJob
> logIngestModuleErrors
>
> SEVERE: Email Parser experienced an error analyzing LogicalFileSet2
> (jobId=4)
>
> java.lang.OutOfMemoryError: Java heap space:
>
>                 java.lang.StringCoding$StringDecoder.decode(Unknown Source)
>
>                 java.lang.StringCoding.decode(Unknown Source)
>
>                 java.lang.StringCoding.decode(Unknown Source)
>
>                 java.lang.String.<init>(Unknown Source)
>
>                 java.lang.String.<init>(Unknown Source)
>
>                 com.pff.LZFu.decode(LZFu.java:115)
>
>                 com.pff.PSTMessage.getRTFBody(PSTMessage.java:79)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractEmailMessage(PstParser.java:179)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:149)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.parse(PstParser.java:86)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.processPst(ThunderbirdMboxFileIngestModule.java:142)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.process(ThunderbirdMboxFileIngestModule.java:105)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestPipeline$PipelineModule.process(FileIngestPipeline.java:222)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestPipeline.process(FileIngestPipeline.java:122)
>
>
> org.sleuthkit.autopsy.ingest.DataSourceIngestJob.process(DataSourceIngestJob.java:703)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestTask.execute(FileIngestTask.java:44)
>
>
> org.sleuthkit.autopsy.ingest.IngestManager$ExecuteIngestJobsTask.run(IngestManager.java:989)
>
>
> java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>
>                 java.util.concurrent.FutureTask.run(Unknown Source)
>
>                 java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)
>
>                 java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
>
>                 java.lang.Thread.run(Unknown Source)
>
>
>
> 2016-02-19 14:00:03.618 org.sleuthkit.autopsy.ingest.DataSourceIngestJob
> finishFirstStage
>
> INFO: Finished first stage analysis for LogicalFileSet2 (jobId=4)
>
> 2016-02-19 14:00:03.619 org.sleuthkit.autopsy.ingest.DataSourceIngestJob
> finish
>
> INFO: Finished analysis for LogicalFileSet2 (jobId=4)
>
> 2016-02-19 14:00:03.619 org.sleuthkit.autopsy.ingest.IngestManager
> finishIngestJob
>
> INFO: Ingest job 4 completed
>
>
>
> *From:* Simson Garfinkel [mailto:si...@gm...]
> *Sent:* Friday, February 19, 2016 2:50 PM
> *To:* MATT PIERCE <mat...@ad...>
> *Subject:* Re: [sleuthkit-users] GC overhead limit exceeded on pst ingest.
>
>
>
> But autopsy does not display the mail headers. So you are missing valuable
> and important metadata
>
>
>
> ----
>
> Sent from my phone.
>
>
> On Feb 19, 2016, at 3:37 PM, MATT PIERCE <mat...@ad...> wrote:
>
> I'm using Autopsy to scan psts to find wich might contain relevant.
>
>
>
>
>
>
>
> Sent from my Verizon Wireless 4G LTE smartphone
>
>
>
>
>
> -------- Original message --------
>
> From: Simson Garfinkel <si...@gm...>
>
> Date: 2/19/2016 2:35 PM (GMT-06:00)
>
> To: MATT PIERCE <mat...@ad...>
>
> Cc: sle...@li...
>
> Subject: Re: [sleuthkit-users] GC overhead limit exceeded on pst ingest.
>
>
>
> Matt,
>
>
>
> How are you handling the mail headers? Are you ignoring them in your
> investigation?
>
> Sent from my iPhone
>
>
> On Feb 19, 2016, at 1:42 PM, MATT PIERCE <mat...@ad...> wrote:
>
> I’m performing a logical file ingest of a pst file.  I would like to
> search through the contents of this mail file for particular keywords.
> Unfortunately this pst file is generating the error GC overhead limit
> exceeded.  An OST file from the same case is working fine.  Is there
> anything I can do to complete the ingest?  I found a few 9other users have
> experienced this error but no suggestions concerning what to do.
>
>
>
> Forum Post.
>
>
> http://forum.sleuthkit.org/viewtopic.php?f=6&t=2337&p=2478&hilit=gc+overhead#p2478
>
>
>
> Version info.
>
> *Product Version:* Autopsy 4.0.0 (RELEASE)
> *Sleuth Kit Version:* 4.2.0
> *Netbeans RCP Build:* 201411181905
> *Java:* 1.8.0_66; Java HotSpot(TM) 64-Bit Server VM 25.66-b17
> *System:* Windows 7 version 6.1 running on amd64; Cp1252; en_US (autopsy)
>
>
>
> Here is what I believe to be the relevant log entry
>
>
>
> 2016-02-19 12:16:51.052 org.sleuthkit.autopsy.ingest.DataSourceIngestJob
> logIngestModuleErrors
>
> SEVERE: Email Parser experienced an error analyzing LogicalFileSet1
> (jobId=2)
>
> java.lang.OutOfMemoryError: Java heap space:
>
>                 com.pff.LZFu.decode(LZFu.java:60)
>
>                 com.pff.PSTMessage.getRTFBody(PSTMessage.java:79)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.extractEmailMessage(PstParser.java:179)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:149)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.processFolder(PstParser.java:140)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.PstParser.parse(PstParser.java:86)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.processPst(ThunderbirdMboxFileIngestModule.java:142)
>
>
> org.sleuthkit.autopsy.thunderbirdparser.ThunderbirdMboxFileIngestModule.process(ThunderbirdMboxFileIngestModule.java:105)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestPipeline$PipelineModule.process(FileIngestPipeline.java:222)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestPipeline.process(FileIngestPipeline.java:122)
>
>
> org.sleuthkit.autopsy.ingest.DataSourceIngestJob.process(DataSourceIngestJob.java:703)
>
>
> org.sleuthkit.autopsy.ingest.FileIngestTask.execute(FileIngestTask.java:44)
>
>
> org.sleuthkit.autopsy.ingest.IngestManager$ExecuteIngestJobsTask.run(IngestManager.java:989)
>
>
> java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>
>                 java.util.concurrent.FutureTask.run(Unknown Source)
>
>                 java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)
>
>                 java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
>
>                 java.lang.Thread.run(Unknown Source)
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>
> _______________________________________________
> sleuthkit-users mailing list
> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
> http://www.sleuthkit.org
>
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> _______________________________________________
> sleuthkit-users mailing list
> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
> http://www.sleuthkit.org
>
>