sleuthkit-users Mailing List for The Sleuth Kit (Page 38)
Brought to you by:
carrier
You can subscribe to this list here.
2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(6) |
Aug
|
Sep
(11) |
Oct
(5) |
Nov
(4) |
Dec
|
---|---|---|---|---|---|---|---|---|---|---|---|---|
2003 |
Jan
(1) |
Feb
(20) |
Mar
(60) |
Apr
(40) |
May
(24) |
Jun
(28) |
Jul
(18) |
Aug
(27) |
Sep
(6) |
Oct
(14) |
Nov
(15) |
Dec
(22) |
2004 |
Jan
(34) |
Feb
(13) |
Mar
(28) |
Apr
(23) |
May
(27) |
Jun
(26) |
Jul
(37) |
Aug
(19) |
Sep
(20) |
Oct
(39) |
Nov
(17) |
Dec
(9) |
2005 |
Jan
(45) |
Feb
(43) |
Mar
(66) |
Apr
(36) |
May
(19) |
Jun
(64) |
Jul
(10) |
Aug
(11) |
Sep
(35) |
Oct
(6) |
Nov
(4) |
Dec
(13) |
2006 |
Jan
(52) |
Feb
(34) |
Mar
(39) |
Apr
(39) |
May
(37) |
Jun
(15) |
Jul
(13) |
Aug
(48) |
Sep
(9) |
Oct
(10) |
Nov
(47) |
Dec
(13) |
2007 |
Jan
(25) |
Feb
(4) |
Mar
(2) |
Apr
(29) |
May
(11) |
Jun
(19) |
Jul
(13) |
Aug
(15) |
Sep
(30) |
Oct
(12) |
Nov
(10) |
Dec
(13) |
2008 |
Jan
(2) |
Feb
(54) |
Mar
(58) |
Apr
(43) |
May
(10) |
Jun
(27) |
Jul
(25) |
Aug
(27) |
Sep
(48) |
Oct
(69) |
Nov
(55) |
Dec
(43) |
2009 |
Jan
(26) |
Feb
(36) |
Mar
(28) |
Apr
(27) |
May
(55) |
Jun
(9) |
Jul
(19) |
Aug
(16) |
Sep
(15) |
Oct
(17) |
Nov
(70) |
Dec
(21) |
2010 |
Jan
(56) |
Feb
(59) |
Mar
(53) |
Apr
(32) |
May
(25) |
Jun
(31) |
Jul
(36) |
Aug
(11) |
Sep
(37) |
Oct
(19) |
Nov
(23) |
Dec
(6) |
2011 |
Jan
(21) |
Feb
(20) |
Mar
(30) |
Apr
(30) |
May
(74) |
Jun
(50) |
Jul
(34) |
Aug
(34) |
Sep
(12) |
Oct
(33) |
Nov
(10) |
Dec
(8) |
2012 |
Jan
(23) |
Feb
(57) |
Mar
(26) |
Apr
(14) |
May
(27) |
Jun
(27) |
Jul
(60) |
Aug
(88) |
Sep
(13) |
Oct
(36) |
Nov
(97) |
Dec
(85) |
2013 |
Jan
(60) |
Feb
(24) |
Mar
(43) |
Apr
(32) |
May
(22) |
Jun
(38) |
Jul
(51) |
Aug
(50) |
Sep
(76) |
Oct
(65) |
Nov
(25) |
Dec
(30) |
2014 |
Jan
(19) |
Feb
(41) |
Mar
(43) |
Apr
(28) |
May
(61) |
Jun
(12) |
Jul
(10) |
Aug
(37) |
Sep
(76) |
Oct
(31) |
Nov
(41) |
Dec
(12) |
2015 |
Jan
(33) |
Feb
(28) |
Mar
(53) |
Apr
(22) |
May
(29) |
Jun
(20) |
Jul
(15) |
Aug
(17) |
Sep
(52) |
Oct
(3) |
Nov
(18) |
Dec
(21) |
2016 |
Jan
(20) |
Feb
(8) |
Mar
(21) |
Apr
(7) |
May
(13) |
Jun
(35) |
Jul
(34) |
Aug
(11) |
Sep
(14) |
Oct
(22) |
Nov
(31) |
Dec
(23) |
2017 |
Jan
(20) |
Feb
(7) |
Mar
(5) |
Apr
(6) |
May
(6) |
Jun
(22) |
Jul
(11) |
Aug
(16) |
Sep
(8) |
Oct
(1) |
Nov
(1) |
Dec
(1) |
2018 |
Jan
|
Feb
|
Mar
(16) |
Apr
(2) |
May
(6) |
Jun
(5) |
Jul
|
Aug
(2) |
Sep
(4) |
Oct
|
Nov
(16) |
Dec
(13) |
2019 |
Jan
|
Feb
(1) |
Mar
(25) |
Apr
(9) |
May
(2) |
Jun
(1) |
Jul
(1) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2020 |
Jan
(2) |
Feb
|
Mar
(1) |
Apr
|
May
(1) |
Jun
(3) |
Jul
(2) |
Aug
|
Sep
|
Oct
(5) |
Nov
|
Dec
|
2021 |
Jan
|
Feb
|
Mar
(1) |
Apr
|
May
|
Jun
(4) |
Jul
(1) |
Aug
|
Sep
(1) |
Oct
|
Nov
(1) |
Dec
|
2022 |
Jan
|
Feb
(2) |
Mar
|
Apr
|
May
(2) |
Jun
|
Jul
(3) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2023 |
Jan
(2) |
Feb
|
Mar
(1) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(1) |
Nov
|
Dec
|
2024 |
Jan
|
Feb
(3) |
Mar
|
Apr
(1) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2025 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(1) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: Jason L. <jle...@ba...> - 2014-09-09 17:07:24
|
Basis Technology has new Autopsy modules on its website. All of these work on Autopsy 3.1.0. Law Enforcement Bundle v1.0 This new module allows you to use Autopsy along with Project Vic and C4P/All databases. These databases are used to share hash values for child exploitation cases. This module allows Autopsy to flag files that have hashes in one of these databases. This free module is restricted to law enforcement users. http://www.basistech.com/digital-forensics/autopsy/le-bundle/ Video Triage v1.1 This module allows you to see what kind of images are in a long video. Instead of needing to watch it, this module will show you thumbnail images at periodic moments in the video. It integrates directly into the Autopsy UI. This module is a free download. New in this version: Updated to run in Autopsy 3.1 http://www.basistech.com/digital-forensics/autopsy/video-triage/ Text Gister v0.2 This module allows you to get a summary of people, places, and concepts that are mentioned in a non-English document. It’s use case is to help prioritize documents in a time critical situation so that a translator reviews the most important ones first. It integrates text analytic components with Basis Technologies. http://www.basistech.com/digital-forensics/autopsy/text-gisting/ ------------------------------------------------ Jason Letourneau Product Manager, Digital Forensics Basis Technology jle...@ba... 617-386-2000 ext. 152 |
From: Luís F. N. <lfc...@gm...> - 2014-09-05 23:37:29
|
Hi Stuart, Yes, I think so. I can read file contents from some starting offset within the file, but did not know how to query the file data runs. The API enables to convert a virtual file (eg. unallocated) offset to an image offset, but not a regular file offset. I think the idea to sort by file starting offset before doing any king of processing with the files will result in great speedups when ingesting images stored into spinning magnetic drives, as said by Simson. Luis 2014-09-05 20:53 GMT-03:00 Stuart Maclean <st...@ap...>: > On 09/05/2014 04:02 PM, Luís Filipe Nassif wrote: > >> Hi Simson, >> >> I have had thoughts about implementing this "sort by sector number of >> first run" approach in a forensic tool based on TskJavaBindings, but I did >> not see how to get the file first sector number through the API. Do you >> know if it is possible with tsk java bindings? >> >> Hi Luis, I have slowly been developing my own set of Java bindings to > tsk. The ones that exist seem to only be for extraction of data from some > db?? I wanted to use Java in the actual data acquisition phase. I have > yet to upload it to github but will do so shortly. > > Stuart > > |
From: Luís F. N. <lfc...@gm...> - 2014-09-05 23:02:09
|
Hi Simson, I have had thoughts about implementing this "sort by sector number of first run" approach in a forensic tool based on TskJavaBindings, but I did not see how to get the file first sector number through the API. Do you know if it is possible with tsk java bindings? Regards, Luis Nassif 2014-09-04 20:13 GMT-03:00 Simson Garfinkel <si...@ac...>: > Hi Stuart. > > You are correct — I put this in numerous presentations but never published > it. > > The MD5 algorithm won't let you combine a partial hash from the middle of > the file with one from the beginning. You need to start at the beginning > and hash through the end. (That's one of the many problems with MD5 for > forensics, BTW.) So I believe that the only approach is sorting the files > by the sector number of the first run, and just leaving it at that. > > I saw speedup with both HDs and SSDs, strangely enough, but not as much > with SSDs. There may be a prefetch thing going on here. > > I think that the Autopsy framework should hash this way, but currently it > doesn't. On the other hand, it may be more useful to hash based on the > "importance" of the files. > > Simson > > > > > On Sep 4, 2014, at 7:04 PM, Stuart Maclean <st...@ap...> > wrote: > > > I am tracking recent efforts in STIX and Cybox and all things Mitre. > > One indicator of compromise is an md5 hash of some file. Presumably you > > compare the hash with all files on some file system to see if there is a > > match. Obviously this requires a walk of the host fs, using e.g. fls or > > fiwalk or the tsk library in general. > > > > Is this a common activity, the hashing of a complete filesystem that > > is? If yes, some experiments I have done with minimising total disk > > seek time by ordering Runs, reading content from the ordered Runs and > > piecing each file's hash back together would show that this is indeed a > > worthy optimization since it can decrease the time spent deriving the > > full hash table considerably. > > > > I did see a slide deck by Simson G where he alluded to a similar win > > situation when disk reads are ordered so as to minimise seek time, but > > wonder if much has been published on the topic, specifically relating to > > the digital forensics arena, i.e. when an entire file system contents is > > to be read in a single pass, for the purposes of producing an 'md5 -> > > file path' map. > > > > Opinions and comments welcomed. > > > > Stuart > > > > > > > ------------------------------------------------------------------------------ > > Slashdot TV. > > Video for Nerds. Stuff that matters. > > http://tv.slashdot.org/ > > _______________________________________________ > > sleuthkit-users mailing list > > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users > > http://www.sleuthkit.org > > > > ------------------------------------------------------------------------------ > Slashdot TV. > Video for Nerds. Stuff that matters. > http://tv.slashdot.org/ > _______________________________________________ > sleuthkit-users mailing list > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users > http://www.sleuthkit.org > |
From: anthony s. <ant...@gm...> - 2014-09-05 19:22:46
|
Good afternoon, I’m starting to use Autopsy as my main suite vs EnCase and since my IEF trial ran out, but it was pointed out to me that file metadata is not included in the html report. Also, there is no hash in the report. I’m finding that the reporting is not very flexible at this point. What good is separating the evidence into separate tags if its just going to report it all under one “tagged files” link? Any word on a fix for this? I tried using the excel report but it puts in more info into the report than I want it to. What are others doing to work around this? Thank you AJS |
From: Simson G. <si...@ac...> - 2014-09-05 19:14:01
|
As I indicated, I spent a significant amount of time looking at this, and decided that simply sorting by the block number of the first file fragment and then imaging each file in order provided excellent speed up. There was no need to do any in-memory buffering. On Sep 5, 2014, at 3:41 PM, Stuart Maclean <st...@ap...> wrote: > On 09/04/2014 06:01 PM, Simson Garfinkel wrote: >> On Sep 4, 2014, at 8:51 PM, RB <ao...@gm...> wrote: >>> Although md5 is not a subdivisible hash (as Simson pointed out), one >>> could conceivably still do a single-pass checksum of a filesystem, the >>> tradeoff would be the memory consumption of tens of thousands of >>> "simultaneous" checksum calculation states. >> >> This doesn't work unless you are prepared to buffer the later fragments of a file when they appear on disk before earlier fragments. So in the worst case, you need to hold the entire disk in RAM. > I have been experimenting with exactly this approach. And yes, you are right, in the worst case there is simply too much to buffer. In a 250GB ext4 image, one file was 8GB. Its block allocation was such that I struggled to buffer 4GB of it, at which point my Java VM collapsed out-of-memory. > > I guess we could use a hybrid approach of all files under some size limit use the 'in block order' logic for hashing, with monster files defaulting to the 'regular, file-offset' logic. Somehow I think this will largely defeat the whole point of the exercise and negate most of the time gains. > > Another approach would be to externally store the 'pending data', which might be feasible if you had some Live CD with the tools plus some raw (i.e. no file system) usb or other data drive to use for scratch storage. > > Stuart > > Stuart |
From: Stuart M. <st...@ap...> - 2014-09-05 19:04:08
|
On 09/04/2014 06:01 PM, Simson Garfinkel wrote: > On Sep 4, 2014, at 8:51 PM, RB <ao...@gm...> wrote: >> Although md5 is not a subdivisible hash (as Simson pointed out), one >> could conceivably still do a single-pass checksum of a filesystem, the >> tradeoff would be the memory consumption of tens of thousands of >> "simultaneous" checksum calculation states. > > This doesn't work unless you are prepared to buffer the later fragments of a file when they appear on disk before earlier fragments. So in the worst case, you need to hold the entire disk in RAM. I have been experimenting with exactly this approach. And yes, you are right, in the worst case there is simply too much to buffer. In a 250GB ext4 image, one file was 8GB. Its block allocation was such that I struggled to buffer 4GB of it, at which point my Java VM collapsed out-of-memory. I guess we could use a hybrid approach of all files under some size limit use the 'in block order' logic for hashing, with monster files defaulting to the 'regular, file-offset' logic. Somehow I think this will largely defeat the whole point of the exercise and negate most of the time gains. Another approach would be to externally store the 'pending data', which might be feasible if you had some Live CD with the tools plus some raw (i.e. no file system) usb or other data drive to use for scratch storage. Stuart Stuart |
From: Stuart M. <st...@ap...> - 2014-09-05 17:09:35
|
Hi all, I'm glad to have provoked some conversation on the merits (or otherwise!) of md5 sums as useful representations of the state of a file system. Can anyone enlighten me as to the meaning of the 'flags' member in a TSK_FS_ATTR_RUN? Specifically, what does this comment mean? TSK_FS_ATTR_RUN_FLAG_FILLER = 0x01, ///< Entry is a filler for a run that has not been seen yet in the processing (or has been lost) In a fs I am walking and inspecting the runs for, I am seeing run structs with addr 0 and flags 1. I was under the impression that any run address of 0 represented a 'missing run' i.e. that this part of the file content is N zeros, where N = run.length * fs.blocksize. I presume that would be the case were the run flags value 2: TSK_FS_ATTR_RUN_FLAG_SPARSE = 0x02 ///< Entry is a sparse run where all data in the run is zeros If I use istat, I can see inodes which have certain 'Direct Blocks' of value 0, and when I see M consecutive 0 blocks that matches up to a 'missing run' when inspecting the runs using the tsk lib (actually my tsk4jJava binding, which is now finally showing its worth since I can do all data structure manipulation in Java, nicer than in C, for me at least). I am worried at being 'filler' and not 'sparse', the partial file content represented by the run(s) with addr 0 is not necessarily a sequence of zeros. Anyone shed light on this? Brian? Thanks Stuart |
From: RB <ao...@gm...> - 2014-09-05 01:21:19
|
On Thu, Sep 4, 2014 at 7:15 PM, Simson Garfinkel <si...@ac...> wrote: > You are confused between the physical layout of the disk and the logical layout of the files. <snip> > The problem is that files are fragmented, and frequently the second fragment of a file comes earlier on the disk than the first fragment. Thanks for your patience, I was indeed failing to take this into account. |
From: Simson G. <si...@ac...> - 2014-09-05 01:15:59
|
On Sep 4, 2014, at 9:11 PM, RB <ao...@gm...> wrote: > On Thu, Sep 4, 2014 at 7:01 PM, Simson Garfinkel <si...@ac...> wrote: >> This doesn't work unless you are prepared to buffer the later fragments of a file when they appear on disk before earlier fragments. So in the worst case, you need to hold the entire disk in RAM. > > Perhaps I'm being dense, but "dd if=file | md5sum - " in no way holds > the entire file in RAM, and the process can be slept/interrupted/etc; > all this means that md5 can be calculated over a stream. You are confused between the physical layout of the disk and the logical layout of the files. You have proposed reading the disk in physical block order. If you are reading the disk in block order, what happens if you have a 30GB file where the first block of the file is at the end of the disk and the rest of the file is at the beginning? You have to buffer the portions of the file that come first on the disk but logically later in the file. Then when you reach the beginning of the file (at the end of the disk) you can start hashing. The problem is that files are fragmented, and frequently the second fragment of a file comes earlier on the disk than the first fragment. > > Looking at the API for Perl & Python MD5 libraries (expected to be the > simplest), they have standard functionality for adding data to a hash > object, and I don't expect it holds that in memory either. This would > mean you should be able to make a linear scan through the disk and, as > you read blocks associated with a file, append them to the md5 object > for that file, and move on. You'd have a lot of md5 objects > in-memory, but it shouldn't be of a size equivalent to the entire > [used] disk. |
From: RB <ao...@gm...> - 2014-09-05 01:11:34
|
On Thu, Sep 4, 2014 at 7:01 PM, Simson Garfinkel <si...@ac...> wrote: > This doesn't work unless you are prepared to buffer the later fragments of a file when they appear on disk before earlier fragments. So in the worst case, you need to hold the entire disk in RAM. Perhaps I'm being dense, but "dd if=file | md5sum - " in no way holds the entire file in RAM, and the process can be slept/interrupted/etc; all this means that md5 can be calculated over a stream. Looking at the API for Perl & Python MD5 libraries (expected to be the simplest), they have standard functionality for adding data to a hash object, and I don't expect it holds that in memory either. This would mean you should be able to make a linear scan through the disk and, as you read blocks associated with a file, append them to the md5 object for that file, and move on. You'd have a lot of md5 objects in-memory, but it shouldn't be of a size equivalent to the entire [used] disk. |
From: Simson G. <si...@ac...> - 2014-09-05 01:01:01
|
On Sep 4, 2014, at 8:51 PM, RB <ao...@gm...> wrote: > > Although md5 is not a subdivisible hash (as Simson pointed out), one > could conceivably still do a single-pass checksum of a filesystem, the > tradeoff would be the memory consumption of tens of thousands of > "simultaneous" checksum calculation states. This doesn't work unless you are prepared to buffer the later fragments of a file when they appear on disk before earlier fragments. So in the worst case, you need to hold the entire disk in RAM. |
From: RB <ao...@gm...> - 2014-09-05 00:51:31
|
On Thu, Sep 4, 2014 at 5:04 PM, Stuart Maclean <st...@ap...> wrote: > Is this a common activity, the hashing of a complete filesystem that > is? If yes, some experiments I have done with minimising total disk > seek time by ordering Runs, reading content from the ordered Runs and > piecing each file's hash back together would show that this is indeed a > worthy optimization since it can decrease the time spent deriving the > full hash table considerably. Yes, this is a question we've actually discussed on this list in recent memory. Fiwalk/DFXML is great for automation, but you can use "tsk_gettimes -m" to both pull listings and checksums at the same time for a quick win. The output is in traditional bodyfile form with the md5 field actually populated (instead of being "0"). I've incorporated it into a script that does other things (pulls other useful files from the disk), but all told it takes 8-40 minutes (averaging around 20) to burn through an average 120-300GB disk, usually CPU or IOPS-bound. This is, as you indirectly noted, heavily affected by fragmentation. Although md5 is not a subdivisible hash (as Simson pointed out), one could conceivably still do a single-pass checksum of a filesystem, the tradeoff would be the memory consumption of tens of thousands of "simultaneous" checksum calculation states. |
From: Simson G. <si...@ac...> - 2014-09-05 00:22:25
|
Yes, fiwalk hashes in SleuthKit order. If you want to hash in block order you need to generate the DFXML for the entire drive, sort by the index of the first On Sep 4, 2014, at 7:53 PM, Stuart Maclean <st...@ap...> wrote: > On 09/04/2014 03:46 PM, Simson Garfinkel wrote: >> Hi Stuart. >> >> You are correct — I put this in numerous presentations but never published it. >> >> The MD5 algorithm won't let you combine a partial hash from the middle of the file with one from the beginning. You need to start at the beginning and hash through the end. (That's one of the many problems with MD5 for forensics, BTW.) So I believe that the only approach is sorting the files by the sector number of the first run, and just leaving it at that. >> >> I saw speedup with both HDs and SSDs, strangely enough, but not as much with SSDs. There may be a prefetch thing going on here. >> >> I think that the Autopsy framework should hash this way, but currently it doesn't. On the other hand, it may be more useful to hash based on the "importance" of the files. >> >> Simson >> >> > Hi Simson, currently I have just got as far as noting the 'seek distances' between consecutive runs, across ALL files. I have yet to actually read the file content. But I don't think it's that hard. As you point out, md5 summing must be done with the file content in correct order. I see an analogy between the 'runs ordered by block address but not necessarily file offset' and the problem the IP layer has in tcp/ip as it tries to reassemble the fragments of a datagram that may arrive in any order. We may have to have some 'pending data' structure for runs whose content has been read but which cannot yet be offered to the md5 hasher due to an as yet unread run being needed first. > > I'll let you know if/when I nail this. Pehaps Autopsy could benefit? Is fiwalk doing it the 'regular way' too, i.e reading all file content of each file as the walk proceeds? > > Stuart |
From: Simson G. <si...@ac...> - 2014-09-04 23:31:44
|
Hi Stuart. You are correct — I put this in numerous presentations but never published it. The MD5 algorithm won't let you combine a partial hash from the middle of the file with one from the beginning. You need to start at the beginning and hash through the end. (That's one of the many problems with MD5 for forensics, BTW.) So I believe that the only approach is sorting the files by the sector number of the first run, and just leaving it at that. I saw speedup with both HDs and SSDs, strangely enough, but not as much with SSDs. There may be a prefetch thing going on here. I think that the Autopsy framework should hash this way, but currently it doesn't. On the other hand, it may be more useful to hash based on the "importance" of the files. Simson On Sep 4, 2014, at 7:04 PM, Stuart Maclean <st...@ap...> wrote: > I am tracking recent efforts in STIX and Cybox and all things Mitre. > One indicator of compromise is an md5 hash of some file. Presumably you > compare the hash with all files on some file system to see if there is a > match. Obviously this requires a walk of the host fs, using e.g. fls or > fiwalk or the tsk library in general. > > Is this a common activity, the hashing of a complete filesystem that > is? If yes, some experiments I have done with minimising total disk > seek time by ordering Runs, reading content from the ordered Runs and > piecing each file's hash back together would show that this is indeed a > worthy optimization since it can decrease the time spent deriving the > full hash table considerably. > > I did see a slide deck by Simson G where he alluded to a similar win > situation when disk reads are ordered so as to minimise seek time, but > wonder if much has been published on the topic, specifically relating to > the digital forensics arena, i.e. when an entire file system contents is > to be read in a single pass, for the purposes of producing an 'md5 -> > file path' map. > > Opinions and comments welcomed. > > Stuart > > > ------------------------------------------------------------------------------ > Slashdot TV. > Video for Nerds. Stuff that matters. > http://tv.slashdot.org/ > _______________________________________________ > sleuthkit-users mailing list > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users > http://www.sleuthkit.org |
From: Stuart M. <st...@ap...> - 2014-09-04 23:16:17
|
On 09/04/2014 03:46 PM, Simson Garfinkel wrote: > Hi Stuart. > > You are correct — I put this in numerous presentations but never published it. > > The MD5 algorithm won't let you combine a partial hash from the middle of the file with one from the beginning. You need to start at the beginning and hash through the end. (That's one of the many problems with MD5 for forensics, BTW.) So I believe that the only approach is sorting the files by the sector number of the first run, and just leaving it at that. > > I saw speedup with both HDs and SSDs, strangely enough, but not as much with SSDs. There may be a prefetch thing going on here. > > I think that the Autopsy framework should hash this way, but currently it doesn't. On the other hand, it may be more useful to hash based on the "importance" of the files. > > Simson > > Hi Simson, currently I have just got as far as noting the 'seek distances' between consecutive runs, across ALL files. I have yet to actually read the file content. But I don't think it's that hard. As you point out, md5 summing must be done with the file content in correct order. I see an analogy between the 'runs ordered by block address but not necessarily file offset' and the problem the IP layer has in tcp/ip as it tries to reassemble the fragments of a datagram that may arrive in any order. We may have to have some 'pending data' structure for runs whose content has been read but which cannot yet be offered to the md5 hasher due to an as yet unread run being needed first. I'll let you know if/when I nail this. Pehaps Autopsy could benefit? Is fiwalk doing it the 'regular way' too, i.e reading all file content of each file as the walk proceeds? Stuart |
From: Stuart M. <st...@ap...> - 2014-09-04 22:27:27
|
I am tracking recent efforts in STIX and Cybox and all things Mitre. One indicator of compromise is an md5 hash of some file. Presumably you compare the hash with all files on some file system to see if there is a match. Obviously this requires a walk of the host fs, using e.g. fls or fiwalk or the tsk library in general. Is this a common activity, the hashing of a complete filesystem that is? If yes, some experiments I have done with minimising total disk seek time by ordering Runs, reading content from the ordered Runs and piecing each file's hash back together would show that this is indeed a worthy optimization since it can decrease the time spent deriving the full hash table considerably. I did see a slide deck by Simson G where he alluded to a similar win situation when disk reads are ordered so as to minimise seek time, but wonder if much has been published on the topic, specifically relating to the digital forensics arena, i.e. when an entire file system contents is to be read in a single pass, for the purposes of producing an 'md5 -> file path' map. Opinions and comments welcomed. Stuart |
From: Brian S. <bhs...@ya...> - 2014-09-03 19:04:58
|
Using hfind, I created two hash db indexes (1 custom, 1 based on NSRL) and added them to Autopsy 3.1.0. Command line searching using hfind works as expected, so I know the indexes are good. When I go to ingest new media, I get the messages listed below. I'd expect the "No known bad..." entry to appear, but not the second. Is this a result of user error or a bug? I've found the wiki documentation on the hash db "import" process to be pretty weak, so it easily could be user error. Thanks for your time. -Brian Module Subject Hash lookup "No known bad hash database set" Hash lookup "No known hash database set" |
From: Stuart M. <st...@ap...> - 2014-09-03 19:02:35
|
To follow up my recent post concerning icat and dd differences, I suspect my file in question has 'missing runs', i.e. the file content fits in a single block, and so the remainder of the advertised file size can be produced by the OS as all zeros. Looking at the docs for tsk_fs_file_read, which I suspect is used by icat, I note "0s are returned for missing runs of files" Does that mean the 'return value', or that '0s are inserted into the user buffer' and "Returns the number of bytes read or -1 on error" Does that mean the actual byte count read off disk, which would be 0 for a 'missing run' or does the return value accommodate a missing run, and return any count of 'produced zeros'? Again, and help appreciated. Stu |
From: Stuart M. <st...@ap...> - 2014-09-03 19:02:25
|
I am using tsk 4.1.3 on Ubuntu, 64-bit machine. /dev/sda1 is a ext4 filessytem. I have an inode for which istat claims allocated inode: 1322012 size: 4296704 direct blocks: 5289177 If I dd the file, I do indeed see 4296704 bytes produced. Somewhat curiously, the first 1876 bytes appear to be 'regular content', in fact utf-16 text (the file itself is some sort of kde cache file), while the remainder of the file, over 4MB, are all zeros. According to dd that is. Now, if I icat this file (icat also from 4.1.3), the icat produces only 4096 bytes of content. I presume this number reflects the fact that istat said there was only a single block, and the fs block size is 4096. The icat output shows the same 1876 leading bytes as dd did, and further has all zeros from there up to its 4096 byte length. I am not quite sure what is going on. I was under the impression that icat and dd would give the same result for this file (and would for all allocated files in general). Any help appreciated. Stuart |
From: Richard C. <rco...@ba...> - 2014-09-02 17:52:39
|
Ken McFadden wrote: > Will the java bindings be necessary for Autopsy? Yes. Ken, are you building the SleuthKit master branch or the develop branch? Looking at bindings/java/ivy.xml in the master branch of SleuthKit, I see that a dependency on sqlite-jdbc-3.8.0-SNAPSHOT is specified. The same file in the release-4.2.0 and develop branch lists a dependency on sqlite-jdbc- 3.7.15-M1. However, the build.xml file for both the master and develop branches of Autopsy Core has a getTSKJars target that copies sqlite-jdbc-3.7.15-M1.jar from {env.TSK_HOME}/bindings/java/lib to ${basedir}/release/modules/ex. There are other things in the current Autopsy master and develop branches that are not compatible with the current SleuthKit master branch. So to build both SleuthKit and Autopsy, please first fetch and build the latest revision of either the SleuthKit release-4.2..0 branch or the develop branch , then build either Autopsy master (i.e., Autopsy 3.1.0) or develop (i.e., what will eventually be Autopsy release 3.1.1). I believe the SleuthKit release-4.2.0 branch will be merged into, or will replace, the current SleuthKit master branch in the near future. |
From: Alessandro F. <at...@gm...> - 2014-09-02 16:14:51
|
Yes. If I select the partition in the partitions tree, nothing is show in the detail window. 2014-08-21 4:09 GMT+02:00 Brian Carrier <ca...@sl...>: > So the image has four partitions, but one of them isn't showing any files? > > > On Aug 14, 2014, at 5:42 AM, Alessandro Farina <at...@gm...> wrote: > > > Hi > > I'm analysing an image (EWF) extracted from an IMAC. > > The disk (image) has 4 partition: 2 HFS+ and 2 NTFS (BOOTCAMP). > > I'm using Autopsy 3.0.10 on Window 7 SP1. > > From the partition browser I can't access to one of the HFS+ partition. > > The image file is ok, infact I can mount and browse all the partition in > > linux (via ewfmount) without any problem. The same happens if I access > > the image via ftk mounter on windows. > > I think there is some sort of problem with Autopsy and I would like to > > help whith analysis and debug. > > I can't send to many info on the contents because is part of an ongoing > > investigation, but I think I can share info on disk and partition > structure. > > Any help will be very appreciated. > > > > Thanks in advance > > Alessandro > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > > sleuthkit-users mailing list > > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users > > http://www.sleuthkit.org > > |
From: anthony s. <ant...@gm...> - 2014-09-02 15:15:15
|
Good morning, Is there any way to increase the number of thumbnails on a page? Or is that perhaps something in the works? Its a bit slow to click through 183 pages of thumbs. Thanks ajs |
From: McFadden K. M. <ken...@mc...> - 2014-08-28 20:35:40
|
Will the java bindings be necessary for Autopsy? Thanks, Ken On 28 Aug 2014, at 15:21, Alex Nelson <ajn...@cs...> wrote: > If you don't need the Java bindings, you can pass '--disable-java' to ./configure. > > --Alex > > > On Thu, Aug 28, 2014 at 4:06 PM, McFadden Kenneth Michael <ken...@mc...> wrote: > When compiling the latest version of Sleuthkit I keep getting an unresolved dependency for "org.xerial#sqlite-jdbc;3.8.0-SNAPSHOT: not found" > > I have tried downloading xerial/sqlite-jdbc (which the latest version is 3.8.2) and putting into a path where it should pick up that jar file, changed the XML code to look for this version vice the previous of which I now receive the same error but now "org.xerial#sqlite-jdbc;3.8.2-SNAPSHOT: not found". Is there something else I need to load before compiling sleuthkit? In googling for an answer I saw something that might indicate that I need a Server version of OS X (of which I find hard to believe). Anyone else who might have seen this issue, I'm curious as to what the fix is. > Thanks, > Ken > ------------------------------------------------------------------------------ > Slashdot TV. > Video for Nerds. Stuff that matters. > http://tv.slashdot.org/ > _______________________________________________ > sleuthkit-users mailing list > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users > http://www.sleuthkit.org > |
From: Alex N. <ajn...@cs...> - 2014-08-28 20:21:53
|
If you don't need the Java bindings, you can pass '--disable-java' to ./configure. --Alex On Thu, Aug 28, 2014 at 4:06 PM, McFadden Kenneth Michael < ken...@mc...> wrote: > When compiling the latest version of Sleuthkit I keep getting an > unresolved dependency for "org.xerial#sqlite-jdbc;3.8.0-SNAPSHOT: not found" > > I have tried downloading xerial/sqlite-jdbc (which the latest version is > 3.8.2) and putting into a path where it should pick up that jar file, > changed the XML code to look for this version vice the previous of which I > now receive the same error but now "org.xerial#sqlite-jdbc;3.8.2-SNAPSHOT: > not found". Is there something else I need to load before compiling > sleuthkit? In googling for an answer I saw something that might indicate > that I need a Server version of OS X (of which I find hard to believe). > Anyone else who might have seen this issue, I'm curious as to what the fix > is. > Thanks, > Ken > > ------------------------------------------------------------------------------ > Slashdot TV. > Video for Nerds. Stuff that matters. > http://tv.slashdot.org/ > _______________________________________________ > sleuthkit-users mailing list > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users > http://www.sleuthkit.org > |
From: McFadden K. M. <ken...@mc...> - 2014-08-28 20:19:19
|
When compiling the latest version of Sleuthkit I keep getting an unresolved dependency for "org.xerial#sqlite-jdbc;3.8.0-SNAPSHOT: not found" I have tried downloading xerial/sqlite-jdbc (which the latest version is 3.8.2) and putting into a path where it should pick up that jar file, changed the XML code to look for this version vice the previous of which I now receive the same error but now "org.xerial#sqlite-jdbc;3.8.2-SNAPSHOT: not found". Is there something else I need to load before compiling sleuthkit? In googling for an answer I saw something that might indicate that I need a Server version of OS X (of which I find hard to believe). Anyone else who might have seen this issue, I'm curious as to what the fix is. Thanks, Ken |