Re: [sleuthkit-users] Hash MD5 and HMAC
Brought to you by:
carrier
Menu
▾
▴
Re: [sleuthkit-users] Hash MD5 and HMAC
|
From: maría e. d. <dar...@gm...> - 2014-11-12 19:28:24
|
I believe that is really important to include in the new version of Autopsy (3.11) a keyed hash or hmac, it is so relevant in the legal context, to certificate a digital file behind the vulnerabilities that a criptographic hash could have, because it is possible to offer the judge a reliable digital evidence. I was reading some documentation from Encase but I did not found anything about hmac. There is a sourceforge project that generated the tool fehashmac ( http://sourceforge.net/projects/fehashmac/files/), I tried it and works fine. But, it will be really important that this type of keyed hash could be added to sleuthkit and then in Autopsy, because of the legal aspect. (when a file is keyed hash, the key is a way to mantain reliable the evidence and it is transfer in a close envelope with the form of chain of custody). I think that working with open source code is the best way to demostrate that any tool does what is expected (as Brian Carrier wrote in his article) and therefore is reliable and as we say here: transparent. Best regards. María Elena 2014-11-12 14:56 GMT-03:00 <sle...@li...>: > Send sleuthkit-users mailing list submissions to > sle...@li... > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users > or, via email, send a message with subject or body 'help' to > sle...@li... > > You can reach the person managing the list at > sle...@li... > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of sleuthkit-users digest..." > > > Today's Topics: > > 1. Re: fiwalk (Jason Wright) > 2. Re: fiwalk (RB) > 3. Re: fiwalk (Simson Garfinkel) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 12 Nov 2014 12:19:22 -0500 > From: Jason Wright <jwr...@gm...> > Subject: Re: [sleuthkit-users] fiwalk > To: Simson Garfinkel <si...@ac...> > Cc: "sle...@li... users" > <sle...@li...> > Message-ID: > <CAOm9=EmaxQ0cg2v8dJoVkHghV5EYhWUD67= > 7Fz...@ma...> > Content-Type: text/plain; charset="utf-8" > > Separately from this, but somewhat related, given the chatter regarding MD5 > of late, e.g. > > http://arstechnica.com/security/2014/11/crypto-attack-that-hijacked-windows-update-goes-mainstream-in-amazon-cloud/ > , > is there any concern for relying on MD5 in digital forensics? > > I ask because it seems that tsk_loaddb only calculates MD5s for an > image.Typically, we use fiwalk to gather MD5, SHA1, and SHA256 from files > in a drive image. We use those for whitelisting/blacklisting, but do turn > immediately to MD5 for filtering knowns (good and bad) from unknowns. SHA1, > and then of course SHA256, are secondary to that process. Outside of this > article, has there been any other MD5 collisions encountered? > > > On Wed, Nov 12, 2014 at 11:59 AM, Simson Garfinkel <si...@ac...> > wrote: > > > Hi, Jason. > > > > Sadly, tsk_loaddb does not grab file signatures. It can hash, but I don't > > know if it just does MD5 or if it does SHA1. It does not do SHA256. The > > byte runs are available. > > > > I gather you are running fiwalk on Linux? > > > > > > > > On Nov 12, 2014, at 11:23 AM, Jason Wright <jwr...@gm...> > wrote: > > > > Simson et al, > > > > In my organization, we still use fiwalk too. We use it to gather as much > > file metadata as possible, to include file signatures, MD5, SHA1, and > > SHA256, and byte runs for file allocation along with the regular > complement > > from the MFT indices. We have not used tsk_loaddb and if need be we can > > switch to that, provided we still get all of that info. Does it provide > > that breadth of detail? > > > > V/R, > > > > Jason > > > > On Mon, Nov 10, 2014 at 3:43 PM, Alex Nelson <ajn...@cs...> > wrote: > > > >> Hi Simson, all, > >> > >> I'm making extensive use of Fiwalk. I believe the BitCurator folks are > >> as well. > >> > >> --Alex > >> > >> > >> > >> On Mon, Nov 10, 2014 at 3:18 PM, Simson Garfinkel <si...@ac...> > >> wrote: > >> > >>> Derrick, > >>> > >>> My recommendation is that you transition away from fiwalk. > >>> > >>> My one concern with tsk_loaddb is that the filenames it produces are > not > >>> necessarily UTF-8. It seems to be putting in the database whatever is > on > >>> the disk, which can cause problems in post-analysis. I'm not sure how > >>> others are dealing with this. My problem is that on Windows, I'm > reading > >>> these values with Python and I'm getting exceptions when I attempt to > write > >>> them to a file. > >>> > >>> > >>> > >>> > On Nov 10, 2014, at 1:17 PM, Derrick Karpo <dk...@gm...> wrote: > >>> > > >>> > I am still using fiwalk but have been transitioning to tsk_loaddb. > >>> > > >>> > My primary reason is that my forensic indexer (Xapian) automatically > >>> > indexes fiwalk text output and I haven't configured it to include > >>> > sqlite files yet. It's a simple config change for me to fully > >>> > transition over. > >>> > > >>> > Derrick > >>> > > >>> > > >>> > On Mon, Nov 10, 2014 at 10:43 AM, Simson Garfinkel <si...@ac...> > >>> wrote: > >>> >> I see. > >>> >> > >>> >> The other approach would be to abandon fiwalk and move the things > >>> that use it over to using the database produced by tsk_loaddb. > >>> >> > >>> >> Is anyone other than me using fiwalk at this point? > >>> >> > >>> >> Simson > >>> >> > >>> >> > >>> >>> On Nov 10, 2014, at 12:40 PM, Brian Carrier <ca...@sl... > > > >>> wrote: > >>> >>> > >>> >>> We don't use mingw for the TSK packaging. Just Visual Studio. So, > >>> it would be much easier to include if there were a visual studio > project > >>> for it. > >>> >>> > >>> >>> > >>> >>> > >>> >>> > >>> >>> > >>> >>> On Nov 10, 2014, at 12:16 PM, Simson Garfinkel <si...@ac...> > >>> wrote: > >>> >>> > >>> >>>> Hi. I see that fiwalk.exe is not being compiled as part of the > >>> pre-compiled SleuthKit download. > >>> >>>> > >>> >>>> Is there some reason why fiwalk is not included, and is there any > >>> packaging change that I could make to make it more likely to include > the > >>> executable in the future? > >>> >>>> > >>> >>>> Simson > >>> >>>> > >>> >>>> > >>> >>>> > >>> > ------------------------------------------------------------------------------ > >>> >>>> _______________________________________________ > >>> >>>> sleuthkit-users mailing list > >>> >>>> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users > >>> >>>> http://www.sleuthkit.org > >>> >>> > >>> >> > >>> >> > >>> >> > >>> > ------------------------------------------------------------------------------ > >>> >> _______________________________________________ > >>> >> sleuthkit-users mailing list > >>> >> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users > >>> >> http://www.sleuthkit.org > >>> > >>> > >>> > >>> > ------------------------------------------------------------------------------ > >>> Comprehensive Server Monitoring with Site24x7. > >>> Monitor 10 servers for $9/Month. > >>> Get alerted through email, SMS, voice calls or mobile push > notifications. > >>> Take corrective actions from your mobile device. > >>> > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk > >>> _______________________________________________ > >>> sleuthkit-users mailing list > >>> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users > >>> http://www.sleuthkit.org > >>> > >> > >> > >> > >> > ------------------------------------------------------------------------------ > >> Comprehensive Server Monitoring with Site24x7. > >> Monitor 10 servers for $9/Month. > >> Get alerted through email, SMS, voice calls or mobile push > notifications. > >> Take corrective actions from your mobile device. > >> > >> > http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk > >> _______________________________________________ > >> sleuthkit-users mailing list > >> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users > >> http://www.sleuthkit.org > >> > >> > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > > ------------------------------ > > Message: 2 > Date: Wed, 12 Nov 2014 10:55:57 -0700 > From: RB <ao...@gm...> > Subject: Re: [sleuthkit-users] fiwalk > To: Jason Wright <jwr...@gm...> > Cc: "sle...@li... users" > <sle...@li...> > Message-ID: > <CADkMHCmgby8h8YRoFhvtyypDJd= > 8ae...@ma...> > Content-Type: text/plain; charset=UTF-8 > > On Wed, Nov 12, 2014 at 10:19 AM, Jason Wright <jwr...@gm...> > wrote: > > Separately from this, but somewhat related, given the chatter regarding > MD5 > > of late, e.g. > > > http://arstechnica.com/security/2014/11/crypto-attack-that-hijacked-windows-update-goes-mainstream-in-amazon-cloud/ > , > > is there any concern for relying on MD5 in digital forensics? > > I'd love to jump right in there. MD5 is passing its use-by date > should generally not be used for new projects, but it's not quite the > "abandon ship" picture the original author and everyone else seems to > want to paint. Hooray for headline-chasing rather than realism. > Let's inject a little of the latter. > > The linked attack takes two dissimilar images of the same size and > requires their differences be "of a particular form." Someone closer > to cryptography can explain the "particular form" better than I, but > what it boils down to is that it is NOT a generalized attack for any > file type. The attack then modifies _both_ files progressively, > adding semi-random data until their MD5 matches, something of a > "birthday attack meets in the middle" approach. > > This is bad, but any analyst with their head on their shoulders will > recognize that: > a) to be exploited it would require access to modify the "original" as > well as the "new" file and > b) the data added is non-meaningful > > Full stop. Nobody has figured out a way to change "mary had a little > lamb" to "drink your ovaltine", this is random data appended to an > already lossy format in a highly detectable manner. Those files you > have with only md5 checksums from 10 years ago are still mostly safe > so long as your attacker doesn't have access to modify them as well. > > Should you use more than just MD5 to track or identify files? Yes. > Size and multiple checksums are a great way to guard against this kind > of attack. > > Should new applications depend solely on MD5? Probably not, there are > better alternatives whose additional cost (both programming and > computational) are effectively zero. > > Someone wake me up when (or at least stop the sensationalism until) an > attacker can generate an arbitrary, meaningful file with minimal size > difference from and the same MD5 as the original without altering the > original. Then MD5 is completely dead. > > > > ------------------------------ > > Message: 3 > Date: Wed, 12 Nov 2014 12:56:41 -0500 > From: Simson Garfinkel <si...@ac...> > Subject: Re: [sleuthkit-users] fiwalk > To: Jason Wright <jwr...@gm...> > Cc: "sle...@li... users" > <sle...@li...> > Message-ID: <A23...@ac...> > Content-Type: text/plain; charset="us-ascii" > > My take: > > - MD5 collisions are generally not an issue in digital forensics if you > are looking for known content. If you have the MD5 of a piece of stolen IP, > it is unlikely that an adversary will attempt to craft multiple files to > have the same MD5 as that stolen IP. > > - MD5 collisions may be an issue if you are using MD5s as an "ignore" list > rather than as an "alert" list. That is, if you have a list of MD5s that > you routinely ignore (e.g. executables), then there is a chance that an > adversary may modify the MD5 of one of their files to match one of the > files that you ignore. However it's hard. Currently it's possible to make > two files have the same MD5, but I'm not aware that it is possible to tune > a second file to match the MD5 of an arbitrary first file. That is, MD5 no > longer has collision resistance, but I believe it still has preimage > resistance. > > In your case below, using MD5 for filtering known bads from unknowns is > fine. However, using them for known goods is potentially problematic if you > think that the adversary can get their known good into your known goods > list. > > Simson > > > > > On Nov 12, 2014, at 12:19 PM, Jason Wright <jwr...@gm...> > wrote: > > > > Separately from this, but somewhat related, given the chatter regarding > MD5 of late, e.g. > http://arstechnica.com/security/2014/11/crypto-attack-that-hijacked-windows-update-goes-mainstream-in-amazon-cloud/ > < > http://arstechnica.com/security/2014/11/crypto-attack-that-hijacked-windows-update-goes-mainstream-in-amazon-cloud/>, > is there any concern for relying on MD5 in digital forensics? > > > > I ask because it seems that tsk_loaddb only calculates MD5s for an > image.Typically, we use fiwalk to gather MD5, SHA1, and SHA256 from files > in a drive image. We use those for whitelisting/blacklisting, but do turn > immediately to MD5 for filtering knowns (good and bad) from unknowns. SHA1, > and then of course SHA256, are secondary to that process. Outside of this > article, has there been any other MD5 collisions encountered? > > > > > > On Wed, Nov 12, 2014 at 11:59 AM, Simson Garfinkel <si...@ac... > <mailto:si...@ac...>> wrote: > > Hi, Jason. > > > > Sadly, tsk_loaddb does not grab file signatures. It can hash, but I > don't know if it just does MD5 or if it does SHA1. It does not do SHA256. > The byte runs are available. > > > > I gather you are running fiwalk on Linux? > > > > > > > >> On Nov 12, 2014, at 11:23 AM, Jason Wright <jwr...@gm... > <mailto:jwr...@gm...>> wrote: > >> > >> Simson et al, > >> > >> In my organization, we still use fiwalk too. We use it to gather as > much file metadata as possible, to include file signatures, MD5, SHA1, and > SHA256, and byte runs for file allocation along with the regular complement > from the MFT indices. We have not used tsk_loaddb and if need be we can > switch to that, provided we still get all of that info. Does it provide > that breadth of detail? > >> > >> V/R, > >> > >> Jason > >> > >> On Mon, Nov 10, 2014 at 3:43 PM, Alex Nelson <ajn...@cs... > <mailto:ajn...@cs...>> wrote: > >> Hi Simson, all, > >> > >> I'm making extensive use of Fiwalk. I believe the BitCurator folks are > as well. > >> > >> --Alex > >> > >> > >> > >> On Mon, Nov 10, 2014 at 3:18 PM, Simson Garfinkel <si...@ac... > <mailto:si...@ac...>> wrote: > >> Derrick, > >> > >> My recommendation is that you transition away from fiwalk. > >> > >> My one concern with tsk_loaddb is that the filenames it produces are > not necessarily UTF-8. It seems to be putting in the database whatever is > on the disk, which can cause problems in post-analysis. I'm not sure how > others are dealing with this. My problem is that on Windows, I'm reading > these values with Python and I'm getting exceptions when I attempt to write > them to a file. > >> > >> > >> > >> > On Nov 10, 2014, at 1:17 PM, Derrick Karpo <dk...@gm... <mailto: > dk...@gm...>> wrote: > >> > > >> > I am still using fiwalk but have been transitioning to tsk_loaddb. > >> > > >> > My primary reason is that my forensic indexer (Xapian) automatically > >> > indexes fiwalk text output and I haven't configured it to include > >> > sqlite files yet. It's a simple config change for me to fully > >> > transition over. > >> > > >> > Derrick > >> > > >> > > >> > On Mon, Nov 10, 2014 at 10:43 AM, Simson Garfinkel <si...@ac... > <mailto:si...@ac...>> wrote: > >> >> I see. > >> >> > >> >> The other approach would be to abandon fiwalk and move the things > that use it over to using the database produced by tsk_loaddb. > >> >> > >> >> Is anyone other than me using fiwalk at this point? > >> >> > >> >> Simson > >> >> > >> >> > >> >>> On Nov 10, 2014, at 12:40 PM, Brian Carrier <ca...@sl... > <mailto:ca...@sl...>> wrote: > >> >>> > >> >>> We don't use mingw for the TSK packaging. Just Visual Studio. So, > it would be much easier to include if there were a visual studio project > for it. > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> On Nov 10, 2014, at 12:16 PM, Simson Garfinkel <si...@ac... > <mailto:si...@ac...>> wrote: > >> >>> > >> >>>> Hi. I see that fiwalk.exe is not being compiled as part of the > pre-compiled SleuthKit download. > >> >>>> > >> >>>> Is there some reason why fiwalk is not included, and is there any > packaging change that I could make to make it more likely to include the > executable in the future? > >> >>>> > >> >>>> Simson > >> >>>> > >> >>>> > >> >>>> > ------------------------------------------------------------------------------ > >> >>>> _______________________________________________ > >> >>>> sleuthkit-users mailing list > >> >>>> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users < > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users> > >> >>>> http://www.sleuthkit.org <http://www.sleuthkit.org/> > >> >>> > >> >> > >> >> > >> >> > ------------------------------------------------------------------------------ > >> >> _______________________________________________ > >> >> sleuthkit-users mailing list > >> >> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users < > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users> > >> >> http://www.sleuthkit.org <http://www.sleuthkit.org/> > >> > >> > >> > ------------------------------------------------------------------------------ > >> Comprehensive Server Monitoring with Site24x7. > >> Monitor 10 servers for $9/Month. > >> Get alerted through email, SMS, voice calls or mobile push > notifications. > >> Take corrective actions from your mobile device. > >> > http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk > < > http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk > > > >> _______________________________________________ > >> sleuthkit-users mailing list > >> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users < > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users> > >> http://www.sleuthkit.org <http://www.sleuthkit.org/> > >> > >> > >> > ------------------------------------------------------------------------------ > >> Comprehensive Server Monitoring with Site24x7. > >> Monitor 10 servers for $9/Month. > >> Get alerted through email, SMS, voice calls or mobile push > notifications. > >> Take corrective actions from your mobile device. > >> > http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk > < > http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk > > > >> _______________________________________________ > >> sleuthkit-users mailing list > >> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users < > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users> > >> http://www.sleuthkit.org <http://www.sleuthkit.org/> > >> > >> > > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > > ------------------------------ > > > ------------------------------------------------------------------------------ > Comprehensive Server Monitoring with Site24x7. > Monitor 10 servers for $9/Month. > Get alerted through email, SMS, voice calls or mobile push notifications. > Take corrective actions from your mobile device. > > http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk > > ------------------------------ > > _______________________________________________ > sleuthkit-users mailing list > sle...@li... > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users > > > End of sleuthkit-users Digest, Vol 101, Issue 7 > *********************************************** > -- Prof. Ing. María Elena Darahuge M P Copitec 5100 |
