Donate Share

md5deep

Tracker: Bugs

5 Ending offset in piecewise mode wrong if using stdin - ID: 2855502
Last Update: Settings changed ( jessekornblum )

The CVS build gives the following output on a 128-byte file named tst:

$ md5deep -p64 <tst
3a1f1394df3d94828c9e20de752ad889 stdin offset 0-0
8ebc94056b9bf03c3f55464964d14c2a stdin offset 64-0
d41d8cd98f00b204e9800998ecf8427e stdin offset 128-0

After applying the attached patch, the output is as follows:
$ md5deep/md5deep -p64 <tst
3a1f1394df3d94828c9e20de752ad889 stdin offset 0-64
8ebc94056b9bf03c3f55464964d14c2a stdin offset 64-128
d41d8cd98f00b204e9800998ecf8427e stdin offset 128-128

This patch stores the bytes read before computing the hash, and delays
naming the file until after computed. That allows the true bytes read to
be used, permitting correct display when stdin is used.


Nobody/Anonymous ( nobody ) - 2009-09-09 20:59

5

Closed

Fixed

Jesse Kornblum

cosmetic

None

Public


Comments ( 11 )

Date: 2009-09-29 12:55
Sender: jessekornblumProject AdminAccepting Donations

The fix checked into SVN has passed testing and is accepted.


Date: 2009-09-25 16:06
Sender: petiepooo

It works under my test environment. And the last block is omitted as you
intended.

I've been holding off on the progressive patch until you checked this into
SVN. I had to rework it a bit, but will submit shortly.

Thanks.


Date: 2009-09-24 10:38
Sender: jessekornblumProject AdminAccepting Donations

petiepooo, can you please test out the 'fixed' version now in SVN? (I put
fixed in quotes as it's certainly possible that the current version has
bugs..) [grin] thanks!


Date: 2009-09-24 10:36
Sender: jessekornblumProject AdminAccepting Donations

After thinking more about the issue, I've added a 'last block' check that
avoids printing out a hash if no data was read in the block (e.g. bytes
512-511 in your example). Whether the C style offset or Python style offset
is more intuitive is a toss-up. Regardless, users must learn *some*
programming language's style. As such I'm sticking with the one they can
use in a hex editor.


Date: 2009-09-18 13:50
Sender: petiepooo

Oh, and the bug title should say piecemeal instead of progressive. I had
progressive on the brain when I wrote it... :-)


Date: 2009-09-18 13:49
Sender: petiepooo

A correction: in my progressive example below, I meant to use -g512 instead
of -p512. -p512 would give ranges of 0-512, 512-1024, ...
A comment: the -m and -a options work out of the box with the progressive
patch I'm preparing. A sign of a good coding framework!
A question: have you considered my comment about the range displays? I've
been watching for some indication, but haven't seen anything.
Thanks!


Date: 2009-09-10 16:11
Sender: petiepooo

After reviewing your revised patch, I'd recommend not subtracting one from
the ending offset for two reasons:
1) If the size of a file is a multiple of the block size (eg. bs=128,
fs=256), you'll get ranges of 0-127, 128-255, 256-255. The last one is
nonsensical, as it will be the hash of an empty string, not a string of -1
length. It could be omitted, I suppose, but that would require more flow
logic specific to piecemeal mode.
2) Ranges would correspond to (the arguably more intuitive) python-style
slice notation. Eg, a range of [0:0] is the empty set, whereas [0:1]
contains the first value only, and [0:128] contains the first 128 values.
See http://sp2hari.blogspot.com/2007/09/slice.html for more.

Yeah, I know an index of 0 in C is the first array element, but users
shouldn't have to be coders to understand the program output.

My 2 cents...


Date: 2009-09-10 15:49
Sender: petiepooo

You're welcome. I realize now I hadn't logged in before submitting the
bug/patch, so it came from nobody...
To help focus your debug efforts, the patch was tested only on Ubuntu
8.10, not Windows. Also, only the md5deep binary was tested before
submitting.
BTW, this issue came out while developing a new progressive hash feature,
which I will submit for you once I test it a bit more here. I used the -g
option since -p was already used. Its similar to piecemeal, except the
starting offset is always zero ( -p512 gives ranges from 0-512, 0-1024,
0-1536, 0-2048, ...), and the output looks very much like it. Basically, I
save the hash state before finalizing it, and restore the state instead of
initializing it. I needed it to recover the original size of a hashed
forensic image when the number of sectors wasn't specified.


Date: 2009-09-09 23:35
Sender: jessekornblumProject AdminAccepting Donations

Also, an important note, the project is using Subversion (SVN) now instead
of CVS!


Date: 2009-09-09 21:47
Sender: jessekornblumProject AdminAccepting Donations

I have edited the patch to simplify the overall logic, but this appears to
work now. Thank you! I will do some further testing before checking in the
code.


Date: 2009-09-09 21:28
Sender: jessekornblumProject AdminAccepting Donations

I have verified the issue and am testing the patch.


Attached Files ( 2 )

Filename Description Download
revised-patch.txt Revised patch for common/hash.c Download
hash-stdin.patch Patch for md5deep/hash.c Download

Changes ( 8 )

Field Old Value Date By
close_date - 2009-09-29 12:55 jessekornblum
allow_comments 1 2009-09-29 12:55 jessekornblum
status_id Open 2009-09-29 12:55 jessekornblum
summary Ending offset in progressive mode wrong if using stdin 2009-09-24 10:38 jessekornblum
resolution_id None 2009-09-24 10:37 jessekornblum
File Added 342383: revised-patch.txt 2009-09-09 21:46 jessekornblum
assigned_to nobody 2009-09-09 21:28 jessekornblum
File Added 342379: hash-stdin.patch 2009-09-09 20:59 nobody