[Blocks-development] Secure hash discussion...hopefully easy-to-read this time! ^_^

[Spencer Jr., M. <sp...@ac...>]

I'm remaking an argument I promised I would leave until later...being
grateful enough for just the content hash...but we can look at it now in the
discussion of securely resuming files.

SECURE HASH  --  WHY USE IT?

The only problem with resuming files by hash is if you have a malicious
client.  The malicious client could advertise a file with a certain
hash...but the hash doesn't really match the content.  With the current
hashing algorithm, you can't possibly know that the hash doesn't match the
content until you have the whole file -- and then it's too late.

This could be even more evil with automatic resumes:  if I'm a malicious
client, I just listen for file advertisements, and readvertise a file of my
own that doesn't really exist, but has the same content hash as the original
file.  People would then try to resume from me, but I'm going to send them
bogus data instead of the file they think they're requesting.

As I understand the transfer of blocks and the file header, when the clients
retrieve the file header, they have a list of block ID's to request.  So is
it possible for a malicous client to construct a block that the client will
download and accept, and not realize it's a bogus block until they
reconstruct the whole file at the end?  If that is possible...then this
could open us up to a wider-scope file sharing attack.

SECURE HASH  --  OK, WHAT IS IT THEN?

In my opinion, a 'secure hash' is a hash that you can verify WHILE STILL
DOWNLOADING the file, and an 'insecure hash' is a hash that you cannot
verify until you have the whole file.

From here, you guys are free to make up your own secure hash.

My idea for a secure hash:  for each block of the file (not the encrypted
file, the unencrypted original) create a hash (RIPEMD-128 perhaps) and store
it in a block-of-hashes.  Once you have that big ball of hashes, all put
into a file end-to-end, hash that file.

UPLOADING:  When you upload, you create your normal file-header block (just
like we're doing now) and also create a block-of-hashes.  Create another
hash of the block-of-hashes, and put THAT hash in the file advert you send
out.

DOWNLOADING:  When you download, you obtain the normal file-header block
(just like we do now) and also obtain the block-of-hashes.  Compare the
block-of-hashes against the hash that was advertised in the file advert (as
if the block-of-hashes is a file).  If the block-of-hashes doesn't match the
file advert's hash, one or the other is corrupt (deliberately or
accidentally) -- HALT THE DOWNLOAD.  Otherwise, continue.  Request the first
block of data...and compare the block of data against the first block in the
block-of-hashes.  If that hash doesn't match the downloaded block, again one
or the other is corrupt -- HALT THE DOWNLOAD.  Otherwise, repeat until the
file's complete.

WHY IT WORKS

That malicious client advertising bogus files can be detected and defeated
(people will still waste those two blocks worth of transfer, but their
download will eventually continue, and it won't get corrupted.  This is
because you can tell you're downloading bogus data once you have a whole
block, not the whole file.  (And if you wait until you have the whole file
to decide your data is bogus, you've pissed off a user.)


NECESSARY PLUG FOR THE AUTHOR

This is a cool feature to implement, and even without the web-of-trust, this
would help make resumes safer against malicious users or spammers.

But this is freakin 0.14!  :)  I don't know about the rest of us...but I'm
very impressed with the extreme amount of work our coder is pumping out, and
I'll be the last person in the world to rush him.  Maybe around 0.25 or
something, we can start to implement this.  But maybe we can discuss it now.
:)


--Michael Spencer
bl...@ms...


-----Original Message-----
From: Blat Froop [mailto:pet...@ho...]
Sent: Thursday, August 03, 2000 10:02 AM
To: blo...@li...
Subject: [Blocks-development] V0.15 progress update


so far Ive done the following...

1) Fix timeouts and problems when uploading &
   downloading at the same time.

2) All whole now multiple uploads capability for
   bulk uploading.

3) Add content hash to adverts and header blocks,
   and display this in the serach window.

4) Increase the smartness of the 'bad route' logic
   so that search results are more accurate.

5) Change readvertisement logic so that it now
   readvertises most recent 1024 non-local ads.

6) Increase netcode efficiency (might reduce CPU,
   and increase throughput on unlimited connections).

The one outstanding thing to look at was the automatic
switching of routes on failed downloads. Im not sure
about this, on one hand it seems like a nice feature,
but on the other, it would be nice to investigate
downloading errors during the testing phase, so I may
leave this out for now.

Now, we need a plan on an upgrade strategy... I think
we're probably still small enough to simply switch
everything from V0.14 to V0.15 simulateneously, but do
we want to keep a V0.14 public server running until
we can verify that V0.15 is reasonably stable?

Any comments?

ttfn

PG.

________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com


_______________________________________________
Blocks-development mailing list
Blo...@li...
http://lists.sourceforge.net/mailman/listinfo/blocks-development