Re: [Blocks-development] comments on the Blocks architecture...

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Hmm...your comment about the number of connections we need makes sense.
Maybe we'll eventually come up with a way to auto-balance the network so we
still have multiple routes to files, but not so many routes that broadcast
traffic is amplified so much.

PRNG == Pseudo Random Number Generator

We're throwing around some interesting ideas that address problems similar
to your scenario.  If someone broadcasts a very large number of bogus
files...well, we don't know they're bogus without actually attempting to
download the files.  There's a fine line between violating someone's right
to advertise lots of little files, and preventing spam.

Possible defenses could be:

*  attempt to download blocks from a small random number of the file adverts
that come across...and drop the originating connection if too many of them
are bogus
*  shape the broadcast bandwidth used by each connection

I really have no idea.  I'd recommend reading about this crazy web-of-trust
idea I've been trying to sell, in the mailing list archives.  (Archive link
is on the top of the page you used to sign up for the mailing list.)  The
Blocks network is very new, but I think we suffer from the same classes of
weaknesses that can kill Gnutella.  Then again, our initial freshmeat.net
announcement was, what, three weeks ago?  :)

WRT that simulator...I have a few ideas for it, but I can't contribute code
yet.  Then again, I've just recently decided to halt wasting any more time
learning C++ for Win32 coding, and my first three Linux programming books
just arrived in the mail.  (Although it seems "C Programming for Linux in 21
Days" was a waste of money -- I'm already quite good with C and C++)

(So basically my role so far has been to suggest elaborate and crazy, yet
very secure, content authentication schemes...but I can't code well enough
to do any of the work.  I need to change that.)

--Michael Spencer
bl...@ms...

----- Original Message -----
From: "Ben Houston" <be...@ex...>
To: "Blat Froop" <pet...@ho...>;
<blo...@li...>
Sent: Wednesday, August 16, 2000 5:13 PM
Subject: [Blocks-development] comments on the Blocks architecture...

> Hi PG,
>
> > so, servers with < 7 connections always react, while servers
> > with 14 connections have a 20% chance of trying to connect.
> > The new server should accept on a first-come-first-served
> > basis.
>
> Why do you have to many connections?  Did you know that in a truly
randomly
> connected graph you only need only around 2.5 - 3 connections per node to
> guarantee that the network is connected?  You could go for 4 or 5 if you
> really wanted to be secure.
>
> > 'File advertisements' are broadcast through out the
> > network. Your Blocks application needs to be running
> > to see them. When you do a 'search' you are actually
> > searching the local list maintained by your Blocks
> > application, searches are never broadcast.
>
> Okay, so I guess the point of this is that you encourage
> people to remain on the network for longer periods of times
> since it increases the number of files that they can see.
>
> When a new client connects do you pass along the index of files
> of the clients that it connects to?
>
> How do you track which files have dropped off the network because
> those clients left the network?
>
> > Other servers
> > then choose to connect to the new server depending on how many
> > connections they have, and a PRNG.
>
> What's a PRNG?
>
> Also how do you protect against someone broadcastings a bunch of lies
> instead of a real file index?
>
> > Perhaps Michael & Erik's simulator will clarify this.
>
> Neat, I'm writing a simulator too.  To start with I've created
> a few classes for creating random graphs and measuring
> the connectivity. ...it should be really easy to figure out:
>   http://www.exocortex.org/~ben/graphtools_v01.zip
>
> I'm planning on making a multithreaded simulator once I get some time that
> will simulate the behavior of clients rather than just a weighted graph.
I
> hope to make a visualizer for it a la NetViz:
>   http://www.exocortex.org/netviz
>
> Later,
> -ben houston
> www.exocortex.org/~ben
>
>
> > -----Original Message-----
> > From: Blat Froop [mailto:pet...@ho...]
> > Sent: August 15, 2000 9:45 PM
> > To: be...@ex...; blo...@li...
> > Subject: Re: [Blocks-development] the GRID proposal
> >
> >
> > >From: "Ben Houston" <be...@ex...>
> > >To: <blo...@li...>
> > >Subject: [Blocks-development] the GRID proposal
> > >Date: Tue, 15 Aug 2000 20:35:52 -0400
> > >
> > >Hi,
> > >
> > >A few people (Jonathan Byron and Michael Spencer) have mentioned in the
> > >last
> > >days the BLOCKS project.  I thought that I would drop by and
> > tell you about
> > >my self organizing protocol that I am developing:
> > >
> > >Check it out at: http://www.exocortex.org/grid.html
> > >
> > >And yesterday I developed the start of a growing preference algorithm
in
> > >order to grow it in a manner that will maintain coherence:
> > >http://www.exocortex.org/growing_v01.ppt
> > >
> > >Although I always talk in two dimensions (planar) all the algorithms
are
> > >scalable to any dimension greater than one.
> > >
> > >I'm working full time right now so I do not have time to polish these
> > >ideas.
> > >Also we are in a very fast moving field (distributed file sharing,
> > >distributed applications) so I want to get them out as quickly
> > as possible.
> >
> > Thanks for the input. I dont know much about GNUtella other than
> > that it broadcasts searches. I would have thought that it does
> > a similar sort of routing as Blocks does with file advertisements
> > (searches are not transmitted in Blocknets).
> >
> > Each Blocks server has upto 16 connections to other servers, and
> > when a new file is uploaded, it is given a unique ID (which is
> > placed in a blocklist) an advertisement is send to each
> > connection. If the advert ID is not blacklisted it is forwarded
> > to all connections (apart from the one it came from), and so
> > on for upto 6 hops (its a bit like the old flood fill algorithm).
> >
> > The interconnection logic in blocks is currently quite trivial,
> > when a new server connects to the Blocknet, the server it connects
> > to advertises the new server to all connections. Other servers
> > then choose to connect to the new server depending on how many
> > connections they have, and a PRNG.
> >
> > The formulae works a bit like this...
> >
> > connect to new server if (16-N)*10 > rand()%100
> > where N is the number of current connections.
> >
> > so, servers with < 7 connections always react, while servers
> > with 14 connections have a 20% chance of trying to connect.
> > The new server should accept on a first-come-first-served
> > basis.
> >
> > This hasnt been tuned but seems to work well enough for
> > small Blocknets. Its unclear what will happen with massive
> > Blocknets. Perhaps Michael & Erik's simulator will clarify this.
> >
> > Blocks also relies on each 'connection' to be indistinguishable
> > from all the other ones so that statistical analysis of the
> > Blocknet cannot predict the location in the Blocknet of the
> > originator of published files. This makes designing Blocknet
> > topologies particularly difficult since servers are restricted
> > in the amount of information they are allowed to have about
> > the topology.
> >
> > All comments welcome :-)
> >
> > ttfn
> >
> > PG.
> >
> >
> >
> > ________________________________________________________________________
> > Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com
> >
>
>
> _______________________________________________
> Blocks-development mailing list
> Blo...@li...
> http://lists.sourceforge.net/mailman/listinfo/blocks-development