Re: [Seeks-users] DHT code evolution (repost)

[Emmanuel B. <ebe...@se...>]

On Wed, Dec 22, 2010 at 10:05:17AM +0100, Loic Dachary wrote:
> Hi,
> 
> I completed a rewrite of the stabilization of the successor list of the
> seeks DHT today, with the associated tests. It is available at
> http://seeks.dachary.org/seeks.git

Thanks this is great work.
 
> In my opinion the changes that shoud be done to estabish a sound base
> for the DHT are the following:
> 
> * replacing the stabilization loop with libevent

OK. This is ticket #39:
http://redmine.seeks-project.info/issues/39

> * reducing to a single thread instead of two (with libevent)

You mean, instead of one for the RPC server + one for the stabilization
calls in client mode ?

> * use UDT instead of UDP

What would be the rational behind this change given that the protocol buffer
messages are very small at this time, between 54 and 128 bytes.

> * cache DNS resolution

OK.

> * use a pool of remote nodes modeling their accessibity accross virtual
> nodes

This would be shared board of information about the quality and freshness
of known nodes and data transfers to them, right ?

> * use Location instead of DHTKey + NetAddress

OK.

> * replace the protobuf wrappers with a hierarchy of derived classes

Can you please elaborate about what you mean with this modification ?

> Each of them taken individually are relatively easy to agree on. All
> together they look like more work than necessary to get started. It
> would certainly be possible to have a working DHT without doing this
> ground work. But I believe it would seriously jeopardize our ability to
> build on safe grounds.

I agree. Everything that is about to be released should be as clean as
possible. I will only try to avoid postponing the release of p2p for
reasons such as building up more features than needed for a safe basic
start.
 
> In the past weeks I wrote tests and extensively explored the existing
> code. I don't think I missed an area that would contradict the
> diagnostic above. I adapted and refactored locally to the best of my
> abilities. But I believe it is time, using our combined experiences, to
> restructure the code to match all of the above constraints at once.

The restructured code makes the DHT much better than what it was before
you put your magic spell on it :) The unit tests are a blessing. 

I can help with the refactoring at all levels. My attention is more to
soundness of the theoretical foundations of the DHT. All of this can be
discussed here.
 
> One could argue that it would be a waste of time to carefull restructure
> and that it is more important to get something working quickly. But if
> that was the case, chosing a ready made solution such as maidsafe would
> be a lot quicker. The main reason to implement a DHT is to make it an
> integral part of seeks instead of a tool box imported from a third party.

Exactly. This doesn't mean we have to come up with a framework as advanced
as maidsafe before the first p2p release, but that the ground should be
clean and prepare for gradual improvements, theoretical and empirical.
The latter is another reason to release a p2p-enabled version in a not
too distant future.

Again, thanks for the good work, and all the time you've put on this
difficult part of Seeks' pool of code.

Em.