Re: [Seeks-users] DHT code evolution (repost)

[Loic D. <lo...@da...>]

>   
>> * reducing to a single thread instead of two (with libevent)
>>     
>
> You mean, instead of one for the RPC server + one for the stabilization
> calls in client mode ?
>
>   
Yes.
>> * use UDT instead of UDP
>>     
>
> What would be the rational behind this change given that the protocol buffer
> messages are very small at this time, between 54 and 128 bytes.
>
>   
I'm not entirely sure. I researched it today and found features
important to the DHT
1) multiplexing communication channels allowing for an asynchronous version
    of do_rpc_call (nice to have)
2) keeps information about the connection health which is essential to
figuring
    out if a node is dead or alive (required)
3) the application can use the UDT file descriptor to estabish a
communication
    channel using the same port without disrupting the DHT messages
(nice to have)
4) NAT traversal (nice to have)
    
>> * cache DNS resolution
>>     
I think getaddrinfo_a does the trick (
http://redmine.seeks-project.info/issues/288 ). The fallback is the
standard getaddrinfo which only has the disadvantage of blocking.  This
system call  is apparently available on every GNU/Linux distributions
which should be good for most users willing to go fast with seeks.

>> * use a pool of remote nodes modeling their accessibity accross virtual
>> nodes
>>     
>
> This would be shared board of information about the quality and freshness
> of known nodes and data transfers to them, right ?
>
>   
Yes. And it turns out UDT already collects the necessary information.
>
>> * replace the protobuf wrappers with a hierarchy of derived classes
>>     
>
> Can you please elaborate about what you mean with this modification ?
>
>   
When the server receives a message, it needs to unserialize it to
extract the DHTKey and route the message to the appropriate virtual
node. It should be done by registering a factory to the Transport
object. The factory would be called with the incoming buffer and return
a Message based object. The searchgroup factory woud be derived from the
l1 factory and able to interpret searchgroup specific messages.

It is necessary to avoid the double unserialization that is currently done.
> I agree. Everything that is about to be released should be as clean as
> possible. I will only try to avoid postponing the release of p2p for
> reasons such as building up more features than needed for a safe basic
> start.
>  
>   
I agree and I think we will have a hard time handling security and
keeping it simple.
> The restructured code makes the DHT much better than what it was before
> you put your magic spell on it :) The unit tests are a blessing. 
>
>   
:-)
>
> Exactly. This doesn't mean we have to come up with a framework as advanced
> as maidsafe before the first p2p release, but that the ground should be
> clean and prepare for gradual improvements, theoretical and empirical.
>   
I think we can do simpler.
> The latter is another reason to release a p2p-enabled version in a not
> too distant future.
>
>   
Yes, this is my main concern. Trying to find the right balance.
> Again, thanks for the good work, and all the time you've put on this
> difficult part of Seeks' pool of code.
>   
It is exciting to explore all aspects of this delicate and yet essential
piece of the puzzle ;-)

Cheers