Re: [Seeks-users] DHT: blocking vs non-blocking

[Emmanuel B. <ebe...@se...>]

Hi Loic,

in my opinion, non-blocking I/O allow for better performance, especially
on DHT nodes supporting a few hundreds of virtual nodes.

The reason I see is that in blocking mode the stabilizer iterates among
elements to stabilize (finger table, successors). Thus every blocking
operation delays the whole queue of waiting elements.
Also, from what I understand modern DHT such as 'maidsafe' are event-based.

Now, how do we get there, to the non-blocking I/O ? What I see are many
changes to the DHT. Do we have a working (or almost) working blocking 
design right now ? If yes, I would propose we go with it and review
the operations before moving ahead with non-blocking. Typically there 
are potential bugs that I see in the (old) 'dht' branch that need to be 
discussed.
If no, I'd say we go straight for the non-blocking I/O, not waiting 
'en route'.

Typically, could you give an estimate of the design & development time 
needed in both cases ?

Thanks for the clear exposure of both architectures. Personnally I don't
find the non-blocking I/O code to be much more difficult to read than
the blocking one.

thanks for the good work!

Em.

On Sun, Jan 16, 2011 at 01:03:11AM +0100, Loic Dachary wrote:
> Hi,
> 
> After discussing the pros and cons of using blocking I/O versus
> non-blocking I/O for the implementation of the DHT, it boils down to the
> following:
> 
>     blocking => trivial implementation but threads
>     non blocking => single thread but deferred style implementation
> 
> After reviewing other aspects (performances, scaling, resource usages,
> complexity of the dependencies and more) I did not find any with a
> significant influence, either on the maintainability or on the usability.
> 
> As a friend suggested tonight, when there is no compelling reason to
> chose one solution over the other, start a 60 seconds countdown and
> decide when it expires. Unless anyone has a particular insight to share
> with me, it's what I'm going to do on Monday ;-)
> 
> Cheers
> 
> * blocking vs non blocking
> 
>   The chord algorithms are documented as if I/O were blocking.
>   Iteratively finding the successor for a key is a loop
>   using nodes retrieved closer and closer to the key.
> 
>     node = self.closest_predecessor(key)
>     until(key in [node,successor[)
>        node, successor = node.closest_predecessor(key)
> 
>   With non blocking I/O and deferred
>   ( http://en.wikipedia.org/wiki/Workflow_patterns#State-based_patterns )
>   it could be:
> 
>   function find_successor(searched_key)
> 
>     function find_successor(predecessor, successor)
>         if(searched_key in [predecessor, successor[)
>           return successor    node = self.closest_predecessor(key)
> 
>         else
>           deferred = predecessor.send(find_closest_predecessor)
>           deferred.addCalback(closest_predecessor)
>           return deferred
> 
>     function closest_predecessor(predecessor)
>         deferred = predecessor.send(get_successor)
>         deferred.addCalback(lambda successor:
> find_successor(predecessor, successor))
>         return deferred
> 
>     node = self.closest_predecessor(key)
>     deferred = node.send(get_successor)
>     deferred.addCallback(lambda successor: find_successor(node, successor))
>     return deferred
> 
>   function my_function(successor)
>      ... do my stuff ...
> 
>   find_successor(key).addCallback(my_function)
> 
>   The downside of the non blocking approach shows : the implementation
> is more
>   complex and significantly harder to read.
> 
>   The upside is that there is no need to isolate every call to
>   find_successor in a separate thread. Even in moderately large Chord
>   rings, a find_successor involving 5 nodes with a RTT of 100ms on
>   average would takes 5 * 100 * 2 = 1 second (the 2 is for successor +
>   find_closest_predecessor).
> 

> begin:vcard
> fn:Loic Dachary
> n:Dachary;Loic
> org:pokersource
> adr:;;12 bd Magenta;Paris;;75010;France
> email;internet:lo...@da...
> title:Senior Developer
> tel;work:+33 4 84 25 08 05
> tel;home:+33 9 51 18 43 38
> tel;cell:+33 6 64 03 29 07
> note:Born 131414404 before EPOCH.
> x-mozilla-html:FALSE
> url:http://dachary.org/
> version:2.1
> end:vcard
> 




> ------------------------------------------------------------------------------
> Protect Your Site and Customers from Malware Attacks
> Learn about various malware tactics and how to avoid them. Understand 
> malware threats, the impact they can have on your business, and how you 
> can protect your company and customers by using code signing.
> http://p.sf.net/sfu/oracle-sfdevnl

> _______________________________________________
> Seeks-users mailing list
> See...@li...
> https://lists.sourceforge.net/lists/listinfo/seeks-users