[Seeks-users] DHT: blocking vs non-blocking

[Loic D. <lo...@da...>]

Hi,

After discussing the pros and cons of using blocking I/O versus
non-blocking I/O for the implementation of the DHT, it boils down to the
following:

    blocking => trivial implementation but threads
    non blocking => single thread but deferred style implementation

After reviewing other aspects (performances, scaling, resource usages,
complexity of the dependencies and more) I did not find any with a
significant influence, either on the maintainability or on the usability.

As a friend suggested tonight, when there is no compelling reason to
chose one solution over the other, start a 60 seconds countdown and
decide when it expires. Unless anyone has a particular insight to share
with me, it's what I'm going to do on Monday ;-)

Cheers

* blocking vs non blocking

  The chord algorithms are documented as if I/O were blocking.
  Iteratively finding the successor for a key is a loop
  using nodes retrieved closer and closer to the key.

    node = self.closest_predecessor(key)
    until(key in [node,successor[)
       node, successor = node.closest_predecessor(key)

  With non blocking I/O and deferred
  ( http://en.wikipedia.org/wiki/Workflow_patterns#State-based_patterns )
  it could be:

  function find_successor(searched_key)

    function find_successor(predecessor, successor)
        if(searched_key in [predecessor, successor[)
          return successor    node = self.closest_predecessor(key)

        else
          deferred = predecessor.send(find_closest_predecessor)
          deferred.addCalback(closest_predecessor)
          return deferred

    function closest_predecessor(predecessor)
        deferred = predecessor.send(get_successor)
        deferred.addCalback(lambda successor:
find_successor(predecessor, successor))
        return deferred

    node = self.closest_predecessor(key)
    deferred = node.send(get_successor)
    deferred.addCallback(lambda successor: find_successor(node, successor))
    return deferred

  function my_function(successor)
     ... do my stuff ...

  find_successor(key).addCallback(my_function)

  The downside of the non blocking approach shows : the implementation
is more
  complex and significantly harder to read.

  The upside is that there is no need to isolate every call to
  find_successor in a separate thread. Even in moderately large Chord
  rings, a find_successor involving 5 nodes with a RTT of 100ms on
  average would takes 5 * 100 * 2 = 1 second (the 2 is for successor +
  find_closest_predecessor).