Re: [bailey-developers] SF.net SVN: bailey: [10] trunk/src

[Ning L. <nin...@gm...>]

On Mon, Mar 17, 2008 at 4:12 PM, Doug Cutting <cu...@ap...> wrote:
> Ning Li wrote:
>  > None of the host related protocols are there yet. We should decide
>  > a few things before we add those?
>  > 1 Should nodes run as threads within a host process or as separate
>  >    processes on a host? Do we still need a host process in the latter case?
>
>  I think we'll at least initially implement these as threads.  I don't
>  see many advantages of making these separate processes, and it will
>  complicate lots of things.

They have their pros and cons. But yes, let's start with nodes as threads.


>  > 2 In the former case, do we have HostToMasterProtocol and
>  >    ClientToHostProtocol, or do we have NodeToMasterProtocol and
>  >    ClientToNodeProtocol, or both sets?
>
>  With threads, we don't need both Host and Node protocols, so, unless we
>  think it will make implementation cleaner, we should skip this.  Will
>  each node have its own thread, or might a single thread host per send
>  hearbeats for all nodes, with another thread fetching and applying
>  updates, etc.  (Since Lucene itself now multithreads updates, we might
>  not want all nodes updating in parallel.)
>
>  Given that, I think I'd opt for just HostToMasterProtocol and
>  ClientToHostProtocol, with nodes in parameters.  Does that sound
>  reasonable to you?

This sounds reasonable. We'd also have a logger thread (or threads)
for all the nodes. I still think we should allow multiple update threads?
I'll change Node protocols to Host protocols.


>  > I changed range to be <long, long> for now. However, a document
>  > id is of type String, so we have to convert from String to long. It is
>  > hard if we want to maintain the order of the ids after we convert
>  > them to longs.
>
>  Right.  We need to decide on the bits per point on the ring.  Is 64
>  enough?  We also need a Document method to get its ring coordinate.
>  This will by default be a hash of the ID string, but we should store it
>  in a separate field, so that folks can someday specify it explicitly.

Why cann't we use the string for a point on the ring? Each document
can have a unique point and all the documents have an order.


Ning