Re: [bailey-developers] lattice master

[Yonik S. <yo...@ap...>]

On Thu, Feb 28, 2008 at 4:39 PM, Doug Cutting <cu...@ap...> wrote:
> Yonik Seeley wrote:
>  > I think one problem we are running into is due to using the same hash
>  > ring for both partitioning the key space and selecting placement of
>  > those keys.  Putting many virtual nodes on the hash ring to make the
>  > hash function fair, in conjunction with using that hash ring to do
>  > replication, causes all the indicies to be mixed together.
>
>  That's a feature, not a bug!  Seriously, I'm not sure what your concern
>  here is.  If documents are indexed in the N nodes clockwise from their
>  point, then we do get documents indexed together, but we don't need to
>  search but every Nth index (in most cases).  So I don't see the problem.
>   What am I missing?

Given that you and Ning don't see it, it's highly likely I'm the one
with the disconnect.

w/o virtual nodes, everything makes sense to me... I can see how we
can query every 3rd host to cover the entire index once.
I'm using "virtual nodes" in the sense used here:
http://weblogs.java.net/blog/tomwhite/archive/2007/11/consistent_hash.html

To be explicit, with no virtual nodes, and N=3, I think this is what
we would end up with if we added a single node per host:
HostA: indexA,  indexC, indexD
HostB: indexB,  indexD, indexA
HostC: indexC,  indexA, indexB
HostD: indexD,  indexB, indexC

So we can query two nodes and get complete coverage (with appropriate
filtering).

Now if we addded 100 nodes (virtual nodes) per host in order to even
key distribution, only a single host will have all of the keys for a
node (defined as all of it's virtual nodes).

HostA: indexA,  big mix, big mix
HostB: indexB,  big mix, big mix
HostC: indexC,  big mix, big mix
HostD: indexD,  big mix, big mix

we could no longer say that nodeC is available from both hostA and
hostB, because there are 100 nodeCs.
So now it seems like we must query all hosts to get complete coverage,
and if HostA goes off-line, we must query all other hosts to get
coverage of indexA.

-Yonik