Re: [bailey-developers] lattice master

[Ning L. <nin...@gm...>]

On Wed, Feb 27, 2008 at 6:04 PM, Yonik Seeley <yo...@ap...> wrote:
> On Wed, Feb 27, 2008 at 3:58 PM, Ning Li <nin...@gm...> wrote:
>
> >  At the same time, should we also discuss what the update
>  >  model should be:
>  >  1 One updatable replica vs. all updatable replicas. The former
>  >    is simple. The latter is powerful. Is there sufficient need for
>  >    the latter to justify its complexity?
>
>  We should always be able to update (so if the "updateable" replica is
>  down, we need to be able to update another replica).  Given that, what
>  is the extra complexity of those two choices?

In the case of one updateable replica, if the updateable replica is down,
we need to make another replica updateable. The down side is that
the shard is not updateable during the switch. But it should be short
and thus be fine for some applications. The complexity involved with
all updateable replicas is conflict resolution...

>  >  2 The atomicity of an insert/delete/update operation. When
>  >    an insert/delete/update operation is done, does it mean:
>  >    1) the new doc is indexed in the memory of the node
>  >    2) the new doc is indexed on the local disk of the node
>  >    3) the new doc is logged on the local disk of the node
>  >    4) the new doc is logged in some fault-tolerant shared FS
>  >       (e.g. HDFS)
>  >    5) the new doc is indexed in the memory of at least X
>  >       nodes
>  >    The probability of the operation getting lost is from high
>  >    to low: 1), then 2) and 3), then 4) and 5).
>
>  Very good questions...and the answers are probably tied into how we
>  solve question #1.
>  #5 would seem to be higher performance than #3, but is it a strong
>  enough guarantee?

I think you implied and I agree, that #4 is a stronger guarantee than #5.
But you think #3 is a stronger guarantee than #5? What if one replica
in #5 must be from a different rack? In such a system, if a node is down,
the log on the local disk is gone with it - we don't know when the node
will be up and the log be available again.

If we choose to support all updateable replicas, the problem with simply
indexing and no logging (#1, #2 and #5) is what changes to propagate?

In case of one updateable replica, we simply propagate delta index files.

>  I think the decision partially depends on how much of
>  a document storage system this is, vs just an index that can be
>  rebuilt.

This is a very good point and I have been thinking about this. If we
choose to support all updateable replicas and provide a relatively
strong atomicity support, this can easily be a data storage system.

Ning