Re: [bailey-developers] lattice master

[Doug C. <cu...@ap...>]

Ning Li wrote:
> A copy of a document will be stored in a stored field, right?

If we don't keep complete documents in the log, then I figured that all 
fields would be stored, so that document retrieved from the index could 
be directly added to another index.

> First, keeping both a doc and its inverted form in an index
> means storing the doc and indexing the doc are done in the
> same "transaction". A traditional document database often
> store a doc first and then index it later (hopefully soon).

It should be possible to configure Lucene so that adding a document 
never triggers any blocking merges.  So folks would have to wait for the 
single-document inversion, but no more.  Today, it won't yet be visible 
to search for a bit, until a commit, but perhaps in a future version of 
Lucene it might be possible to search unflushed segments.

> Second, a traditional document database often supports
> updating a doc's "metadata" such as author or date. We
> don't support this or we say a document is name-value
> pairs and we reconstruct from stored fields and support
> such update?

I assume that incrementally updateable fields must not be indexed?  In 
any case, I hope that indexing is fast enough that updating by 
add+delete is okay.  CouchDB seems to get away with it...

> #5 with non-sync light-weight logging? Should work.

Yes.

Doug