AFAIK, Lucene only supports to partition index by document.
But I'm searching a tool that could partition index by term, would Egothor make it?
If not, could you reccommend a tool?
Thanks in advance.
The feature is not yet implemented.
If org.egothor.core.query.Rider class was not implemented for a single barrel, you could have the feature. On the other hand, I cannot see a practical use -- what are you trying to implement?
@leog, for research...Since Lucene doesnot support term-partition, so I'm searching such a tool.
But what do you mean about "If org.egothor.core.query.Rider class was not implemented for a single barrel, you could have the feature", I donot understand(I havenot used egothor before).
Thanks for your reply.
Sorry, I didn't notice your first sentence.
Egothor is able to read inverted lists of a single barrel. If you reimplement the Rider class, then you could read inverted lists of many barrels - this is necessary feature for term-partition.
The modification is simple, but nobody wrote the respective code...
We aim to implement a distributed search engine.
Indices are distrubuted on many machines(the machines may belong to different clusters, so we are limited by the low bandwidth between clusters).We can locate the machines by using hash on the query terms.
IMHO you would rather consider column-oriented (distributed) database. Unfortunately, I am not aware of any robust OSS product in this category.