Menu
▾
▴
Re: [Hibernate] Plan for a full text search facility built on top ofHibernate Lucene annotations
|
From: Max R. A. <max...@jb...> - 2006-06-01 14:10:10
|
On Thu, 01 Jun 2006 15:47:48 +0200, Emmanuel Bernard = <emm...@hi...> wrote: > I had some additional discussions with Gavin on that yesterday. What w= e > should do first is: > - implement the Lucene update system as a 2nd level cache provider. > It's a much better approach to properly update Lucene indexes Yes. (but you wouldn't always want to resolve objects from this cache; = only put things) > - write / reuse a Criteria API to query the lucene indexes which > returns ids, then use hibernate to resolve the objects Why is another API than Lucene querying needed here ? For encapsulation or is there some performance issue here ? > your object, which is usually not what is needed. I don't know how > Lucene cope with wide amount of data. For certain usages it might be ok. But wouldn't the most often case be to just store enough in the index to query on and get id's ? > I, personally, am not convinced by the partial loading from the index > thing using bytecode enhancement. And it is not trivial to integrate t= o > Hibernate anyway. My solution to the issue is projection over the Luce= ne > querie. Yes, partial objects are hard to handle cleanly. > This architecture would fit nicely with the public opening of the fetc= h > profile Gavin recently proposed. and probably the Cache enhancement > Steve proposed last night :-) Exactly ;) /max > Sylvain Vieujot wrote: >> After chatting with Emmanuel, here is a draft plan for a closer >> integration between Hibernate and Lucene for performing full text = >> queries. >> Hibernate annotations for Lucene helps keeping the lucene indexes up >> to date, but doesn't provide a query facility. >> It also lacks converters that would for example help store a Date wit= h >> the proper format in Lucene, so that the alphabetic order matches the= >> Object's natural order. >> >> A framework like Compass ( http://www.opensymphony.com/compass ) is >> meant to fix this problem, by implementing it's own OSEM (Object, >> Search Engine Mapping), and having a query facility that mimics what >> hibernate is doing with database side. >> Compass can even reuse Hibernate's mapping thus minimizing the >> configuration effort. >> >> One short coming I've found with Compass though is that the objects >> that you get when you query the full text engine aren't connected to >> the ones in the database. >> So if you manipulate them, the changes aren't persisted or can >> actually erase some of the information in the database. >> >> The best way to have a simple and risk free integration is to build a= >> Full Text query facility that would be closely integrated with >> Hibernate & Hibernate Lucene annotations. >> >> So, querying the Full Text indexes would return objects, like Compass= >> does, but those objects would be fetched from the database. >> Actually, for performance reasons, they could be initialized with the= >> information from the FT index, and, through byte code enhancement, if= >> an uninitialized property is read, or a property is set, the real >> object could be fetched from the database and read/set accordingly. >> Here are a few examples : >> >> _1) Just make a full text search :_ >> query "toto" would fetch all the object with an indexed field >> containing toto from the Lucene index. >> If the objects are initialized from the Lucene index, just one >> query to the Lucene index is done, and the search results can be >> displayed. >> =3D> Best performance. >> Loading the objects from the database is useless here, and would >> only lead to poorer performances. >> >> _2) Make a full text search AND manipulate the objects :_ >> You want to query all the objects with "toto", and increment thei= r >> "searchHits" property. >> You do the query, with a Load.EAGER parameter. >> Only the objects' ids are retrieved from Lucene, and the real >> objects are retrieved from Hibernate >> >> _3) Mix both approaches_ >> Requires byte code enhancements. >> Can be useful for cases where for some types of objects you don't= >> want to store all the properties required to display the search >> view results in the index. >> Only those objects will be loaded from Hibernate. >> >> All 3 modes should work, but we can always begin by implementing >> mode 2 only (retrieving only the id's from Lucene, and >> initializing the objects from Hibernate). >> Everything will still work, but performance will not be optimal. >> Later on we can implement mode 3 (which would also solve situatio= n >> 1), and the changes will be transparent to the user. >> Only the performance will be better. >> >> Another advantage of integrating the Full Text query closely with >> Hibernate is that in some cases where a field isn't indexed but the >> query is still simple (fiels x like toto%), the Lucene index would no= t >> be needed, and some queries can be performed directly via Hibernate i= n >> a transparent way for the user. >> >> To summarize this, the biggest changes would be : >> >> - Add converters to Hibernate Lucene annotations, like what >> Compass is doing : >> http://www.opensymphony.com/compass/versions/0.9.1/html/core-sett= ings.html#config-converter >> >> - Build a Full Text query facility similar to Hql / Criteria, but= >> focussed on full text search (also like Compass's one : >> http://www.opensymphony.com/compass/versions/0.9.1/html/core-work= ingwithobjects.html#Searching >> ) but that would make sure the objects retrieved from the Lucene >> index behave as if they were retrieved from the database. >> >> I would be glad to ear from your feedback on this. >> >> Thanks, >> >> Sylvain. >> ---------------------------------------------------------------------= --- >> >> _______________________________________________ >> hibernate-devel mailing list >> hib...@li... >> https://lists.sourceforge.net/lists/listinfo/hibernate-devel >> > > > > _______________________________________________ > hibernate-devel mailing list > hib...@li... > https://lists.sourceforge.net/lists/listinfo/hibernate-devel -- = -- Max Rydahl Andersen callto://max.rydahl.andersen Hibernate ma...@hi... http://hibernate.org JBoss Inc max...@jb... |
