Re: [Hibernate] Plan for a full text search facility built on top of Hibernate Lucene annotations

[Max R. A. <max...@jb...>]

All sounds cool ;)

I can see the advantage of "converters" which can put elements into  =

Lucence in a better/human manner.

The loading of objects from Lucene + "yet another QL" I'm a bit more  =

critical about.

Would it not be better to do the following:

1. Use whatever QL Lucene supports to express the query. (What does  =

another QL helps here ?)

2. Do the query against the Lucence index and return id's which then is =
 =

resolved via Hibernate
and possible in 2nd lvl cache. (We could maybe optimize the id lookups v=
ia  =

some targetted queries)

3. IFF you really want look into have Lucene be a 2nd lvl cache provider=
 ?  =

(would probably require a "chainable" cacheprovider to have both lucence=
  =

and ehcache queries in the same app...but that is "sugar")

...maybe there is something I miss because I don't understand what the  =

"mixed mode means" and why you
want bytecode enhancement mixed in here ?

/max

> After chatting with Emmanuel, here is a draft plan for a closer
> integration between Hibernate and Lucene for performing full text
> queries.
> Hibernate annotations for Lucene helps keeping the lucene indexes up t=
o
> date, but doesn't provide a query facility.
> It also lacks converters that would for example help store a Date with=

> the proper format in Lucene, so that the alphabetic order matches the
> Object's natural order.
>
> A framework like Compass ( http://www.opensymphony.com/compass ) is
> meant to fix this problem, by implementing it's own OSEM (Object, Sear=
ch
> Engine Mapping), and having a query facility that mimics what hibernat=
e
> is doing with database side.
> Compass can even reuse Hibernate's mapping thus minimizing the
> configuration effort.
>
> One short coming I've found with Compass though is that the objects th=
at
> you get when you query the full text engine aren't connected to the on=
es
> in the database.
> So if you manipulate them, the changes aren't persisted or can actuall=
y
> erase some of the information in the database.
>
> The best way to have a simple and risk free integration is to build a
> Full Text query facility that would be closely integrated with Hiberna=
te
> & Hibernate Lucene annotations.
>
> So, querying the Full Text indexes would return objects, like Compass
> does, but those objects would be fetched from the database.
> Actually, for performance reasons, they could be initialized with the
> information from the FT index, and, through byte code enhancement, if =
an
> uninitialized property is read, or  a property is set, the real object=

> could be fetched from the database and read/set accordingly.
> Here are a few examples :
>
>         1) Just make a full text search :
>         query "toto" would fetch all the object with an indexed field
>         containing toto from the Lucene index.
>         If the objects are initialized from the Lucene index, just one=

>         query to the Lucene index is done, and the search results can =
be
>         displayed.
>         =3D> Best performance.
>         Loading the objects from the database is useless here, and wou=
ld
>         only lead to poorer performances.
>        2) Make a full text search AND manipulate the objects :
>         You want to query all the objects with "toto", and increment
>         their "searchHits" property.
>         You do the query, with a Load.EAGER parameter.
>         Only the objects' ids are retrieved from Lucene, and the real
>         objects are retrieved from Hibernate
>        3) Mix both approaches
>         Requires byte code enhancements.
>         Can be useful for cases where for some types of objects you
>         don't want to store all the properties required to display the=

>         search view results in the index.
>         Only those objects will be loaded from Hibernate.
>        All 3 modes should work, but we can always begin by implementin=
g
>         mode 2 only (retrieving only the id's from Lucene, and
>         initializing the objects from Hibernate).
>         Everything will still work, but performance will not be optima=
l.
>         Later on we can implement mode 3 (which would also solve
>         situation 1), and the changes will be transparent to the user.=

>         Only the performance will be better.
>
> Another advantage of integrating the Full Text query closely with
> Hibernate is that in some cases where a field isn't indexed but the
> query is still simple (fiels x like toto%), the Lucene index would not=

> be needed, and some queries can be performed directly via Hibernate in=
 a
> transparent way for the user.
>
> To summarize this, the biggest changes would be :
>
>         - Add converters to Hibernate Lucene annotations, like what
>         Compass is doing :
>         http://www.opensymphony.com/compass/versions/0.9.1/html/core-s=
ettings.html#config-converter
>        - Build a Full Text query facility similar to Hql / Criteria,
>         but focussed on full text search (also like Compass's one :
>         http://www.opensymphony.com/compass/versions/0.9.1/html/core-w=
orkingwithobjects.html#Searching  =

> ) but that would make sure the objects retrieved from the Lucene index=
  =

> behave as if they were retrieved from the database.
>
> I would be glad to ear from your feedback on this.
>
> Thanks,
>
> Sylvain.



-- =

--
Max Rydahl Andersen
callto://max.rydahl.andersen

Hibernate
ma...@hi...
http://hibernate.org

JBoss Inc
max...@jb...