Re: [CLucene-dev] newbie confusion

[Isidor Z. <cl...@qu...>]

Hello Mateusz,

> I'm looking for a full-text search engine, and recently got interested
> in CLucene. However, I'm having some troubles with it, apparently
> mainly with the memory management. For starters, I've downloaded the
> "stable" 0.9.21 package, but trying to run the "cl_demo" (I'm using MS
> VC++ 2008 Express, compiling the default solution files provided,
> auto-converted to 2008 format) results in some memory leaks reported
> thanks to the "_CrtSetDbgFlag()" call. Now, that makes me feel uneasy:
> shouldn't the demo app be running perfectly clean? is this a problem
> with the app, with the library, with VC++, with some compiler flags,
> or what?
> 

Can't tell much about 0.9.21 as I'm only working with the
development branch, but cl_demo isn't supposed to leak memory. So
you might want to report a bug on our tracker.

For starters, you might also want to have a look at the test suite, as
it contains very different uses of the CLucene API. For me, it proved
as a more useful resource to learn how to use CLucene than cl_demo.

> A bigger problem for me is that I'm overally pretty confused on how to
> allocate objects when using CLucene. The demo app again seems pretty
> inconsistent: it mostly uses the _CLNEW macro, but sometimes normal
> local objects (e.g. when instantiating the StandardAnalyzer). Which
> approaches can I use, or should I use, and why? And what about the
> regular "new" operator? Now, I thought I should be pretty safe with a
> simple "all objects auto/local" policy, which I believe to be default
> for C++; but I've tried to reduce the demo app to some simpler one
> using that policy and it still reports errors on runtime.
> 

As of the current development branch, most classes are initialized and
destroyed using _CLNEW and _CLDELETE for historical reasons (_CLNEW is
now #define-ed as "new" anyway). This is due to the reference counting
implementation previously used. We are currently in the process of
using boost::shared_ptr to replace this, which will allow us to go
back to the more C++-like scope-based memory management.

> Furthermore, the demo ends in a couple of functions like
> "_lucene_shutdown()" and "__cl_ClearMemory()" - are they mandatory,
> and both of them? And if so, then I believe that's quite confusing for
> them to start with an underscore.
> 

As of the current development branch, I think __cl_ClearMemory() is
not used anymore, but an application should still call
_lucene_shutdown() to clean up.

I think you raise some very valid points regarding the confusing
memory management and the underscore-prefixed API function. I hope to
get some more developer feedback on these issues and find better
solutions for them.

> Finally, having looked over several posts in this mailing list, I have
> an impression that the main work is now on the "2.3.2" branch, and
> that it is meant to be soon released oficially to the public. Is that
> so? Should I abandon the "stable public 0.9.21" branch and change to
> the new one, and can I expect it to be easier and more stable/less
> surprising?
> 

We had the same concern some weeks ago on the mailing
list. Personally, I'd say the current git development code is far
more useful for 2009 search engine application development. It is
still a work in progress and API changes are still possible, but it is
already very stable and apart from that, way faster and more powerful
than the previous stable release.

> I'd be grateful for any help, also for pointing it out if I'm simply
> doing something wrong.

I hope my comments were of some help. Maybe another developer more
experienced with the stable release can give you more comments
regarding that codebase.

I appreciate your observations because I think they allow us to
improve documentation and API architecture.

Best regards,

Isidor