#221 Memory leak in Analyzer::reusableTokenStream() call

core (32)

In file src\core\CLucene\index\DocumentsWriterThreadState.cpp, function void DocumentsWriter::ThreadState::FieldData::invertField()
around line 892, the stream = analyzer->reusableTokenStream(fieldInfo->name, reader); call is supposed to create a stream "reusable", but most of the analyzers are just creating a new stream.

For now I am not sure how to implement the reusable Token stream correctly (maybe should read latest Lucene code), but in my local build I just delete the stream and the memory leak is gone.


  • Xiaoman Dong

    Xiaoman Dong - 2013-04-04
    • priority: 5 --> 9
  • Xiaoman Dong

    Xiaoman Dong - 2013-04-04

    I would like to help with this issue and borrow ideas from Lucene latest development.

    The multi-thread support is a good improvement for my project. The bottleneck are actually string inverting and using more threads will reduce time cost.


Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.

No, thanks