Menu
▾
▴
#148 Windows crash in IndexWriter::mergeSegments
We have lots of customers reporting CLucence crashes on startup on Windows. Our solution so far has been to tell them to delete their index.
We recently upgraded from clucene 0.9.16 to 0.9.21b and the crash still happens.
Since the same index does not crash on Mac, that would imply that the index is not corrupted, right?
Don't hesitate to contact me if you need more information
Bruno Calvignac
Senior Software Engineer
FLOCK, Inc.
bruno@flock.com
Crash stack
I have an index to reproduce the crash. It's 1.1 MB and too large to attach.
Hi Bruno,
I've also had a similar problem on Windows, though it only occured in debug mode.
To fix it, I added this test:
if (inputLen > 0)
before
reader->norms(fi->name, input);
on line 671 of src/clucene/index/SegmentMerger.cpp.
Hope this helps.
Thank you very much Benoit for your suggestion. I tried it but unfortunately it didn't solve the problem.
Bruno
File Added: p.diff
These are the safeguards I added to fix the crash
By the way, I think the profile I have got corrupted somehow. It does not crash on Mac because it seems that the Memory Manager is more leniant regarding accessing out of bounds memory.
The docCode returned by freqStream->readVInt() in SegmentTermDocs.cpp#120 is -1 which seems like the root of the problem.
Can you please try the 2_3_2 branch, see if it crashes there too?
Where can I get the 2_3_2 branch? Is it the one that 0.9.23 uses? Thanks
From the SVN, under branches. This is what 0.9.23 uses, yes, only since we published that version many other commits were made.
I get the following SVN error when I try to check out: http://clucene.svn.sourceforge.net/viewvc/clucene/branches/lucene2_3_2/
Repository moved temporarily to '/viewvc/clucene/branches/lucene2_3_2/'; please relocate
Just checked and the SVN is working fine... Point TortoiseSVN or a similar client to https://clucene.svn.sourceforge.net/svnroot/clucene/branches/lucene2_3_2/
Also, if this still happens with 2.3.2 too, please generate an index to reproduce the crash with. If you cannot create a small one that you can attach, e-mail it directly to me. Don't forget to explain how to reproduce.
Hello Synhershko,
Thanks for the new link. I am having trouble to compile the 2.3.2 branch under Windows.
I can send you the index by email. What's your address?
Bruno
Not sure why, it works great on my end. In case you're unfamiliar with CMake, download it and use this in a .bat file under branches/lucene2_3_2/msvc8:
echo Deleting previous CMake files...
del cmake_install.cmake
del *.user
del *.vcproj
del clucene.sln
attrib -R -S -H clucene.suo
del clucene.suo /q /f
del clucene.ncb
del cmake_install.cmake
rmdir CMakeFiles /s /q
rmdir src /s /q
rmdir test /s /q
rem Re-create the solution using CMake
"c:\Program Files\cmake\bin\cmake.exe" -G "Visual Studio 8 2005" ..
pause
----
(above is assuming VS2005 and cmake has been extracted to Program Files as is).
You can send the index to the e-mail provided in my profile here. Since it's going to take a while until I'll get to it anyway, I suggest you'll try the 2_3_2 branch again.
i've added the patch to the branch and trunk. does this finalise this bug?
did you find any solution for this?
The attached patch does fix the crash on Windows so we can consider this issue closed.
It would however be better if we didn't need this work around and never get into a situation where the docCode returned by freqStream->readVInt() in SegmentTermDocs.cpp#120 becomes invalid.
committed to branch.,,