From: Itamar Syn-H. <it...@co...> - 2010-09-16 15:24:59
|
Hi folks, Fortunately, development of CLucene has become quite active lately. Unfortunately, things got a bit out of hand and branches weren't merged to master before they staled. I did some work today to improve the situation. More specifically, I merged Veit's work along with Borek and Jiri's (veit_working, intensive_testing and jiri_working -- respectively) to master. I tested it on Win/VC and it worked just fine, so I pushed it as master. And there's Ben's branch. Well, ben's is ben's.... :) This means master now has SpanQueries, Query::extractTerms and many other fixes. It should be even more stable now, unless the merge had errors I did not spot. Veit, Borek and Jiri - please have a look at master and make sure the merge did not ruin your changes (this sometimes happens), and that I resolved all merge conflicts correctly. Once you confirmed the merge, please test the branch as intensively as possible. After you have done this, please remove your merged branches from the repo. From now on, everyone with a repo access bit, please have a branch per feature / fix and make sure it doesn't become stale until it is accepted and merged to master. I'll do my best to merge features and fixes as quickly as possible - having one branch per change is going to make this simpler anyway. Once all the above is done, I suggest we'll start planning forward. Itamar. |
From: Kostka B. <ko...@to...> - 2010-09-17 08:52:14
|
Itamar, I'm not sure, if I understand your strategy for creating branches. Do you mean I should create new branch for each test (and related core fixes, if any)? Borek > -----Original Message----- > From: Itamar Syn-Hershko [mailto:it...@co...] > Sent: Thursday, September 16, 2010 5:25 PM > To: clu...@li... > Subject: [CLucene-dev] Current branches state > Importance: High > > Hi folks, > > > Fortunately, development of CLucene has become quite active lately. > Unfortunately, things got a bit out of hand and branches weren't merged > to master before they staled. > > > I did some work today to improve the situation. More specifically, I > merged Veit's work along with Borek and Jiri's (veit_working, > intensive_testing and jiri_working -- respectively) to master. I tested > it on Win/VC and it worked just fine, so I pushed it as master. > > > And there's Ben's branch. Well, ben's is ben's.... :) > > > This means master now has SpanQueries, Query::extractTerms and many > other fixes. It should be even more stable now, unless the merge had > errors I did not spot. > > > Veit, Borek and Jiri - please have a look at master and make sure the > merge did not ruin your changes (this sometimes happens), and that I > resolved all merge conflicts correctly. Once you confirmed the merge, > please test the branch as intensively as possible. After you have done > this, please remove your merged branches from the repo. > > > From now on, everyone with a repo access bit, please have a branch per > feature / fix and make sure it doesn't become stale until it is accepted > and merged to master. I'll do my best to merge features and fixes as > quickly as possible - having one branch per change is going to make this > simpler anyway. > > > Once all the above is done, I suggest we'll start planning forward. > > > Itamar. > > > ------------------------------------------------------------------------------ > Start uncovering the many advantages of virtual appliances > and start using them to simplify application deployment and > accelerate your shift to cloud computing. > http://p.sf.net/sfu/novell-sfdev2dev > _______________________________________________ > CLucene-developers mailing list > CLu...@li... > https://lists.sourceforge.net/lists/listinfo/clucene-developers |
From: Itamar Syn-H. <it...@co...> - 2010-09-17 09:18:54
|
On 17/9/2010 10:52 AM, Kostka Bořivoj wrote: > Itamar, > > I'm not sure, if I understand your strategy for creating branches. > Do you mean I should create new branch for each test (and related core fixes, if any)? > > Borek The idea is simple - to have fixes and features self-contained in their own branch. Meaning, if you port IndexWriter tests for example, then have them all in their own branch with the fixes you found by them. You get to choose what can be self-contained and what is not, but just keep the general idea in mind. This should result in smaller branches, but obviously sometimes they'll grow big anyway... Itamar. |
From: Kostka B. <ko...@to...> - 2010-09-17 10:19:37
|
We (me and Jiri) did tests on new merged master, as well as chck manually changes. Everything seems to be OK, so we moved to master branch. But some indexing tests on huge data are running now. So we would like to leave our branches in repo until tests finishes, if you don't mind. Once these tests pass, we will remove our old branches from repo. Borek > -----Original Message----- > From: Itamar Syn-Hershko [mailto:it...@co...] > Sent: Friday, September 17, 2010 11:19 AM > To: clu...@li... > Subject: Re: [CLucene-dev] Current branches state > > On 17/9/2010 10:52 AM, Kostka Bořivoj wrote: > > Itamar, > > > > I'm not sure, if I understand your strategy for creating branches. > > Do you mean I should create new branch for each test (and related core fixes, if > any)? > > > > Borek > > The idea is simple - to have fixes and features self-contained in their > own branch. Meaning, if you port IndexWriter tests for example, then > have them all in their own branch with the fixes you found by them. You > get to choose what can be self-contained and what is not, but just keep > the general idea in mind. > > This should result in smaller branches, but obviously sometimes they'll > grow big anyway... > > Itamar. > > ------------------------------------------------------------------------------ > Start uncovering the many advantages of virtual appliances > and start using them to simplify application deployment and > accelerate your shift to cloud computing. > http://p.sf.net/sfu/novell-sfdev2dev > _______________________________________________ > CLucene-developers mailing list > CLu...@li... > https://lists.sourceforge.net/lists/listinfo/clucene-developers |
From: Šplíchal J. <spl...@to...> - 2010-09-17 19:16:22
|
Hi, we encountered one more issue that should be solved - at least in some future version: the FSDirectory::FSIndexOutput constructor may throw exception (this happens i.e. if you are out of disk space) and if it happens the destructor of the base class is called (BufferedIndexOutput) which leads to the call of pure virtual flushBuffer method. And the application crashes. We either should not throw the exception or change the pure virtual BufferedIndexOutput::flushbuffer method into an virtual empty one. Jiri -----Original Message----- From: Kostka Bořivoj [mailto:ko...@to...] Sent: Friday, September 17, 2010 12:19 PM To: clu...@li... Subject: Re: [CLucene-dev] Current branches state We (me and Jiri) did tests on new merged master, as well as chck manually changes. Everything seems to be OK, so we moved to master branch. But some indexing tests on huge data are running now. So we would like to leave our branches in repo until tests finishes, if you don't mind. Once these tests pass, we will remove our old branches from repo. Borek > -----Original Message----- > From: Itamar Syn-Hershko [mailto:it...@co...] > Sent: Friday, September 17, 2010 11:19 AM > To: clu...@li... > Subject: Re: [CLucene-dev] Current branches state > > On 17/9/2010 10:52 AM, Kostka Bořivoj wrote: > > Itamar, > > > > I'm not sure, if I understand your strategy for creating branches. > > Do you mean I should create new branch for each test (and related core fixes, if > any)? > > > > Borek > > The idea is simple - to have fixes and features self-contained in their > own branch. Meaning, if you port IndexWriter tests for example, then > have them all in their own branch with the fixes you found by them. You > get to choose what can be self-contained and what is not, but just keep > the general idea in mind. > > This should result in smaller branches, but obviously sometimes they'll > grow big anyway... > > Itamar. > > ------------------------------------------------------------------------------ > Start uncovering the many advantages of virtual appliances > and start using them to simplify application deployment and > accelerate your shift to cloud computing. > http://p.sf.net/sfu/novell-sfdev2dev > _______________________________________________ > CLucene-developers mailing list > CLu...@li... > https://lists.sourceforge.net/lists/listinfo/clucene-developers ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ CLucene-developers mailing list CLu...@li... https://lists.sourceforge.net/lists/listinfo/clucene-developers |
From: Veit J. <nun...@go...> - 2010-09-19 19:51:32
|
2010/9/16 Itamar Syn-Hershko <it...@co...>: > Veit, Borek and Jiri - please have a look at master and make sure the > merge did not ruin your changes (this sometimes happens), and that I > resolved all merge conflicts correctly. Once you confirmed the merge, > please test the branch as intensively as possible. After you have done > this, please remove your merged branches from the repo. Compiles and runs on FreeBSD as well as Win7/VS2010. But the one of the tests for my StringBuffer extension failed. Fixed it. I needed this extension for the GermanAnalyzer. But I havn't completed the port yet, so the merge of veit_working into master was may be a little bit early. But as this is a contrib no harm will be done to the core. > From now on, everyone with a repo access bit, please have a branch per > feature / fix and make sure it doesn't become stale until it is accepted > and merged to master. I'll do my best to merge features and fixes as > quickly as possible - having one branch per change is going to make this > simpler anyway. But in the case of the failing StringBuffer tests mentioned above I pushed the commits directly to the master branch (sorry). The remaing ported code for the GermanAnalyzer will be pushed to a separate branch. Borek/Jiří, one of you mentioned, that there observed memory leaks in tests. Are they still there? I ask, because veit_working had many memory leak fixes. Veit |
From: Ben v. K. <bva...@gm...> - 2010-09-20 07:45:26
|
I've been testing out the opensuse build service. Very cool in that it allows you to compile your code on different architectures and different distributions. I"ve picked up a few build problems already and fixed them. Check out the results at https://build.opensuse.org/project/monitor?project=home:ustramooner. THere are failures, but i think none of them are problems with clucene (dependencies problems and hardware problems only i think). Anyway, anyone wanting to try out their own branch can have a look at my spec and debian.* (or just branch the whole project), and upload their own code. <https://build.opensuse.org/project/monitor?project=home:ustramooner>Oh, and the results are for the ben_working4 branch, with the testWickedLongTerm test commented out (is someone going to fix that?) ben On Sun, Sep 19, 2010 at 9:51 PM, Veit Jahns <nun...@go...>wrote: > 2010/9/16 Itamar Syn-Hershko <it...@co...>: > > Veit, Borek and Jiri - please have a look at master and make sure the > > merge did not ruin your changes (this sometimes happens), and that I > > resolved all merge conflicts correctly. Once you confirmed the merge, > > please test the branch as intensively as possible. After you have done > > this, please remove your merged branches from the repo. > > Compiles and runs on FreeBSD as well as Win7/VS2010. But the one of > the tests for my StringBuffer extension failed. Fixed it. I needed > this extension for the GermanAnalyzer. But I havn't completed the port > yet, so the merge of veit_working into master was may be a little bit > early. But as this is a contrib no harm will be done to the core. > > > From now on, everyone with a repo access bit, please have a branch per > > feature / fix and make sure it doesn't become stale until it is accepted > > and merged to master. I'll do my best to merge features and fixes as > > quickly as possible - having one branch per change is going to make this > > simpler anyway. > > But in the case of the failing StringBuffer tests mentioned above I > pushed the commits directly to the master branch (sorry). The remaing > ported code for the GermanAnalyzer will be pushed to a separate > branch. > > Borek/Jiří, one of you mentioned, that there observed memory leaks in > tests. Are they still there? I ask, because veit_working had many > memory leak fixes. > > Veit > > > ------------------------------------------------------------------------------ > Start uncovering the many advantages of virtual appliances > and start using them to simplify application deployment and > accelerate your shift to cloud computing. > http://p.sf.net/sfu/novell-sfdev2dev > _______________________________________________ > CLucene-developers mailing list > CLu...@li... > https://lists.sourceforge.net/lists/listinfo/clucene-developers > |
From: Itamar Syn-H. <it...@co...> - 2010-09-20 15:31:23
|
On 20/9/2010 9:44 AM, Ben van Klinken wrote: > I've been testing out the opensuse build service. Very cool in that it > allows you to compile your code on different architectures and > different distributions. I"ve picked up a few build problems already > and fixed them. Check out the results at > https://build.opensuse.org/project/monitor?project=home:ustramooner. > THere are failures, but i think none of them are problems with clucene > (dependencies problems and hardware problems only i think). Nice. > Oh, and the results are for the ben_working4 branch, with > the testWickedLongTerm test commented out (is someone going to fix that?) We agreed in another thread this is not a show stopper at this point (since it is failing due to standardtokenizer incompatibility with JL's), and therefore is left for when the core is at a better shape. Question is what is going to be the fate of your ben_working4 branch, now that all other branches are merged into master? Itamar. |
From: Šplíchal J. <spl...@to...> - 2010-09-20 07:51:05
|
I tested once more our code in order to check the memory leaks. Here is the list of leaking test suites: - testatomicupdates - testindexreader - testanalyzers - testRAMDirectory - testindexwriter - testIndexModifier - testAddIndexesNoOptimize - testsearch - testRangeFilter - testConstantScoreQueries - testsort - testdatefilter - testutf8 - testBitSet - testExtractTerms Something must have changed in the code during the merges because I am sure that at least the testExtractTerms test didn't produce memory leaks. Let us get rid of the leaks! I will start from the bottom with the testExtractTerms test. Should we create a new branch for this issue? Jiri -----Original Message----- From: Veit Jahns [mailto:nun...@go...] Sent: Sunday, September 19, 2010 9:51 PM To: clu...@li... Subject: Re: [CLucene-dev] Current branches state 2010/9/16 Itamar Syn-Hershko <it...@co...>: > Veit, Borek and Jiri - please have a look at master and make sure the > merge did not ruin your changes (this sometimes happens), and that I > resolved all merge conflicts correctly. Once you confirmed the merge, > please test the branch as intensively as possible. After you have done > this, please remove your merged branches from the repo. Compiles and runs on FreeBSD as well as Win7/VS2010. But the one of the tests for my StringBuffer extension failed. Fixed it. I needed this extension for the GermanAnalyzer. But I havn't completed the port yet, so the merge of veit_working into master was may be a little bit early. But as this is a contrib no harm will be done to the core. > From now on, everyone with a repo access bit, please have a branch per > feature / fix and make sure it doesn't become stale until it is accepted > and merged to master. I'll do my best to merge features and fixes as > quickly as possible - having one branch per change is going to make this > simpler anyway. But in the case of the failing StringBuffer tests mentioned above I pushed the commits directly to the master branch (sorry). The remaing ported code for the GermanAnalyzer will be pushed to a separate branch. Borek/Jiří, one of you mentioned, that there observed memory leaks in tests. Are they still there? I ask, because veit_working had many memory leak fixes. Veit ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ CLucene-developers mailing list CLu...@li... https://lists.sourceforge.net/lists/listinfo/clucene-developers |
From: Itamar Syn-H. <it...@co...> - 2010-09-20 15:29:08
|
On 20/9/2010 9:44 AM, Šplíchal Jiří wrote: > Something must have changed in the code during the merges because I am sure > that at least the testExtractTerms test didn't produce memory leaks. > > Let us get rid of the leaks! I will start from the bottom > with the testExtractTerms test. Should we create a new branch for this issue? > Yes - one that is tracking master. Use that same branch as long as you're working on the same issue, and when you start working on anything else that is self contained start a new branch tracking master. Its easy to see what has changed between the branches - and I suspect one of the conflict resolutions during merges was incorrect. Don't forget to remove your old branches when they are not needed anymore! Itamar. |
From: Veit J. <nun...@go...> - 2010-09-22 19:31:54
|
2010/9/20 Šplíchal Jiří <spl...@to...>: > I tested once more our code in order to check the memory leaks. > Here is the list of leaking test suites: > > [...] > > Let us get rid of the leaks! I will start from the bottom Ok. As I was examining memory leaks in CLucene, I observed, that some leaks are caused by the tests themselves. I will take a look what is still left from this effort. > with the testExtractTerms test. Should we create a new branch for this issue? AsI understand Itamar proposal, I'd say yes. Veit |
From: Giorgio Z. <gio...@er...> - 2010-09-22 22:59:47
|
The framework used by CLucene for testing is a crap. In Ericsson, we tried the same framework, but then we moved to cmockery for the c part and cppunit or boost::test for the cpp part. 2010/9/22 Veit Jahns <nun...@go...>: > 2010/9/20 Šplíchal Jiří <spl...@to...>: >> I tested once more our code in order to check the memory leaks. >> Here is the list of leaking test suites: >> >> [...] >> >> Let us get rid of the leaks! I will start from the bottom > > Ok. As I was examining memory leaks in CLucene, I observed, that some > leaks are caused by the tests themselves. I will take a look what is > still left from this effort. > >> with the testExtractTerms test. Should we create a new branch for this issue? > > AsI understand Itamar proposal, I'd say yes. > > Veit > > ------------------------------------------------------------------------------ > Start uncovering the many advantages of virtual appliances > and start using them to simplify application deployment and > accelerate your shift to cloud computing. > http://p.sf.net/sfu/novell-sfdev2dev > _______________________________________________ > CLucene-developers mailing list > CLu...@li... > https://lists.sourceforge.net/lists/listinfo/clucene-developers > -- Quiero ser el rayo de sol que cada día te despierta para hacerte respirar y vivir en me. "Favola -Moda". |
From: Šplíchal J. <spl...@to...> - 2010-09-23 21:19:19
|
Hi, I just published several changes in the memory_leaks branch. Now, following tests have been fixed + testRAMDirectory + testsearch + testRangeFilter + testConstantScoreQueries + testsort + testdatefilter + testBitSet + testExtractTerms. We still have to solve the memory leaks produced by the following tests: - testatomicupdates - testindexreader - testanalyzers - testindexwriter - testIndexModifier - testAddIndexesNoOptimize - testutf8 Please, could someone check the changes especially those in the core files: Array.h, DocumentsWriterThreadState.cpp, IndexWriter.cpp? Jiri -----Original Message----- From: Veit Jahns [mailto:nun...@go...] Sent: Wednesday, September 22, 2010 9:32 PM To: clu...@li... Subject: Re: [CLucene-dev] Current branches state 2010/9/20 Šplíchal Jiří <spl...@to...>: > I tested once more our code in order to check the memory leaks. > Here is the list of leaking test suites: > > [...] > > Let us get rid of the leaks! I will start from the bottom Ok. As I was examining memory leaks in CLucene, I observed, that some leaks are caused by the tests themselves. I will take a look what is still left from this effort. > with the testExtractTerms test. Should we create a new branch for this issue? AsI understand Itamar proposal, I'd say yes. Veit ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ CLucene-developers mailing list CLu...@li... https://lists.sourceforge.net/lists/listinfo/clucene-developers |
From: Veit J. <nun...@go...> - 2010-10-03 11:25:31
|
2010/9/23 Šplíchal Jiří <spl...@to...>: > Hi, > I just published several changes in the memory_leaks branch. Now, following tests have been fixed > + testRAMDirectory Some possible lost of memory and some conditions on uninitialised values in MockRAMDirectory. I think we once talked about. Fixed the last one here. The possible lost memory may be related to the FSDirectory. I remember, that I once observed this issue and solved it by extending the shutdown function. I will take a look. > + testsearch Ok, but some possible lost of memory. The same as in testRAMDirectory. > + testRangeFilter No memory leaks with valgrind. Which commits fixed the memory leaks here? > + testConstantScoreQueries No memory leaks with valgrind. Which commits fixed the memory leaks here? > + testsort No memory leaks with valgrind. Which commits fixed the memory leaks here? > + testdatefilter Ok, but some possible lost of memory. The same as in testRAMDirectory. > + testBitSet Ok. No memory leaks with valgrind. > + testExtractTerms. No memory leaks with valgrind. Which commits fixed the memory leaks here? > We still have to solve the memory leaks produced by the following tests: > - testatomicupdates > - testindexreader > - testanalyzers > - testindexwriter > - testIndexModifier > - testAddIndexesNoOptimize > - testutf8 > > Please, could someone check the changes especially those in the core files: > Array.h, DocumentsWriterThreadState.cpp, IndexWriter.cpp? - RefCountArray::deleteValues() [1]: The gcc misses a template parameter here (corrected it), but the fix seems to be plausible. - ArrayBase<>::resize() [2]: Plausible. - DocumentsWriter::ThreadState::trimFields() [3]: Plausible. - IndexWriter::setMergeScheduler() [4]: Plausible. - IndexWriter::closeInternal() [4]: Here, I don't understand, why deleting rollbackSegmentInfos outside the mutex protected code block fixes a memory leak. Veit [1] http://clucene.git.sourceforge.net/git/gitweb.cgi?p=clucene/clucene;a=commit;h=73bac8355bba837028e532b975d0694fe5caf0c5 [2] http://clucene.git.sourceforge.net/git/gitweb.cgi?p=clucene/clucene;a=commit;h=136ba14e03e08afa71d366dbe7c60538ddb58ae5 [3] http://clucene.git.sourceforge.net/git/gitweb.cgi?p=clucene/clucene;a=blobdiff;f=src/core/CLucene/index/DocumentsWriterThreadState.cpp;h=cd5e6ffcc65ac734ebe726f441cd35ee41483743;hp=eef494622c3eb3d89f367b46ac44723d79433da9;hb=c87131a924443c1c1c66080614ea5fe3dda220ee;hpb=136ba14e03e08afa71d366dbe7c60538ddb58ae5 |
From: Itamar Syn-H. <it...@co...> - 2010-10-07 21:38:47
|
I just had a quick look on Array.h. ArrayBase<T>::resize seems to be flawed in so many ways that I just went ahead and rewrote it. Jiri, your fix was correct, but it wasn't complete (for example: if deleteValues was passed true before values was initialized, it would crash). I checked in what I have done to master since the tests pass and I don't think I could break anything with it. Please test and let me know. I'm a bit swamped atm so I hope to review the rest of the pending items soon. Itamar. On 23/9/2010 11:19 PM, Šplíchal Jiří wrote: > Hi, > I just published several changes in the memory_leaks branch. Now, following tests have been fixed > + testRAMDirectory > + testsearch > + testRangeFilter > + testConstantScoreQueries > + testsort > + testdatefilter > + testBitSet > + testExtractTerms. > > We still have to solve the memory leaks produced by the following tests: > - testatomicupdates > - testindexreader > - testanalyzers > - testindexwriter > - testIndexModifier > - testAddIndexesNoOptimize > - testutf8 > > Please, could someone check the changes especially those in the core files: > Array.h, DocumentsWriterThreadState.cpp, IndexWriter.cpp? > > Jiri > > > -----Original Message----- > From: Veit Jahns [mailto:nun...@go...] > Sent: Wednesday, September 22, 2010 9:32 PM > To: clu...@li... > Subject: Re: [CLucene-dev] Current branches state > > 2010/9/20 Šplíchal Jiří<spl...@to...>: >> I tested once more our code in order to check the memory leaks. >> Here is the list of leaking test suites: >> >> [...] >> >> Let us get rid of the leaks! I will start from the bottom > Ok. As I was examining memory leaks in CLucene, I observed, that some > leaks are caused by the tests themselves. I will take a look what is > still left from this effort. > >> with the testExtractTerms test. Should we create a new branch for this issue? > AsI understand Itamar proposal, I'd say yes. > > Veit > > ------------------------------------------------------------------------------ > Start uncovering the many advantages of virtual appliances > and start using them to simplify application deployment and > accelerate your shift to cloud computing. > http://p.sf.net/sfu/novell-sfdev2dev > _______________________________________________ > CLucene-developers mailing list > CLu...@li... > https://lists.sourceforge.net/lists/listinfo/clucene-developers > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America contest > Create new apps& games for the Nokia N8 for consumers in U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > _______________________________________________ > CLucene-developers mailing list > CLu...@li... > https://lists.sourceforge.net/lists/listinfo/clucene-developers |
From: Kostka B. <ko...@to...> - 2010-10-21 08:23:21
|
Itamar, I found the FSDirectory_fix branch is not merged into master. Small but possibly critical bug is fixed here. I just send this message for case you miseed this fix. Borek > -----Original Message----- > From: Itamar Syn-Hershko [mailto:it...@co...] > Sent: Thursday, October 07, 2010 11:38 PM > To: clu...@li... > Subject: Re: [CLucene-dev] Current branches state > > > > I just had a quick look on Array.h. ArrayBase<T>::resize seems to be > flawed in so many ways that I just went ahead and rewrote it. Jiri, your > fix was correct, but it wasn't complete (for example: if deleteValues > was passed true before values was initialized, it would crash). > > > I checked in what I have done to master since the tests pass and I don't > think I could break anything with it. Please test and let me know. > > > I'm a bit swamped atm so I hope to review the rest of the pending items > soon. > > > Itamar. > > > On 23/9/2010 11:19 PM, Šplíchal Jiří wrote: > > > Hi, > > I just published several changes in the memory_leaks branch. Now, following tests > have been fixed > > + testRAMDirectory > > + testsearch > > + testRangeFilter > > + testConstantScoreQueries > > + testsort > > + testdatefilter > > + testBitSet > > + testExtractTerms. > > > > We still have to solve the memory leaks produced by the following tests: > > - testatomicupdates > > - testindexreader > > - testanalyzers > > - testindexwriter > > - testIndexModifier > > - testAddIndexesNoOptimize > > - testutf8 > > > > Please, could someone check the changes especially those in the core files: > > Array.h, DocumentsWriterThreadState.cpp, IndexWriter.cpp? > > > > Jiri > > > > > > -----Original Message----- > > From: Veit Jahns [mailto:nun...@go...] > > Sent: Wednesday, September 22, 2010 9:32 PM > > To: clu...@li... > > Subject: Re: [CLucene-dev] Current branches state > > > > 2010/9/20 Šplíchal Jiří<spl...@to...>: > >> I tested once more our code in order to check the memory leaks. > >> Here is the list of leaking test suites: > >> > >> [...] > >> > >> Let us get rid of the leaks! I will start from the bottom > > Ok. As I was examining memory leaks in CLucene, I observed, that some > > leaks are caused by the tests themselves. I will take a look what is > > still left from this effort. > > > >> with the testExtractTerms test. Should we create a new branch for this issue? > > AsI understand Itamar proposal, I'd say yes. > > > > Veit > > > > ------------------------------------------------------------------------------ > > Start uncovering the many advantages of virtual appliances > > and start using them to simplify application deployment and > > accelerate your shift to cloud computing. > > http://p.sf.net/sfu/novell-sfdev2dev > > _______________________________________________ > > CLucene-developers mailing list > > CLu...@li... > > https://lists.sourceforge.net/lists/listinfo/clucene-developers > > ------------------------------------------------------------------------------ > > Nokia and AT&T present the 2010 Calling All Innovators-North America contest > > Create new apps& games for the Nokia N8 for consumers in U.S. and Canada > > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing > > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > > http://p.sf.net/sfu/nokia-dev2dev > > _______________________________________________ > > CLucene-developers mailing list > > CLu...@li... > > https://lists.sourceforge.net/lists/listinfo/clucene-developers > > ------------------------------------------------------------------------------ > Beautiful is writing same markup. Internet Explorer 9 supports > standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. > Spend less time writing and rewriting code and more time creating great > experiences on the web. Be a part of the beta today. > http://p.sf.net/sfu/beautyoftheweb > _______________________________________________ > CLucene-developers mailing list > CLu...@li... > https://lists.sourceforge.net/lists/listinfo/clucene-developers |
From: Itamar Syn-H. <it...@co...> - 2011-01-27 20:46:51
|
Borek, All is merged into master now. What is the status of your intensive_testing branch? Itamar. On 21/10/2010 10:23 AM, Kostka Bořivoj wrote: > Itamar, > > I found the FSDirectory_fix branch is not merged into master. Small but possibly critical bug is fixed here. > I just send this message for case you miseed this fix. > > Borek > >> -----Original Message----- >> From: Itamar Syn-Hershko [mailto:it...@co...] >> Sent: Thursday, October 07, 2010 11:38 PM >> To: clu...@li... >> Subject: Re: [CLucene-dev] Current branches state >> >> >> >> I just had a quick look on Array.h. ArrayBase<T>::resize seems to be >> flawed in so many ways that I just went ahead and rewrote it. Jiri, your >> fix was correct, but it wasn't complete (for example: if deleteValues >> was passed true before values was initialized, it would crash). >> >> >> I checked in what I have done to master since the tests pass and I don't >> think I could break anything with it. Please test and let me know. >> >> >> I'm a bit swamped atm so I hope to review the rest of the pending items >> soon. >> >> >> Itamar. >> >> >> On 23/9/2010 11:19 PM, Šplíchal Jiří wrote: >> >>> Hi, >>> I just published several changes in the memory_leaks branch. Now, following tests >> have been fixed >>> + testRAMDirectory >>> + testsearch >>> + testRangeFilter >>> + testConstantScoreQueries >>> + testsort >>> + testdatefilter >>> + testBitSet >>> + testExtractTerms. >>> >>> We still have to solve the memory leaks produced by the following tests: >>> - testatomicupdates >>> - testindexreader >>> - testanalyzers >>> - testindexwriter >>> - testIndexModifier >>> - testAddIndexesNoOptimize >>> - testutf8 >>> >>> Please, could someone check the changes especially those in the core files: >>> Array.h, DocumentsWriterThreadState.cpp, IndexWriter.cpp? >>> >>> Jiri >>> >>> >>> -----Original Message----- >>> From: Veit Jahns [mailto:nun...@go...] >>> Sent: Wednesday, September 22, 2010 9:32 PM >>> To: clu...@li... >>> Subject: Re: [CLucene-dev] Current branches state >>> >>> 2010/9/20 Šplíchal Jiří<spl...@to...>: >>>> I tested once more our code in order to check the memory leaks. >>>> Here is the list of leaking test suites: >>>> >>>> [...] >>>> >>>> Let us get rid of the leaks! I will start from the bottom >>> Ok. As I was examining memory leaks in CLucene, I observed, that some >>> leaks are caused by the tests themselves. I will take a look what is >>> still left from this effort. >>> >>>> with the testExtractTerms test. Should we create a new branch for this issue? >>> AsI understand Itamar proposal, I'd say yes. >>> >>> Veit >>> >>> ------------------------------------------------------------------------------ >>> Start uncovering the many advantages of virtual appliances >>> and start using them to simplify application deployment and >>> accelerate your shift to cloud computing. >>> http://p.sf.net/sfu/novell-sfdev2dev >>> _______________________________________________ >>> CLucene-developers mailing list >>> CLu...@li... >>> https://lists.sourceforge.net/lists/listinfo/clucene-developers >>> ------------------------------------------------------------------------------ >>> Nokia and AT&T present the 2010 Calling All Innovators-North America contest >>> Create new apps& games for the Nokia N8 for consumers in U.S. and Canada >>> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing >>> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store >>> http://p.sf.net/sfu/nokia-dev2dev >>> _______________________________________________ >>> CLucene-developers mailing list >>> CLu...@li... >>> https://lists.sourceforge.net/lists/listinfo/clucene-developers >> ------------------------------------------------------------------------------ >> Beautiful is writing same markup. Internet Explorer 9 supports >> standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2& L3. >> Spend less time writing and rewriting code and more time creating great >> experiences on the web. Be a part of the beta today. >> http://p.sf.net/sfu/beautyoftheweb >> _______________________________________________ >> CLucene-developers mailing list >> CLu...@li... >> https://lists.sourceforge.net/lists/listinfo/clucene-developers > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America contest > Create new apps& games for the Nokia N8 for consumers in U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > _______________________________________________ > CLucene-developers mailing list > CLu...@li... > https://lists.sourceforge.net/lists/listinfo/clucene-developers |
From: Kostka B. <ko...@to...> - 2011-01-27 22:44:00
|
Hi Itamar, I tried to delete branch long time ago, but for some reason it is still listed in GIT and I'm not able to delete it. As everything was merged into master long time ago and I'm not using this branch any more it is safe to delete it. Please feel free to do this, if you can Unfortunately I was busy last few months, so I didn't port other tests. Hope I can spend some time next month Borek > -----Original Message----- > From: Itamar Syn-Hershko [mailto:it...@co...] > Sent: Thursday, January 27, 2011 9:20 PM > To: clu...@li... > Subject: Re: [CLucene-dev] Current branches state > > Borek, > > > All is merged into master now. > > > What is the status of your intensive_testing branch? > > > Itamar. > > > On 21/10/2010 10:23 AM, Kostka Bořivoj wrote: > > > Itamar, > > > > I found the FSDirectory_fix branch is not merged into master. Small but possibly > critical bug is fixed here. > > I just send this message for case you miseed this fix. > > > > Borek > > > >> -----Original Message----- > >> From: Itamar Syn-Hershko [mailto:it...@co...] > >> Sent: Thursday, October 07, 2010 11:38 PM > >> To: clu...@li... > >> Subject: Re: [CLucene-dev] Current branches state > >> > >> > >> > >> I just had a quick look on Array.h. ArrayBase<T>::resize seems to be > >> flawed in so many ways that I just went ahead and rewrote it. Jiri, your > >> fix was correct, but it wasn't complete (for example: if deleteValues > >> was passed true before values was initialized, it would crash). > >> > >> > >> I checked in what I have done to master since the tests pass and I don't > >> think I could break anything with it. Please test and let me know. > >> > >> > >> I'm a bit swamped atm so I hope to review the rest of the pending items > >> soon. > >> > >> > >> Itamar. > >> > >> > >> On 23/9/2010 11:19 PM, Šplíchal Jiří wrote: > >> > >>> Hi, > >>> I just published several changes in the memory_leaks branch. Now, following tests > >> have been fixed > >>> + testRAMDirectory > >>> + testsearch > >>> + testRangeFilter > >>> + testConstantScoreQueries > >>> + testsort > >>> + testdatefilter > >>> + testBitSet > >>> + testExtractTerms. > >>> > >>> We still have to solve the memory leaks produced by the following tests: > >>> - testatomicupdates > >>> - testindexreader > >>> - testanalyzers > >>> - testindexwriter > >>> - testIndexModifier > >>> - testAddIndexesNoOptimize > >>> - testutf8 > >>> > >>> Please, could someone check the changes especially those in the core files: > >>> Array.h, DocumentsWriterThreadState.cpp, IndexWriter.cpp? > >>> > >>> Jiri > >>> > >>> > >>> -----Original Message----- > >>> From: Veit Jahns [mailto:nun...@go...] > >>> Sent: Wednesday, September 22, 2010 9:32 PM > >>> To: clu...@li... > >>> Subject: Re: [CLucene-dev] Current branches state > >>> > >>> 2010/9/20 Šplíchal Jiří<spl...@to...>: > >>>> I tested once more our code in order to check the memory leaks. > >>>> Here is the list of leaking test suites: > >>>> > >>>> [...] > >>>> > >>>> Let us get rid of the leaks! I will start from the bottom > >>> Ok. As I was examining memory leaks in CLucene, I observed, that some > >>> leaks are caused by the tests themselves. I will take a look what is > >>> still left from this effort. > >>> > >>>> with the testExtractTerms test. Should we create a new branch for this issue? > >>> AsI understand Itamar proposal, I'd say yes. > >>> > >>> Veit > >>> > >>> ------------------------------------------------------------------------------ > >>> Start uncovering the many advantages of virtual appliances > >>> and start using them to simplify application deployment and > >>> accelerate your shift to cloud computing. > >>> http://p.sf.net/sfu/novell-sfdev2dev > >>> _______________________________________________ > >>> CLucene-developers mailing list > >>> CLu...@li... > >>> https://lists.sourceforge.net/lists/listinfo/clucene-developers > >>> ------------------------------------------------------------------------------ > >>> Nokia and AT&T present the 2010 Calling All Innovators-North America contest > >>> Create new apps& games for the Nokia N8 for consumers in U.S. and Canada > >>> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing > >>> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > >>> http://p.sf.net/sfu/nokia-dev2dev > >>> _______________________________________________ > >>> CLucene-developers mailing list > >>> CLu...@li... > >>> https://lists.sourceforge.net/lists/listinfo/clucene-developers > >> ------------------------------------------------------------------------------ > >> Beautiful is writing same markup. Internet Explorer 9 supports > >> standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2& L3. > >> Spend less time writing and rewriting code and more time creating great > >> experiences on the web. Be a part of the beta today. > >> http://p.sf.net/sfu/beautyoftheweb > >> _______________________________________________ > >> CLucene-developers mailing list > >> CLu...@li... > >> https://lists.sourceforge.net/lists/listinfo/clucene-developers > > ------------------------------------------------------------------------------ > > Nokia and AT&T present the 2010 Calling All Innovators-North America contest > > Create new apps& games for the Nokia N8 for consumers in U.S. and Canada > > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing > > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > > http://p.sf.net/sfu/nokia-dev2dev > > _______________________________________________ > > CLucene-developers mailing list > > CLu...@li... > > https://lists.sourceforge.net/lists/listinfo/clucene-developers > > ------------------------------------------------------------------------------ > Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! > Finally, a world-class log management solution at an even better price-free! > Download using promo code Free_Logger_4_Dev2Dev. Offer expires > February 28th, so secure your free ArcSight Logger TODAY! > http://p.sf.net/sfu/arcsight-sfd2d > _______________________________________________ > CLucene-developers mailing list > CLu...@li... > https://lists.sourceforge.net/lists/listinfo/clucene-developers |
From: Itamar Syn-H. <it...@co...> - 2011-01-29 16:22:17
|
I removed that branch. Itamar. On 28/1/2011 12:43 AM, Kostka Bořivoj wrote: > Hi Itamar, > > I tried to delete branch long time ago, but for some reason it is still listed in GIT and I'm not able to delete it. As everything was merged into master long time ago > and I'm not using this branch any more it is safe to delete it. Please feel free to do this, if you can > > Unfortunately I was busy last few months, so I didn't port other tests. Hope I can spend some time next month > > Borek > >> -----Original Message----- >> From: Itamar Syn-Hershko [mailto:it...@co...] >> Sent: Thursday, January 27, 2011 9:20 PM >> To: clu...@li... >> Subject: Re: [CLucene-dev] Current branches state >> >> Borek, >> >> >> All is merged into master now. >> >> >> What is the status of your intensive_testing branch? >> >> >> Itamar. >> >> >> On 21/10/2010 10:23 AM, Kostka Bořivoj wrote: >> >>> Itamar, >>> >>> I found the FSDirectory_fix branch is not merged into master. Small but possibly >> critical bug is fixed here. >>> I just send this message for case you miseed this fix. >>> >>> Borek >>> >>>> -----Original Message----- >>>> From: Itamar Syn-Hershko [mailto:it...@co...] >>>> Sent: Thursday, October 07, 2010 11:38 PM >>>> To: clu...@li... >>>> Subject: Re: [CLucene-dev] Current branches state >>>> >>>> >>>> >>>> I just had a quick look on Array.h. ArrayBase<T>::resize seems to be >>>> flawed in so many ways that I just went ahead and rewrote it. Jiri, your >>>> fix was correct, but it wasn't complete (for example: if deleteValues >>>> was passed true before values was initialized, it would crash). >>>> >>>> >>>> I checked in what I have done to master since the tests pass and I don't >>>> think I could break anything with it. Please test and let me know. >>>> >>>> >>>> I'm a bit swamped atm so I hope to review the rest of the pending items >>>> soon. >>>> >>>> >>>> Itamar. >>>> >>>> >>>> On 23/9/2010 11:19 PM, Šplíchal Jiří wrote: >>>> >>>>> Hi, >>>>> I just published several changes in the memory_leaks branch. Now, following tests >>>> have been fixed >>>>> + testRAMDirectory >>>>> + testsearch >>>>> + testRangeFilter >>>>> + testConstantScoreQueries >>>>> + testsort >>>>> + testdatefilter >>>>> + testBitSet >>>>> + testExtractTerms. >>>>> >>>>> We still have to solve the memory leaks produced by the following tests: >>>>> - testatomicupdates >>>>> - testindexreader >>>>> - testanalyzers >>>>> - testindexwriter >>>>> - testIndexModifier >>>>> - testAddIndexesNoOptimize >>>>> - testutf8 >>>>> >>>>> Please, could someone check the changes especially those in the core files: >>>>> Array.h, DocumentsWriterThreadState.cpp, IndexWriter.cpp? >>>>> >>>>> Jiri >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: Veit Jahns [mailto:nun...@go...] >>>>> Sent: Wednesday, September 22, 2010 9:32 PM >>>>> To: clu...@li... >>>>> Subject: Re: [CLucene-dev] Current branches state >>>>> >>>>> 2010/9/20 Šplíchal Jiří<spl...@to...>: >>>>>> I tested once more our code in order to check the memory leaks. >>>>>> Here is the list of leaking test suites: >>>>>> >>>>>> [...] >>>>>> >>>>>> Let us get rid of the leaks! I will start from the bottom >>>>> Ok. As I was examining memory leaks in CLucene, I observed, that some >>>>> leaks are caused by the tests themselves. I will take a look what is >>>>> still left from this effort. >>>>> >>>>>> with the testExtractTerms test. Should we create a new branch for this issue? >>>>> AsI understand Itamar proposal, I'd say yes. >>>>> >>>>> Veit >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> Start uncovering the many advantages of virtual appliances >>>>> and start using them to simplify application deployment and >>>>> accelerate your shift to cloud computing. >>>>> http://p.sf.net/sfu/novell-sfdev2dev >>>>> _______________________________________________ >>>>> CLucene-developers mailing list >>>>> CLu...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/clucene-developers >>>>> ------------------------------------------------------------------------------ >>>>> Nokia and AT&T present the 2010 Calling All Innovators-North America contest >>>>> Create new apps& games for the Nokia N8 for consumers in U.S. and Canada >>>>> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing >>>>> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store >>>>> http://p.sf.net/sfu/nokia-dev2dev >>>>> _______________________________________________ >>>>> CLucene-developers mailing list >>>>> CLu...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/clucene-developers >>>> ------------------------------------------------------------------------------ >>>> Beautiful is writing same markup. Internet Explorer 9 supports >>>> standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2& L3. >>>> Spend less time writing and rewriting code and more time creating great >>>> experiences on the web. Be a part of the beta today. >>>> http://p.sf.net/sfu/beautyoftheweb >>>> _______________________________________________ >>>> CLucene-developers mailing list >>>> CLu...@li... >>>> https://lists.sourceforge.net/lists/listinfo/clucene-developers >>> ------------------------------------------------------------------------------ >>> Nokia and AT&T present the 2010 Calling All Innovators-North America contest >>> Create new apps& games for the Nokia N8 for consumers in U.S. and Canada >>> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing >>> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store >>> http://p.sf.net/sfu/nokia-dev2dev >>> _______________________________________________ >>> CLucene-developers mailing list >>> CLu...@li... >>> https://lists.sourceforge.net/lists/listinfo/clucene-developers >> ------------------------------------------------------------------------------ >> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! >> Finally, a world-class log management solution at an even better price-free! >> Download using promo code Free_Logger_4_Dev2Dev. Offer expires >> February 28th, so secure your free ArcSight Logger TODAY! >> http://p.sf.net/sfu/arcsight-sfd2d >> _______________________________________________ >> CLucene-developers mailing list >> CLu...@li... >> https://lists.sourceforge.net/lists/listinfo/clucene-developers > ------------------------------------------------------------------------------ > Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! > Finally, a world-class log management solution at an even better price-free! > Download using promo code Free_Logger_4_Dev2Dev. Offer expires > February 28th, so secure your free ArcSight Logger TODAY! > http://p.sf.net/sfu/arcsight-sfd2d > _______________________________________________ > CLucene-developers mailing list > CLu...@li... > https://lists.sourceforge.net/lists/listinfo/clucene-developers |
From: Šplíchal J. <spl...@to...> - 2010-10-03 20:14:54
|
Hi, > - IndexWriter::closeInternal() [4]: Here, I don't understand, why > deleting rollbackSegmentInfos outside the mutex protected code block > fixes a memory leak. the rollbackSegmentInfos are not deleted outside of the mutex protected block it happens still inside of it, but outside of the "if (commitPending)" block. Jiri |
From: Veit J. <nun...@go...> - 2010-10-10 10:17:03
|
2010/10/3 Šplíchal Jiří <spl...@to...>: >> - IndexWriter::closeInternal() [4]: Here, I don't understand, why >> deleting rollbackSegmentInfos outside the mutex protected code block >> fixes a memory leak. > > the rollbackSegmentInfos are not deleted outside of the mutex protected block > it happens still inside of it, but outside of the "if (commitPending)" block. Now, I see it. Thanks. Veit |