One more thing that i would like to add is the changes are really working as desired in the actual application that i had problems with.

Thanks for your support.


Regards,
Naveen


On Feb 14, 2008 11:13 PM, Kevin Day <kevin@trumpetinc.com> wrote:
Alex-
 
All tests in the Test.java test suite pass for me, and I have confirmed that the adjustement you make addresses Naveen's situation.  I would still like to get Naveen to shoot me a test case so we can get test coverage on this, but I have made Alex's change and commited it to CVS HEAD.
 
- K
 
----------------------- Original Message -----------------------
  
From: "Alex Boisvert" <boisvert@intalio.com>
To: "Kevin Day" <kevin@trumpetinc.com>
Cc: "JDBM Developer listserv" <jdbm-developer@lists.sourceforge.net>
Date: Thu, 14 Feb 2008 09:32:08 -0800
Subject: Re: [Jdbm-developer] Resending to both of you as this msg exceeds mailing list msg limit
  
I took a quick look at Naveen's test and the BTree code and found that we didn't correctly release the record for the root page of the BTree:

Index: src/main/jdbm/btree/BTree.java
===================================================================
RCS file: /cvsroot/jdbm/jdbm/src/main/jdbm/btree/BTree.java,v
retrieving revision 1.17
diff -c -r1.17 BTree.java
*** src/main/jdbm/btree/BTree.java      31 May 2006 22:25:31 -0000      1.17
--- src/main/jdbm/btree/BTree.java      14 Feb 2008 17:07:29 -0000
***************
*** 396,402 ****
              _height -= 1;
              dirty = true;

!             // TODO:  check contract for BPages to be removed from recman.
              if ( _height == 0 ) {
                  _root = 0;
              } else {
--- 396,402 ----
              _height -= 1;
              dirty = true;

!               _recman.delete(_root);
              if ( _height == 0 ) {
                  _root = 0;
              } else {

So the leak appears to be specific to the case where the BTree is emptied and the root page replaced.  With this change, I don't see any leak with his test case anymore.

Unfortunately, the test cases don't all pass on my machine right now so I hesitate to commit the change until I can be assured that I'm not breaking anything else.   (The failing test don't seem to be released to the code I've changed, though)

   [junit] Tests FAILED
The following tests failed:
jdbm.helper.TestWeakCache
jdbm.recman.TestBufferedRecordInstallManager

Kevin, are all the tests working for you?  I can provide details about the failures if needed.

alex


On 2/14/08, Kevin Day <kevin@trumpetinc.com> wrote:
Naveen-
 
This is pretty basic jdbm stuff - but your code certainly doesn't work with the latest version of jdbm (what you are using looks really, really old...).  Can you please make sure you are running jdbm 1.0 or higher?  http://jdbm.sourceforge.net/
 
All that said, what you really need to provide is a unit test case that can run and show a failure.  If you are not familiar with writing JUnit tests, the JUnit cookbook can get you started:  http://junit.sourceforge.net/doc/cookbook/cookbook.htm
 
Most likely you are going to want the unit test to check the size of the .db file before and after your operations and show that it continues to grow, regardless of the number of insert/remove cycles you perform.
 
 
Once I adjusted your code so it would compile and work with the jdbm 1.0 libraries, I manually watched the size of the db file, and I can confirm that it does appear to be ever-increasing - but in order for this to be useful for development purposes, you've got to create a test case.  The test case will show a failure (presumably), then we can all have a discussion about *why* the test case is failing.
 
So the first step is a unit test that results in a failure that demonstrates the problem.  Without that, the testing can't be automated.
 
If you haven't created test cases before, take a stab at it, and send it to just me (no need to bug Alex with it).  I'll do a code review and make suggestions, then once we have a solid unit test, you can submit it.
 
 
For your reference, her is your code adjusted to work with jdbm 1.0 (I have added comments starting with 'NAVEEN - '  in places that you'll have to adjust to convert this into a unit test.
 
 
package jdbm.btree;
import java.io.PrintStream;
import java.util.Properties;
import jdbm.RecordManager;
import jdbm.RecordManagerFactory;
import jdbm.helper.StringComparator;
import jdbm.helper.Tuple;
import jdbm.helper.TupleBrowser;
public class DBSizeTest {
    static String DATABASE = "testpeople";
    static String BTREE_NAME = "FamousPeople";
    static String[] people =
        { "Greenspan, Alan",
          "Williams-Byrd, Julie",
          "Picasso, Pablo",
        "Stallman, Richard",
          "Fort, Paul",
         "Søndergaard, Ole",
          "Schwarzenegger, Arnold",
        "Dulkinys, Susanna"
          };
    static String[] occupations =
        { "Federal Reserve Board Chairman",
          "Engineer",
          "Painter",
          "Programmer",
          "Poet",
          "Typographer",
         "Actor",
          "Designer"
        };
    static String PREFIX = "S";

    /**
     * Example main entrypoint.
     */
   
    // NAVEEN - need to change this to be a JUnit compatible method signature
    public static void main( String[] args ) {
        RecordManager recman;
        BTree         tree;
        long          recid;
        Tuple         tuple = new Tuple();
        TupleBrowser  browser;
        try {
            // open database and setup an object cache
            Properties props = new Properties();
            recman = RecordManagerFactory.createRecordManager( DATABASE );
            // try to reload an existing B+Tree
            recid = recman.getNamedObject( BTREE_NAME );
            if ( recid != 0 ) {
                tree = BTree.load( recman, recid );
                System.out.println( "Reloaded existing BTree with " + tree.size()
                                    + " famous people." );
            } else {
       &n bsp;   &n bsp;    // create a new B+Tree data structure and use a StringComparator

                // to order the records based on people's name.
                tree = BTree.createInstance(recman, new StringComparator());
                recman.setNamedObject( BTREE_NAME, tree.getRecid() );
                System.out.println( "Created a new empty BTree" );
                              
               
    &nbs p;       }
           
            // NAVEEN - need to get the size of the DB file here
            for (int i = 0; i < 500; i++){
 
                    // insert people with their respective occupation
                for ( int i=0; i<people.length; i++ ) {
                    System.out.println( "Insert: " + people[i] );
                    tree.insert( peopl e[ i ], occupations[ i ], true );< BR>                }

               
                for ( int i=0; i<people.length; i++ ) {
                    System.out.println( "Remove: " + people[i] );
                    tree.remove( people[ i ]);
                }
   
                // make the data persistent in the database
        &nb sp;       r ecman.commit();

            }
           
            // NAVEEN - need to get the size of the db file here and compare it to the original size using an asserTrue() call
        } catch ( Exception except ) {
            except.printStackTrace();
        }
    }

}
 
 
- K
 
 
----------------------- Original Message -----------------------
  
From: "NAVEEN UPADHYAY" <naveen.invincible@gmail.com>
To: "Alex Boisvert" <boisvert@intalio.com>, "Kevin Day" <kevin@trumpetinc.com>
Cc: 
Date: Thu, 14 Feb 2008 17:31:04 +0530
Subject: Resending to both of you as this msg exceeds mailing list msg limit
  
Hi Alex/Kevin,

    Here is the sample program that illustrates the problem.
All that i am doing is adding and removing data from the same Btree.
Ideally such a case should keep the size of the .db file within a threshold.
But as you can see (while running this program) this is NOT the case.
The file size is ever increasing.

I am wondering whether its a problem with JDBM or something wrong at my end.
In case any more input is required i will be online for the next 10 hours .
sagi_naveen@yahoo.com

Please look into this.

Thanks in advance,
Naveen
PS- there is a specific reason why my application has to remove data after every poll cycle.
If you want to know my application logic i can provide the same .However the given sample program(attached) i s itself sufficient!

PSS- The file can be put in /src/examples folder of jdbm-0.12.jar

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Jdbm-developer mailing list
Jdbm-developer@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jdbm-developer



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Jdbm-developer mailing list
Jdbm-developer@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jdbm-developer