Primitive hashmaps...

Kevin Day
2005-08-11
2013-06-03
  • Kevin Day
    Kevin Day
    2005-08-11

    I think someone asked about this in an earlier post.  I strongly suspect that there are areas of jdbm that would be greatly helped by the implementation of a primitive hashmap...  There are a bunch of short lived Long objects that get created in many different levels of the framework...

    Anyway, I came across the following SF project that may be of interest:

    http://pcj.sourceforge.net/

    Would anyone have any objection to adding a dependency to another project like this?  Licensing is LGPL, so I don't think we can just rip the classes we want...

    - K

     
    • Bryan Thompson
      Bryan Thompson
      2005-08-11

      Kevin,

      That was me.  This could be worthwhile, but it would require an audit of jdbm to make sure that it was able to handle long's all the way down.  I suggest that this be combined with a modification of the API to support nio all the way down as well.

      I see this as something that I would be interested in for a 2.x development.  For the current code base, I would like to see the problem with the dirty page transaction buffer fixed (i'll take a look at your patch, but just calling commit() automatically is not a "fix" since it changes the API semantics by changing the rollback checkpoint, though it does keep the memory leak in check).

      The other problem that I would like to see fixed is the eager cache eviction that we have discussed before.  I consider this a bug since it can lead to multiple surrogates for the same persistent object in memory at the same time.  Ouch.

      Given that these changes may take a while to validate, perhaps the best thing to do is to file some bug reports on jdbm and do a 1.0 release of the current code base with known bugs?

      -bryan

       
    • Kevin Day
      Kevin Day
      2005-08-11

      I disagree that the patch changes the API semantics.  If you disable transactions, then there is no rollback.  The API docs say nothing about the roll-back consequences of disabling transactions.

      I would say that we are perfectly justified in clarifying the API so it is consistent with the behavior of most other database packages where disabling transactions disables roll-back.  In fact, in most other OODB implementations (as well as JDO, hibernate and most ORB implementations), the roll-back is always executed within a transaction object - if you have no transaction, then you can't even call rollback().

      I believe I remember reading somewhere that transaction support in jdbm is there to ensure that the database file will never be corrupted, even if the application crashes in the middle of a write.  It was never intended to allow for large scale roll-backs by the user....

      Now that I think about it, the memory leak concern that you have really has nothing to do with transactions being turned on or off - it has to do with the size of any given transaction.  You would run into the same problem if you were doing any batch change to the database without calling commit() (regadless of whether transactions are enabled or not).

      Alex may need to weigh in here regarding actual design intent...  but I'm pretty sure the patch I'm proposing is consistent with the design intent...  Adding support for large scale roll-back of transactions is another matter entirely...

      - K

       
      • Alex Boisvert
        Alex Boisvert
        2005-08-11

        While I can't speak for Cees (the original author of the RecordManager), my understanding is consistent with Kevin's:  if you disable transactions, then you effectively forgo rollback semantics.

        So I don't think Kevin's patch breaks the intended API contract.

        alex

         
    • Alex Boisvert
      Alex Boisvert
      2005-08-11

      I'm in favor of using primite hash maps.

      However, looking at PCJ, it appears the jar size is 2.5M.  That's significant compared to JDBM's size of 86K.

      If all we need is a native hash map for long's, I'd say we write it ourselves.

      alex

       
    • Kevin Day
      Kevin Day
      2005-08-11

      Yikes - didn't realize it was that big!

      I'll get with the team over there to see if they'd be OK with us just pulling the components we need.

      - K

       
    • Kevin Day
      Kevin Day
      2005-08-13

      FYI - I heard back from Sren about including a subset of the PCJ primitive collection library directly in jdbm.  His response:

      "Hi Kevin!

      You may be right about LGPL, but I have no objections to
      your including a subset of PCJ in JDBM.

      - Sren"

      So it looks like we can do this and take advantage of the work that Sren has put into those primitive collections.

      Cheers,

      - K

       
    • Kevin Day
      Kevin Day
      2005-08-23

      I have submitted a starting implementation of a primitive long -> object hashmap if anyone wants to start playing.

      Interestingly, the only place in jdbm that uses hashmap to map longs to objects is the RecordFile class (I *love* eclipse :-)  ).

      It should be fairly trivial to swap those out...

      I'll give it a try, and if it's succesful, I'll upload the code.

      - K