FYI, in bigdata I have use read-write locks in a delegation pattern over the basic bigdata BTree.  The bigdata BTree implementation is safe for concurrent readers (to my knowledge jdbm never tried to ensure that this was true).  The delegation pattern allows code that does not want to use the resource locking mechanisms to allow concurrent readers or a single writer.  It works fine (in my case, which is a different code base), but there is a HUGE performance penalty for small index operations.  I always try to make sure that I am doing some relatively large "batch" operation on the index rather than point tests if I am going to use a read-write lock to prevent concurrent access.  Bigdata uses "index procedures" for this purpose.  Sort of like Java-based stored prodecures.  For bigdata, you can also send them to a remote index (or index partition) for execution.
 
Anyway, my main point is that there is a fairly heavy overhead for locking with a read-write lock and if you really have contention you want to make sure that you are maximizing the work done inside of the lock.
 
-bryan


From: Kevin Day [mailto:kevin@trumpetinc.com]
Sent: Friday, November 14, 2008 7:34 PM
To: JDBM Developer listserv
Subject: Re: [Jdbm-developer] "Fine grained" Read/Write locking

Just a guess:  If you are seeing the Key not found exception, are you maybe accessing the BTree in a non-thread safe way in your test (i.e. you have a list of keys that you are working with, you remove a key in one thread, but do not update the list of keys that you are working with before another thread uses the list).
 
I guess the question is:  Who removed key 16 from the tree, and did they do it without the knowledge of whatever thread is also doing another remove of the key 16 from the tree?  Chances are pretty good that this is a test problem and not a BTree implementation issue (especially b/c you were seeing the problem even with the synchronized blocks in place).
 
 
I think that it's going to be very important to demonstrate the performance advantages of these changes before we roll them into HEAD - especially with the case of the tuple browser (the current API does not forbid anyone from holding onto a tuple browser for a long period of time - if they do so, the browser may not be valid anymore - but it wouldn't lock the tree).
 
If you can come up with some performance tests that we can use to show before and after behavior, I can get the test added to SVN, we can do some performance measurements with the current HEAD build, then perhaps create a branch and get performance metrics with the patch.  Sound good?
 
- K
 

 
 
----------------------- Original Message -----------------------
  
From: "Elias Ross" <genman@noderunner.net>
To: "Kevin Day" <kevin@trumpetinc.com>,  "JDBM Developer listserv" <jdbm-developer@lists.sourceforge.net>
Cc: 
Date: Fri, 14 Nov 2008 13:39:34 -0800
Subject: Re: [Jdbm-developer] "Fine grained" Read/Write locking
  
Re: https://sourceforge.net/tracker/index.php?func=detail&aid=2281809&group_id=4155&atid=304155

I updated the concurrent patch with a small fix to RecMan and changed
BTree to use read-write locks and added an explicit close() method to
the Browser to release the read lock. This is a significant API and
compatibility change but probably one that needs to happen at some
point.

I see this exception occasionally:
java.lang.IllegalArgumentException: Key not found: 16
    at jdbm.btree.BPage.remove(BPage.java:548)
    at jdbm.btree.BPage.remove(BPage.java:561)
    at jdbm.btree.BTree.remove0(BTree.java:436)
    at jdbm.btree.BTree.remove(BTree.java:418)
    at jdbm.btree.Concurrent Test.delete(ConcurrentTest.java:125)
    at jdbm.btree.ConcurrentTest.remove(ConcurrentTest.java:167)

But this happens with the current release of JDBM as well, so I don't
suppose it's a regression.

I haven't measured any improvement with lots of concurrent read-write
but I imagine concurrent reads are faster since they won't block each
other exclusively. This is probably the most important area of
improvement I made.

To improve performance of mutations, some some of optimistic
concurrency could be used, e.g. the appropriate BPage to modify is
first found under a read lock, the version is noted. The read lock is
released, then a write lock is obtained for that BPage, and then a
version check is made if to see if the BPage was since modified. Then
the modification occurs and the lock is released.

-------------------------------------------------------------------------
This SF.Net email is sponso red by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jdbm-developer mailing list
Jdbm-developer@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jdbm-developer