Re: [Exist-open] Locking problem with background process

[Gary L. <gar...@en...>]

I put a sys.out when waiting for a lock in ReentrantReadWriteLock.aquire():
 
  thread http-8080-Processor24 waiting for lock on
/db/netvisn/content/sub42; locks held = 2 by owner:Thread-42
 
The app log shows a hang at storeResource():
 
  DEBUG 79797 [Thread-42] XdbManagerEXist: saveResource() - storeResource on
collection: /db/netvisn/content/sub42
 
 
I tried changing org.exist.xquery.functions.ExtCollection but it did not
help:
 
  private boolean includeSubCollections = false;
to
  private boolean includeSubCollections = true;
 
 
This code is WAY over my head but if very impressive to walk through.  It
seems like this is an eXist problem, but could it be caused by not calling
eXist code properly?
 
I'm in a catch-22 where I need to use sub-collections for delete performance
but can't deal with the hang time.
 
I ended up changing the wait time in ReentrantReadWriteLock:
 
    private long timeOut_ = 240000L;
to
     private long timeOut_ = 4000L;

The hang time is only 8 secs but exceptions are being thrown.  Hopefully
there is a better solution.
 
Thanks,
Gary
 
 


  _____  

From: exi...@li...
[mailto:exi...@li...] On Behalf Of Gary Larsen
Sent: Friday, November 17, 2006 12:19 PM
To: exi...@li...
Subject: Re: [Exist-open] Locking problem with background process



>I don't know anything about locking in eXist, but is it possible that when
storing a resource to a sub-collection a lock is required on the parent
collection also.    
 
Or maybe collection('/db/netvisn/content')/querypath isn't checking for
locks in sub-collections?
 



  _____  

From: exi...@li...
[mailto:exi...@li...] On Behalf Of Gary Larsen
Sent: Friday, November 17, 2006 10:50 AM
To: 'Gary Larsen'; exi...@li...
Subject: Re: [Exist-open] Locking problem with background process


I've just tried eXist-1.0.1-build4311 and have the same problem.  
 
 The lock is occurring when an xQuery is run from the browser thread on a
collection:
 
    collection('/db/netvisn/content')/querypath 
 
while the background thread is storing a resource in a sub-collection:
 
    xmldb:exist:///db/netvisn/content/sub10
 
This problem only started occurring when I needed to spread the resources
across sub-collections due the the performance issue when removing a
resource from a large collection.
 
I cannot make the lock error occur when when storing all resources in the
parent collection. I can consistently make this happen when storing in
sub-collections.
 
I don't know anything about locking in eXist, but is it possible that when
storing a resource to a sub-collection a lock is required on the parent
collection also.    
 
Does this sound like a bug?
 
Thanks for any help here.
Gary


  _____  

From: exi...@li...
[mailto:exi...@li...] On Behalf Of Gary Larsen
Sent: Thursday, November 16, 2006 8:11 PM
To: exi...@li...
Subject: [Exist-open] Locking problem with background process


My Cocoon app has a QuartzScheduler job which adds/deletes eXist resources.
Users can also add/delete resources when navigating the app.  (using
eXist-1.0)
 
I haven't run into this previously, but now I'm getting an occasional lock
which stops the job and the browser. After about 5 minutes the job will
begin processing again.  It could be that I am not managing the adds/deletes
properly but I'm not sure.  Here's the exception:
 
WARN  [http-8080-Processor23] exist.collections.Collection 2006-11-16
19:43:02,640 - time out while acquiring a lock
org.exist.util.LockException: time out while acquiring a lock
 at
org.exist.storage.lock.ReentrantReadWriteLock.acquire(ReentrantReadWriteLock
.java:94)
 at org.exist.collections.Collection.getDocuments(Collection.java:358)
 at org.exist.collections.Collection.allDocs(Collection.java:338)
 at org.exist.collections.Collection.allDocs(Collection.java:319)
 at org.exist.xquery.functions.ExtCollection.eval(ExtCollection.java:129)
 at org.exist.xquery.InternalFunctionCall.eval(InternalFunctionCall.java:49)
 at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:56)
 at org.exist.xquery.PathExpr.eval(PathExpr.java:216)
 at org.exist.xquery.ForExpr.eval(ForExpr.java:139)
 at org.exist.xquery.BindingExpression.eval(BindingExpression.java:109)
 at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:56)
 at org.exist.xquery.PathExpr.eval(PathExpr.java:216)
 at org.exist.xquery.EnclosedExpr.eval(EnclosedExpr.java:66)
 at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:56)
 at org.exist.xquery.PathExpr.eval(PathExpr.java:216)
 at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:233)
 at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:56)
 at org.exist.xquery.PathExpr.eval(PathExpr.java:216)
 at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:56)
 at org.exist.xquery.XQuery.execute(XQuery.java:201)
 at org.exist.xquery.XQuery.execute(XQuery.java:171)
 at
org.exist.xmldb.LocalXPathQueryService.execute(LocalXPathQueryService.java:1
85)
 at org.exist.cocoon.XQueryGenerator.generate(XQueryGenerator.java:328)
 at
org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.processXMLP
ipeline(AbstractProcessingPipeline.java:575)
..................
WARN  [http-8080-Processor23] storage.lock.ReentrantReadWriteLock 2006-11-16
19:43:04,421 - Possible lock problem: thread
Thread[http-8080-Processor23,5,main] released a lock it didn't hold. Either
the thread was interrupted or it never acquired the lock. The lock was owned
by: Thread[Thread-40,5,QuartzScheduler:Cocoon]

Could there be something wrong in the way I'm working with the collection
objects that could be causing this?  Do I need to make sure only one
collection.storeResource is occuring from the app?
 
Thanks,
Gary