Re: [Exist-development] [Exist-open] Performance of concurrent read queries.

[Jason S. <js...@in...>]

Wolfgang, if you are interested, I could merge my code into a branch of eXist (not intended for human consumption), so that we could talk using concrete examples. In particular, about improving the locking mechanism so that it actually works (mine is more of a proof of concept, not a final solution).  It looks to me that a number of the concurrency limitations in eXist originate from having to work around the limitations of the current locking mechanism.

Again, I would not expect a final solution to be implemented the way I have done it.  I just want to be able to show some possibilities.

I am not expecting this conversation to take place quickly.  This is one of those "long-haul" things - perhaps over years, and several major releases.  I'm not in a hurry to do the wrong thing quickly!!!


-----Original Message-----
From: Wolfgang Meier [mailto:wol...@ex...] 
Sent: Monday, August 30, 2010 10:38 AM
To: Jason Smith
Cc: Dmitriy Shabanov; eXist development; Paul Ryan; Michael J. Pelikan; Todd Gochenour
Subject: Re: [Exist-development] [Exist-open] Performance of concurrent read queries.

> I understand.  The solution we have in place right now is similar to the solution you
> mentioned, but we put it in place a while ago.  Augmenting the locking with a singleton
> lock does, indeed, work.

Internally, eXist does need the singleton lock in rare cases only,
mainly when reading or storing the collection configuration document
for a collection, or when locking documents for an XQuery update
expression.

Otherwise, eXist just avoids locking multiple collections at once
wherever possible as it is known to be an expensive operation and
limits concurrency.

> The second replacement I've come up with allows two read queries to run
> simultaneously, even when they target the same collection, and when multiple collections
> are used simultaneously.

As I said before, I welcome any exploration in this area. As James
just suggested, we may want to have a skype telecon on this to discuss
the possibilities and dangers.

Finally, just as a note to other users who code against the internal
API: Dannes' WebDAV reimplementation shows some clean examples of how
to use internals:

http://exist.svn.sourceforge.net/viewvc/exist/branches/dizzzz/trunk-webdav-upgrade/extensions/webdav/src/org/exist/webdav/

Wolfgang