Re: [Exist-development] [Exist-open] Performance of concurrent read queries.

[Adam R. <ad...@ex...>]

> Again, I would not expect a final solution to be implemented the way I have done it.  I just want to be able to show some possibilities.

I am really excited to see such enthusiasm and knowledge of the
eXist-db internals and suggestions for improvements that could be
made.

> I am not expecting this conversation to take place quickly.  This is one of those "long-haul" things - perhaps over years, and several major releases.  I'm not in a hurry to do the wrong thing quickly!!!
>

Lets not loose this though, I think that any performance and
scalability enhancements are great.

And actually there is a flip side here that no one has mentioned:
Whilst everyone has said that you should have optimised queries and
appropriate indexes, which is invariably true for production systems,
what about new users and developers who come to eXist-db?
As a new user of eXist-db you dont necessarily understand about
optimising your queries or even how to create the correct indexes. Not
all eXist-db users are software developers! It would be great if these
new users saw great performance from the start, even if they havent
set up indexes and their queries are doing full scans of the dbx files
:-)


> -----Original Message-----
> From: Wolfgang Meier [mailto:wol...@ex...]
> Sent: Monday, August 30, 2010 10:38 AM
> To: Jason Smith
> Cc: Dmitriy Shabanov; eXist development; Paul Ryan; Michael J. Pelikan; Todd Gochenour
> Subject: Re: [Exist-development] [Exist-open] Performance of concurrent read queries.
>
>> I understand.  The solution we have in place right now is similar to the solution you
>> mentioned, but we put it in place a while ago.  Augmenting the locking with a singleton
>> lock does, indeed, work.
>
> Internally, eXist does need the singleton lock in rare cases only,
> mainly when reading or storing the collection configuration document
> for a collection, or when locking documents for an XQuery update
> expression.
>
> Otherwise, eXist just avoids locking multiple collections at once
> wherever possible as it is known to be an expensive operation and
> limits concurrency.
>
>> The second replacement I've come up with allows two read queries to run
>> simultaneously, even when they target the same collection, and when multiple collections
>> are used simultaneously.
>
> As I said before, I welcome any exploration in this area. As James
> just suggested, we may want to have a skype telecon on this to discuss
> the possibilities and dangers.
>
> Finally, just as a note to other users who code against the internal
> API: Dannes' WebDAV reimplementation shows some clean examples of how
> to use internals:
>
> http://exist.svn.sourceforge.net/viewvc/exist/branches/dizzzz/trunk-webdav-upgrade/extensions/webdav/src/org/exist/webdav/
>
> Wolfgang
> ------------------------------------------------------------------------------
> This SF.net Dev2Dev email is sponsored by:
>
> Show off your parallel programming skills.
> Enter the Intel(R) Threading Challenge 2010.
> http://p.sf.net/sfu/intel-thread-sfd
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb