[Exist-development] Errors from Stress tests on xquery update

[Casey J. <cas...@jo...>]

Hi all,

Just added a stress test for trunk which performs lot's of xquery updates
across 3 large documents in the same collection. The goal was to simulate
collisions between heavy update operations. When only "update insert" was
used the database seemed to preform fine, when "update delete" was added
the database threw many errors and would not restart (See output samples at
end of email)

The test simulates 30 simultaneous users each doing 1000 runs through the
REST api.

There are 3 operations that each run does: (insert attribute, insert
element, delete element). The testing data contains 3 1MB xml files with
lots of <scm:edge> nodes.

The query's look like this:

----- Insert attributes -------
*let $all-nodes :=
collection('/db/xupdate-test-collection')//*[local-name() = 'edge']
let $sub-nodes := subsequence($all-nodes, 0, 50)
*
*return
*
*for $node in $sub-nodes return update insert attrbute foo {'bar'} into
$node*

----- Insert elements -------
*let $all-nodes :=
collection('/db/xupdate-test-collection')//*[local-name() = 'edge']
let $sub-nodes := subsequence($all-nodes, 0, 50)
*
*return
*
*for $node in $sub-nodes return update insert element {'bar'}{} into $node*

----- Delete elements -------
*let $all-nodes :=
collection('/db/xupdate-test-collection')//*[local-name() = 'edge']
let $sub-nodes := subsequence($all-nodes, 0, 50)
*
*return
*
*for $node in $sub-nodes return update delete $node*

When utilizing all three, the logs filled with NPE's like these:

*17 Jul 2012 17:40:00,193 [eXistThread-52] WARN  (ServletHandler.java
[doHandle]:553) - /exist/rest/db/
java.lang.NullPointerException
    at
org.exist.xquery.update.Modification.selectAndLock(Modification.java:157)
    at org.exist.xquery.update.Delete.eval(Delete.java:123)
    at
org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:56)
    at
org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:63)
    at org.exist.xquery.ForExpr.eval(ForExpr.java:303)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156)
    at
org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:56)
    at
org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:63)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:200)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:198)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:268)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.XQuery.execute(XQuery.java:246)
 .....
17 Jul 2012 17:40:00,193 [eXistThread-52] WARN  (ServletHandler.java
[doHandle]:553) - /exist/rest/db/
java.lang.NullPointerException
    at
org.exist.xquery.update.Modification.selectAndLock(Modification.java:157)
    at org.exist.xquery.update.Insert.eval(Insert.java:139)
    at
org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:56)
    at
org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:63)
    at org.exist.xquery.ForExpr.eval(ForExpr.java:303)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156)
    at
org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:56)
.....
*

After shutting down the database and running a consistency check the
following output was observed:

*Running consistency check

File lock last access timestamp: Jul 17, 2012
[path-omitted]/eXist/../data/dbx_dir.lck
Found a stale lockfile. Trying to remove it:
[path-omitted]/eXist/../data/dbx_dir.lck
File lock last access timestamp: Jul 17, 2012
[path-omitted]/eXist/../data/journal.lck
Found a stale lockfile. Trying to remove it:
[path-omitted]/eXist/../data/journal.lck
ERROR: Failed to open database: java.lang.StackOverflowError*

I would like to understand what is causing this better so we can ither
improve the handling in eXist, or offer a workaround to developers which
will be using xquery updates in high load situations.

Thanks,

Casey

-- 
--
Casey Jordan
easyDITA a product of Jorsek LLC
"CaseyDJordan" on LinkedIn, Twitter & Facebook
(585) 348 7399
easydita.com


This message is intended only for the use of the Addressee(s) and may
contain information that is privileged, confidential, and/or exempt from
disclosure under applicable law.  If you are not the intended recipient,
please be advised that any disclosure  copying, distribution, or use of
the information contained herein is prohibited.  If you have received
this communication in error, please destroy all copies of the message,
whether in electronic or hard copy format, as well as attachments, and
immediately contact the sender by replying to this e-mail or by phone.
Thank you.