Menu
▾
▴
Re: [Exist-open] warning message "object at -1:-1 not found" during backup caused by versioned collection
|
From: Alistair M. <ali...@go...> - 2010-07-28 17:10:39
|
Hi exist devs, I've spent a bit of time looking at the VersioningTrigger class, and have a fix that makes this problem go away - see attached patch. I know next to nothing about exist's internals, so I don't have a deep understanding of what I did, but I did narrow the problem down to the way that the base revision is stored in the versions collection. I changed the strategy for storing the base revision, and the problem went away. N.B. there also looks like there's another bug in the file, where sax start and end elements aren't matched. Cheers, Alistair On Tue, Jul 27, 2010 at 05:28:56PM +0100, Alistair Miles wrote: > Just to follow this up with a bit more information, the following is reported > in the log from the consistency check... > > COLLECTION: /db > COLLECTION: /db/test > COLLECTION: /db/system > COLLECTION: /db/system/versions > COLLECTION: /db/system/versions/db > COLLECTION: /db/system/versions/db/test > COLLECTION: /db/system/config > COLLECTION: /db/system/config/db > COLLECTION: /db/system/config/db/test > ---------------------------------------------- > ERR_DOM_INDEX: > Failed to access node 1 through dom.dbx index. Wrong storage address. Expected: -1; got: 12884901888 - > Document ID: 4 > DOCUMENT: 5 of 5 > > However, examining the backup output shows the base revision document > successfully exported. I haven't attempted to restore yet. > > I have also confirmed that if you manually remove the base revision from the > versions collection (i.e., remove /db/system/versions/db/test/foo.xml.base) > then trigger a backup, the error goes away, so it looks like it is something > to do with the way the base revision is being stored. > > This also connects with some other behaviour I am seeing to do with the > base revision, which is that xpath queries don't seem to work against the > base revision document when retrieved via the doc() function, unless you > put wildcards into the xpath. I'll attach a slightly modified version of > the original query I posted, demonstrating this xpath problem. > > So I'm wondering if is there something wrong with the way the base revision > document is being stored by the versioning module? > > Cheers > > Alistair > > On Thu, Jul 22, 2010 at 06:45:55PM +0100, Alistair Miles wrote: > > Hi all, > > > > In eXist 1.4 rev 10440, if I create a collection, then configure the > > collection with the versioning trigger, then store and update a document, > > then trigger a backup, I get the following warning message in the logs... > > > > 2010-07-22 18:23:08,248 [http-8080-1] WARN (SanityCheck.java [showTrace]:93) - Stacktrace: > > org.exist.util.sanity.AssertFailure: TRACE: object at -1:-1 not found. > > at org.exist.util.sanity.SanityCheck.TRACE(SanityCheck.java:72) > > at org.exist.storage.dom.DOMFile.get(DOMFile.java:1509) > > at org.exist.storage.dom.DOMFile.get(DOMFile.java:1501) > > at org.exist.backup.ConsistencyCheck$1.start(ConsistencyCheck.java:262) > > at org.exist.storage.dom.DOMTransaction.run(DOMTransaction.java:111) > > at org.exist.backup.ConsistencyCheck.checkXMLTree(ConsistencyCheck.java:209) > > at org.exist.backup.ConsistencyCheck$DocumentCallback.indexInfo(ConsistencyCheck.java:351) > > at org.exist.storage.btree.BTree$BTreeNode.query(BTree.java:1629) > > at org.exist.storage.btree.BTree$BTreeNode.access$300(BTree.java:736) > > at org.exist.storage.btree.BTree.query(BTree.java:313) > > at org.exist.storage.NativeBroker.getResourcesFailsafe(NativeBroker.java:1917) > > at org.exist.backup.ConsistencyCheck.checkDocuments(ConsistencyCheck.java:192) > > at org.exist.backup.ConsistencyCheck.checkAll(ConsistencyCheck.java:94) > > at org.exist.storage.ConsistencyCheckTask.execute(ConsistencyCheckTask.java:119) > > at org.exist.storage.SystemTaskManager.runSystemTask(SystemTaskManager.java:55) > > at org.exist.storage.SystemTaskManager.processTasks(SystemTaskManager.java:42) > > at org.exist.storage.txn.TransactionManager.processSystemTasks(TransactionManager.java:258) > > at org.exist.storage.SystemTaskManager.triggerSystemTask(SystemTaskManager.java:27) > > at org.exist.storage.txn.TransactionManager.triggerSystemTask(TransactionManager.java:248) > > at org.exist.storage.BrokerPool.triggerSystemTask(BrokerPool.java:1482) > > at org.exist.xquery.functions.system.TriggerSystemTask.eval(TriggerSystemTask.java:82) > > > > Is this something I should be worried about? > > > > I've attach a minimal xquery that should give you the same behaviour. > > > > It looks like the backup is still created, although I haven't verified that > > the backup can be correctly restored (I'll get on to that...). > > > > Cheers > > > > Alistair > > -- > > Alistair Miles > > Head of Epidemiological Informatics > > Centre for Genomics and Global Health <http://cggh.org> > > The Wellcome Trust Centre for Human Genetics > > Roosevelt Drive > > Oxford > > OX3 7BN > > United Kingdom > > Web: http://purl.org/net/aliman > > Email: ali...@gm... > > Tel: +44 (0)1865 287669 > > > xquery version "1.0"; > > > > > > declare function local:page() as item()* > > { > > let $set-status-code := response:set-status-code( 200 ) > > let $set-content-type := response:set-header( "Content-Type" , "text/html" ) > > return > > <html xmlns="http://www.w3.org/1999/xhtml"> > > <head><title>Spike Backup Bug</title></head> > > <body> > > <p> > > This script isolates code required to generate a warning message during backup. > > </p> > > <p> > > To demonstrate the bug, click on the button below, then examine the logs. > > </p> > > <p> > > <form action="" method="post"> > > <input type="submit" value="Go"/> > > </form> > > </p> > > </body> > > </html> > > > > }; > > > > > > > > declare function local:isolate() as item()* > > { > > let $collection-path := "/db/test" > > let $config-collection-path := concat( "/db/system/config" , "/db/test" ) > > let $versions-collection-path := concat( "/db/system/versions" , "/db/test" ) > > > > let $clean := > > for $p in ( $collection-path , $config-collection-path , $versions-collection-path ) > > return > > if (xmldb:collection-available( $p )) then xmldb:remove( $p ) else () > > > > let $collection-config := > > > > <collection xmlns="http://exist-db.org/collection-config/1.0"> > > <triggers> > > <trigger event="store,remove,update" class="org.exist.versioning.VersioningTrigger"> > > <parameter name="overwrite" value="yes"/> > > </trigger> > > </triggers> > > </collection> > > > > let $base-config-collection-created := xmldb:create-collection( "/db/system/config" , "db" ) > > let $config-collection-created := xmldb:create-collection( "/db/system/config/db" , "test" ) > > let $config-stored := xmldb:store( $config-collection-path , "collection.xconf" , $collection-config , "application/xml" ) > > > > let $collection-created := xmldb:create-collection( "/db" , "test" ) > > > > let $doc := > > <foo><bar>baz</bar></foo> > > > > let $stored := xmldb:store( "/db/test" , "foo.xml" , $doc ) > > > > let $doc2 := > > <foo><bar>spong</bar></foo> > > > > let $udpated := xmldb:store( "/db/test" , "foo.xml" , $doc2 ) > > > > let $params := > > <parameters> > > <param name="output" value="export"/> > > <param name="backup" value="yes"/> > > <param name="incremental" value="yes"/> > > </parameters> > > let $backup := system:trigger-system-task("org.exist.storage.ConsistencyCheckTask", $params) > > > > return () > > > > }; > > > > > > declare function local:do-service() as item()* > > { > > > > if (request:get-method() = "GET") > > then local:page() > > > > else if (request:get-method() = "POST") > > then > > let $isolate := local:isolate() > > return local:page() > > > > else () > > > > }; > > > > > > > > > > let $login := xmldb:login( "/" , "admin" , "" ) > > > > return local:do-service() > > > > > -- > Alistair Miles > Head of Epidemiological Informatics > Centre for Genomics and Global Health <http://cggh.org> > The Wellcome Trust Centre for Human Genetics > Roosevelt Drive > Oxford > OX3 7BN > United Kingdom > Web: http://purl.org/net/aliman > Email: ali...@gm... > Tel: +44 (0)1865 287669 > xquery version "1.0"; > > > declare function local:page() as item()* > { > let $set-status-code := response:set-status-code( 200 ) > let $set-content-type := response:set-header( "Content-Type" , "text/html" ) > return > <html xmlns="http://www.w3.org/1999/xhtml"> > <head><title>Spike Backup Bug</title></head> > <body> > <p> > This script isolates code required to generate a warning message during backup. > </p> > <p> > To demonstrate the bug, click on the button below, then examine the logs. > </p> > <p> > <form action="" method="post"> > <input type="submit" value="Go"/> > </form> > </p> > </body> > </html> > > }; > > > > declare function local:isolate() as item()* > { > let $collection-path := "/db/test" > let $config-collection-path := concat( "/db/system/config" , "/db/test" ) > let $versions-collection-path := concat( "/db/system/versions" , "/db/test" ) > > let $clean := > for $p in ( $collection-path , $config-collection-path , $versions-collection-path ) > return > if (xmldb:collection-available( $p )) then xmldb:remove( $p ) else () > > let $collection-config := > > <collection xmlns="http://exist-db.org/collection-config/1.0"> > <triggers> > <trigger event="store,remove,update" class="org.exist.versioning.VersioningTrigger"> > <parameter name="overwrite" value="yes"/> > </trigger> > </triggers> > </collection> > > let $base-config-collection-created := xmldb:create-collection( "/db/system/config" , "db" ) > let $config-collection-created := xmldb:create-collection( "/db/system/config/db" , "test" ) > let $config-stored := xmldb:store( $config-collection-path , "collection.xconf" , $collection-config , "application/xml" ) > > let $collection-created := xmldb:create-collection( "/db" , "test" ) > > let $doc := > <foo><bar>baz</bar></foo> > > let $stored := xmldb:store( "/db/test" , "foo.xml" , $doc ) > > let $doc2 := > <foo><bar>spong</bar></foo> > > let $udpated := xmldb:store( "/db/test" , "foo.xml" , $doc2 ) > > let $params := > <parameters> > <param name="output" value="export"/> > <param name="backup" value="yes"/> > <param name="incremental" value="yes"/> > </parameters> > let $backup := system:trigger-system-task("org.exist.storage.ConsistencyCheckTask", $params) > > (: expose another problem - xpath queries don't work against the base revision unless you use wildcards :) > > return doc( "/db/system/versions/db/test/foo.xml.base" )/* (: try -- doc( "/db/system/versions/db/test/foo.xml.base" )/foo -- instead :) > > }; > > > declare function local:do-service() as item()* > { > > if (request:get-method() = "GET") > then local:page() > > else if (request:get-method() = "POST") > then > let $isolate := local:isolate() > return $isolate > > else () > > }; > > > > > let $login := xmldb:login( "/" , "admin" , "" ) > > return local:do-service() > -- Alistair Miles Head of Epidemiological Informatics Centre for Genomics and Global Health <http://cggh.org> The Wellcome Trust Centre for Human Genetics Roosevelt Drive Oxford OX3 7BN United Kingdom Web: http://purl.org/net/aliman Email: ali...@gm... Tel: +44 (0)1865 287669 |