Menu
▾
▴
exist-development — eXist Developer's List
You can subscribe to this list here.
| 2009 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(22) |
Nov
(85) |
Dec
(20) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2010 |
Jan
(47) |
Feb
(127) |
Mar
(268) |
Apr
(78) |
May
(47) |
Jun
(38) |
Jul
(131) |
Aug
(221) |
Sep
(187) |
Oct
(54) |
Nov
(111) |
Dec
(84) |
| 2011 |
Jan
(152) |
Feb
(106) |
Mar
(94) |
Apr
(90) |
May
(53) |
Jun
(20) |
Jul
(24) |
Aug
(37) |
Sep
(32) |
Oct
(70) |
Nov
(22) |
Dec
(15) |
| 2012 |
Jan
(33) |
Feb
(110) |
Mar
(24) |
Apr
(1) |
May
(11) |
Jun
(8) |
Jul
(12) |
Aug
(37) |
Sep
(39) |
Oct
(81) |
Nov
(38) |
Dec
(50) |
| 2013 |
Jan
(23) |
Feb
(53) |
Mar
(23) |
Apr
(5) |
May
(19) |
Jun
(16) |
Jul
(16) |
Aug
(9) |
Sep
(21) |
Oct
(1) |
Nov
(2) |
Dec
(8) |
| 2014 |
Jan
(16) |
Feb
(6) |
Mar
(27) |
Apr
(1) |
May
(10) |
Jun
(1) |
Jul
(4) |
Aug
(10) |
Sep
(19) |
Oct
(22) |
Nov
(4) |
Dec
(6) |
| 2015 |
Jan
(3) |
Feb
(6) |
Mar
(9) |
Apr
|
May
(11) |
Jun
(23) |
Jul
(14) |
Aug
(10) |
Sep
(10) |
Oct
(9) |
Nov
(18) |
Dec
(4) |
| 2016 |
Jan
(5) |
Feb
(5) |
Mar
|
Apr
(2) |
May
(15) |
Jun
(2) |
Jul
(8) |
Aug
(2) |
Sep
(6) |
Oct
|
Nov
|
Dec
|
| 2017 |
Jan
(2) |
Feb
(12) |
Mar
(22) |
Apr
(6) |
May
|
Jun
|
Jul
(1) |
Aug
(1) |
Sep
(5) |
Oct
(2) |
Nov
|
Dec
|
| 2018 |
Jan
(2) |
Feb
|
Mar
|
Apr
|
May
|
Jun
(5) |
Jul
(3) |
Aug
|
Sep
(7) |
Oct
(19) |
Nov
|
Dec
|
| 2021 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(1) |
Nov
|
Dec
|
| 2022 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(3) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Patrick B. <pat...@jo...> - 2011-11-02 22:53:22
|
Sure, will do
On Wed, Nov 2, 2011 at 6:09 PM, Adam Retter <ad...@ex...> wrote:
> Looks good to me, please by all means commit... perhaps in both 1.4.x and
> trunk?
>
> On 1 November 2011 22:39, Patrick Bosek <pat...@jo...> wrote:
> > Hi everyone,
> >
> > I've found, and fixed, a bug in the xmlrpc.RpcConnection.java. I was
> going
> > to simply commit it, but I thought it might be best to run it past the
> > developers first since I'm only 99% sure of what I've done.
> >
> > The remove(..) function on like 2615 catches any Throwable on line 2643,
> but
> > it doesn't abort it's transaction. So, if there is an exception (like the
> > target file being locked), the transaction never gets committed and never
> > gets aborted, which is bad (obviously).
> >
> > What I did to fix this was remove the aborts that sat above the
> exceptions
> > the function threw it's self and added one into the catch(Throwable)
> >
> > See here:
> >
> >
> > private boolean remove(XmldbURI docUri) throws EXistException,
> > PermissionDeniedException {
> > TransactionManager transact =
> > factory.getBrokerPool().getTransactionManager();
> > Txn transaction = transact.beginTransaction();
> > DBBroker broker = null;
> > Collection collection = null;
> > try {
> > broker = factory.getBrokerPool().get(user);
> > collection =
> broker.openCollection(docUri.removeLastSegment(),
> > Lock.WRITE_LOCK);
> > if (collection == null) {
> > --------- START REMOVED -------------
> > // transact.abort(transaction);
> > --------- START REMOVED -------------
> > throw new EXistException("Collection " +
> > docUri.removeLastSegment() + " not found");
> > }
> > // keep the write lock in the transaction
> > transaction.registerLock(collection.getLock(),
> Lock.WRITE_LOCK);
> >
> > DocumentImpl doc = collection.getDocument(broker,
> > docUri.lastSegment());
> > if (doc == null) {
> > --------- START REMOVED -------------
> > // transact.abort(transaction);
> > --------- END REMOVED -------------
> > throw new EXistException("Document " + docUri + " not
> > found");
> > }
> >
> > if(doc.getResourceType() == DocumentImpl.BINARY_FILE)
> > collection.removeBinaryResource(transaction, broker,
> doc);
> > else
> > collection.removeXMLResource(transaction, broker,
> > docUri.lastSegment());
> > transact.commit(transaction);
> > return true;
> >
> > } catch (Throwable e) {
> > --------- START ADDED -------------
> > transact.abort(transaction);
> > --------- END ADDED -------------
> > handleException(e);
> > return false;
> >
> > } finally {
> > factory.getBrokerPool().release(broker);
> > }
> > }
> >
> >
> >
> > My only question about this is what happens in the case there is an
> > exception thrown from the transaction.commit(..)? From looking at the
> > function it wasn't immediately obvious to me if this was going to create
> an
> > issue.
> >
> >
> > Thoughts?
> >
> >
> > --
> > Patrick Bosek
> > Jorsek Software
> > Cell (585) 820 9634
> > Office (877) 492 2960
> > Jorsek.com
> >
> >
> >
> ------------------------------------------------------------------------------
> > RSA® Conference 2012
> > Save $700 by Nov 18
> > Register now!
> > http://p.sf.net/sfu/rsa-sfdev2dev1
> > _______________________________________________
> > Exist-development mailing list
> > Exi...@li...
> > https://lists.sourceforge.net/lists/listinfo/exist-development
> >
> >
>
>
>
> --
> Adam Retter
>
> eXist Developer
> { United Kingdom }
> ad...@ex...
> irc://irc.freenode.net/existdb
>
--
Patrick Bosek
Jorsek Software
Cell (585) 820 9634
Office (877) 492 2960
Jorsek.com
|
|
From: Adam R. <ad...@ex...> - 2011-11-02 22:10:02
|
Looks good to me, please by all means commit... perhaps in both 1.4.x and trunk?
On 1 November 2011 22:39, Patrick Bosek <pat...@jo...> wrote:
> Hi everyone,
>
> I've found, and fixed, a bug in the xmlrpc.RpcConnection.java. I was going
> to simply commit it, but I thought it might be best to run it past the
> developers first since I'm only 99% sure of what I've done.
>
> The remove(..) function on like 2615 catches any Throwable on line 2643, but
> it doesn't abort it's transaction. So, if there is an exception (like the
> target file being locked), the transaction never gets committed and never
> gets aborted, which is bad (obviously).
>
> What I did to fix this was remove the aborts that sat above the exceptions
> the function threw it's self and added one into the catch(Throwable)
>
> See here:
>
>
> private boolean remove(XmldbURI docUri) throws EXistException,
> PermissionDeniedException {
> TransactionManager transact =
> factory.getBrokerPool().getTransactionManager();
> Txn transaction = transact.beginTransaction();
> DBBroker broker = null;
> Collection collection = null;
> try {
> broker = factory.getBrokerPool().get(user);
> collection = broker.openCollection(docUri.removeLastSegment(),
> Lock.WRITE_LOCK);
> if (collection == null) {
> --------- START REMOVED -------------
> // transact.abort(transaction);
> --------- START REMOVED -------------
> throw new EXistException("Collection " +
> docUri.removeLastSegment() + " not found");
> }
> // keep the write lock in the transaction
> transaction.registerLock(collection.getLock(), Lock.WRITE_LOCK);
>
> DocumentImpl doc = collection.getDocument(broker,
> docUri.lastSegment());
> if (doc == null) {
> --------- START REMOVED -------------
> // transact.abort(transaction);
> --------- END REMOVED -------------
> throw new EXistException("Document " + docUri + " not
> found");
> }
>
> if(doc.getResourceType() == DocumentImpl.BINARY_FILE)
> collection.removeBinaryResource(transaction, broker, doc);
> else
> collection.removeXMLResource(transaction, broker,
> docUri.lastSegment());
> transact.commit(transaction);
> return true;
>
> } catch (Throwable e) {
> --------- START ADDED -------------
> transact.abort(transaction);
> --------- END ADDED -------------
> handleException(e);
> return false;
>
> } finally {
> factory.getBrokerPool().release(broker);
> }
> }
>
>
>
> My only question about this is what happens in the case there is an
> exception thrown from the transaction.commit(..)? From looking at the
> function it wasn't immediately obvious to me if this was going to create an
> issue.
>
>
> Thoughts?
>
>
> --
> Patrick Bosek
> Jorsek Software
> Cell (585) 820 9634
> Office (877) 492 2960
> Jorsek.com
>
>
> ------------------------------------------------------------------------------
> RSA® Conference 2012
> Save $700 by Nov 18
> Register now!
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>
>
--
Adam Retter
eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb
|
|
From: Adam R. <ad...@ex...> - 2011-11-02 22:02:33
|
Okay I see the problem, I think that maybe WebDAV is not correctly URI encoding the filenames when they are stored in eXist-db. Simply if I upload and store a file with the name - ".~lock.Untitled.odt#" into eXist-db using the Java admin client or REST API, then this is fine, and I see this stored in WEB-INF/data/fs/db as the encoded filename - ".%7Elock.Untitled.odt%23" However when I upload a similar file with WebDAV, then I see that the file stored in WEB-INF/data/fs/db has this filename - ".~lock.Untitled.odt" - so somehow WebDAV is not doing the encoding at store time into eXist-db as the # is completely lost somehow, and the ~ has not been URI encoded. When I then try and examine the collection or file in eXist-db I get the same error as you Michael. Perhaps Dannes has some ideas on this one? On 2 November 2011 22:55, Adam Retter <ad...@ex...> wrote: > Im trying to reproduce this here, I have created similar files in my > DB by just uploading them, but I dont see any issue yet, so - > > 1) It maybe a problem because I havent used WebDAV to do this, what > are you using as the WebDav Client? > > 2) What is the XQuery that you are invoking, which is causing this > issue to appear? > > Thanks Adam. > > On 30 October 2011 05:44, Michael Westbay > <wes...@ja...> wrote: >> I had edited the file mlb-po-overall-2011-10-17.ods with LibraOffice >> within the database via WebDAV. This generated a file >> .~lock.mlb-po-overall-2011-10-17.ods#. I then use eXist to extract >> the XML content and process/populate data in the database. At some >> unknown time, well after I thought I had closed this file, I started >> getting the following error: >> >> Invalid URI: xmldb URI scheme has no instance name: >> xmldb:.~lock.mlb-po-overall-2011-10-17.ods# >> >> This error has since caused any WebDAV or admin client (web based and >> application based) to fail with a permission not allowed error to the >> collection *above* where this file resides. XPath/XQuery queries to >> the data in adjacent collections work. The stacktrace that I get >> trying to access the parent collection with the web based admin tool >> follows: >> >> 2011-10-30 11:36:45,611 [DefaultQuartzScheduler_Worker-4] DEBUG >> (NGramIndex.java [sync]:91) - SYNC NGRAM >> 2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO >> (NativeBroker.java [sync]:3141) - Memory: 497,352K total; 748,352K >> max; 31,171K free >> 2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO >> (BTree.java [printStatistics]:2420) - dom.dbx INDEX Buffers occupation >> : 100% (1093 out of 1093) Cache efficiency : 100% >> 2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO >> (DOMFile.java [printStatistics]:1441) - dom.dbx DATA Buffers >> occupation : 100% (255 out of 256) Cache efficiency : 94% >> 2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO >> (BTree.java [printStatistics]:2420) - collections.dbx INDEX Buffers >> occupation : 84% (54 out of 64) Cache efficiency : 100% >> 2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO >> (BFile.java [printStatistics]:434) - collections.dbx DATA Buffers >> occupation : 99% (99 out of 100) Cache efficiency : 99% >> 2011-10-30 11:36:45,613 [DefaultQuartzScheduler_Worker-4] INFO >> (BTree.java [printStatistics]:2420) - values.dbx INDEX Buffers >> occupation : 100% (144 out of 144) Cache efficiency : 100% >> 2011-10-30 11:36:45,613 [DefaultQuartzScheduler_Worker-4] INFO >> (BFile.java [printStatistics]:434) - values.dbx DATA Buffers >> occupation : 100% (1796 out of 1797) Cache efficiency : 99% >> 2011-10-30 11:36:45,613 [DefaultQuartzScheduler_Worker-4] DEBUG >> (Journal.java [checkpoint]:328) - Checkpoint reached >> 2011-10-30 11:36:45,613 [DefaultQuartzScheduler_Worker-4] DEBUG >> (NotificationService.java [debug]:94) - Registered UpdateListeners: >> 2011-10-30 11:37:42,704 [eXistThread-49] DEBUG >> (HttpRequestWrapper.java [<init>]:129) - Retrieved 2 parameters. >> 2011-10-30 11:37:42,706 [eXistThread-49] DEBUG (XQuery.java >> [execute]:249) - Execution took 1 ms >> 2011-10-30 11:37:42,706 [eXistThread-49] DEBUG (HTTPUtils.java >> [addLastModifiedHeader]:61) - mostRecentDocumentTime: 0 >> 2011-10-30 11:37:42,707 [eXistThread-49] DEBUG >> (HttpRequestWrapper.java [<init>]:129) - Retrieved 2 parameters. >> 2011-10-30 11:37:42,714 [eXistThread-49] ERROR (XQueryServlet.java >> [process]:573) - Invalid URI: xmldb URI scheme has no instance name: >> xmldb:.~lock.mlb-po-overall-2011-10-17.ods# >> java.lang.IllegalArgumentException: Invalid URI: xmldb URI scheme has >> no instance name: xmldb:.~lock.mlb-po-overall-2011-10-17.ods# >> at org.exist.xmldb.XmldbURI.create(XmldbURI.java:181) >> at org.exist.dom.BinaryDocument.read(BinaryDocument.java:105) >> at org.exist.storage.NativeBroker$DocumentCallback.indexInfo(NativeBroker.java:3439) >> at org.exist.storage.btree.BTree$BTreeNode.query(BTree.java:1689) >> at org.exist.storage.btree.BTree$BTreeNode.query(BTree.java:1620) >> at org.exist.storage.btree.BTree$BTreeNode.access$3(BTree.java:1591) >> at org.exist.storage.btree.BTree.query(BTree.java:327) >> at org.exist.storage.NativeBroker.getCollectionResources(NativeBroker.java:1872) >> at org.exist.collections.Collection.read(Collection.java:731) >> at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:755) >> at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:724) >> at org.exist.storage.NativeBroker.getCollection(NativeBroker.java:719) >> at org.exist.xmldb.LocalCollection.getCollection(LocalCollection.java:213) >> at org.exist.xmldb.LocalCollection.<init>(LocalCollection.java:154) >> at org.exist.xmldb.LocalCollection.<init>(LocalCollection.java:123) >> at org.exist.xquery.functions.xmldb.XMLDBAbstractCollectionManipulator.createLocalCollection(XMLDBAbstractCollectionManipulator.java:77) >> at org.exist.xquery.functions.xmldb.XMLDBAbstractCollectionManipulator.eval(XMLDBAbstractCollectionManipulator.java:129) >> at org.exist.xquery.BasicFunction.eval(BasicFunction.java:68) >> at org.exist.xquery.InternalFunctionCall.eval(InternalFunctionCall.java:55) >> at org.exist.xquery.LetExpr.eval(LetExpr.java:155) >> at org.exist.xquery.LetExpr.eval(LetExpr.java:208) >> at org.exist.xquery.ForExpr.eval(ForExpr.java:317) >> at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156) >> at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:135) >> at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61) >> at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:278) >> at org.exist.xquery.FunctionCall.eval(FunctionCall.java:205) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.PathExpr.eval(PathExpr.java:243) >> at org.exist.xquery.SequenceConstructor.eval(SequenceConstructor.java:83) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.PathExpr.eval(PathExpr.java:243) >> at org.exist.xquery.EnclosedExpr.eval(EnclosedExpr.java:70) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.PathExpr.eval(PathExpr.java:243) >> at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279) >> at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:135) >> at org.exist.xquery.DynamicCardinalityCheck.eval(DynamicCardinalityCheck.java:72) >> at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61) >> at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:278) >> at org.exist.xquery.FunctionCall.eval(FunctionCall.java:205) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.PathExpr.eval(PathExpr.java:243) >> at org.exist.xquery.EnclosedExpr.eval(EnclosedExpr.java:70) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.PathExpr.eval(PathExpr.java:243) >> at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.PathExpr.eval(PathExpr.java:243) >> at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279) >> at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:56) >> at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:63) >> at org.exist.xquery.LetExpr.eval(LetExpr.java:210) >> at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156) >> at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:135) >> at org.exist.xquery.DynamicCardinalityCheck.eval(DynamicCardinalityCheck.java:72) >> at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61) >> at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:278) >> at org.exist.xquery.FunctionCall.eval(FunctionCall.java:205) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.PathExpr.eval(PathExpr.java:243) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.SwitchExpression.eval(SwitchExpression.java:110) >> at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:56) >> at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:63) >> at org.exist.xquery.LetExpr.eval(LetExpr.java:210) >> at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156) >> at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:135) >> at org.exist.xquery.DynamicCardinalityCheck.eval(DynamicCardinalityCheck.java:72) >> at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61) >> at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:278) >> at org.exist.xquery.FunctionCall.eval(FunctionCall.java:205) >> at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:100) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.PathExpr.eval(PathExpr.java:243) >> at org.exist.xquery.EnclosedExpr.eval(EnclosedExpr.java:70) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.PathExpr.eval(PathExpr.java:243) >> at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.PathExpr.eval(PathExpr.java:243) >> at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.PathExpr.eval(PathExpr.java:243) >> at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.PathExpr.eval(PathExpr.java:243) >> at org.exist.xquery.SequenceConstructor.eval(SequenceConstructor.java:83) >> at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:56) >> at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:63) >> at org.exist.xquery.LetExpr.eval(LetExpr.java:210) >> at org.exist.xquery.LetExpr.eval(LetExpr.java:208) >> at org.exist.xquery.LetExpr.eval(LetExpr.java:208) >> at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.PathExpr.eval(PathExpr.java:243) >> at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) >> at org.exist.xquery.XQuery.execute(XQuery.java:246) >> at org.exist.xquery.XQuery.execute(XQuery.java:201) >> at org.exist.http.servlets.XQueryServlet.process(XQueryServlet.java:512) >> at org.exist.http.servlets.XQueryServlet.doGet(XQueryServlet.java:217) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) >> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:538) >> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:478) >> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) >> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:517) >> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225) >> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937) >> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406) >> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183) >> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871) >> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) >> at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:239) >> at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:115) >> at org.exist.http.urlrewrite.Forward.doRewrite(Forward.java:47) >> at org.exist.http.urlrewrite.XQueryURLRewrite.doRewrite(XQueryURLRewrite.java:525) >> at org.exist.http.urlrewrite.XQueryURLRewrite.doFilter(XQueryURLRewrite.java:344) >> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1323) >> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:476) >> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) >> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:480) >> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225) >> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937) >> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406) >> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183) >> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871) >> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) >> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) >> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110) >> at org.eclipse.jetty.server.Server.handle(Server.java:346) >> at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:589) >> at org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048) >> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:601) >> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:214) >> at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:411) >> at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:531) >> at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40) >> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529) >> at java.lang.Thread.run(Thread.java:636) >> >> I have tried to restore the problem directory from a backup, but I get >> similar exceptions as well. >> >> I have shut down the server, removed the .~lock file from the >> $EXIST_HOME/webapp/WEB-INF/data/fs structure, then restarted, also >> without a change. >> >> Reindexing also hasn't worked. >> >> Do you have any recommendation of other things I can try? >> >> -- >> Michael Westbay >> Writer/System Administrator >> http://www.japanesebaseball.com/ >> >> ------------------------------------------------------------------------------ >> Get your Android app more play: Bring it to the BlackBerry PlayBook >> in minutes. BlackBerry App World™ now supports Android™ Apps >> for the BlackBerry® PlayBook™. Discover just how easy and simple >> it is! http://p.sf.net/sfu/android-dev2dev >> _______________________________________________ >> Exist-development mailing list >> Exi...@li... >> https://lists.sourceforge.net/lists/listinfo/exist-development >> > > > > -- > Adam Retter > > eXist Developer > { United Kingdom } > ad...@ex... > irc://irc.freenode.net/existdb > -- Adam Retter eXist Developer { United Kingdom } ad...@ex... irc://irc.freenode.net/existdb |
|
From: Adam R. <ad...@ex...> - 2011-11-02 21:55:30
|
Im trying to reproduce this here, I have created similar files in my DB by just uploading them, but I dont see any issue yet, so - 1) It maybe a problem because I havent used WebDAV to do this, what are you using as the WebDav Client? 2) What is the XQuery that you are invoking, which is causing this issue to appear? Thanks Adam. On 30 October 2011 05:44, Michael Westbay <wes...@ja...> wrote: > I had edited the file mlb-po-overall-2011-10-17.ods with LibraOffice > within the database via WebDAV. This generated a file > .~lock.mlb-po-overall-2011-10-17.ods#. I then use eXist to extract > the XML content and process/populate data in the database. At some > unknown time, well after I thought I had closed this file, I started > getting the following error: > > Invalid URI: xmldb URI scheme has no instance name: > xmldb:.~lock.mlb-po-overall-2011-10-17.ods# > > This error has since caused any WebDAV or admin client (web based and > application based) to fail with a permission not allowed error to the > collection *above* where this file resides. XPath/XQuery queries to > the data in adjacent collections work. The stacktrace that I get > trying to access the parent collection with the web based admin tool > follows: > > 2011-10-30 11:36:45,611 [DefaultQuartzScheduler_Worker-4] DEBUG > (NGramIndex.java [sync]:91) - SYNC NGRAM > 2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO > (NativeBroker.java [sync]:3141) - Memory: 497,352K total; 748,352K > max; 31,171K free > 2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO > (BTree.java [printStatistics]:2420) - dom.dbx INDEX Buffers occupation > : 100% (1093 out of 1093) Cache efficiency : 100% > 2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO > (DOMFile.java [printStatistics]:1441) - dom.dbx DATA Buffers > occupation : 100% (255 out of 256) Cache efficiency : 94% > 2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO > (BTree.java [printStatistics]:2420) - collections.dbx INDEX Buffers > occupation : 84% (54 out of 64) Cache efficiency : 100% > 2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO > (BFile.java [printStatistics]:434) - collections.dbx DATA Buffers > occupation : 99% (99 out of 100) Cache efficiency : 99% > 2011-10-30 11:36:45,613 [DefaultQuartzScheduler_Worker-4] INFO > (BTree.java [printStatistics]:2420) - values.dbx INDEX Buffers > occupation : 100% (144 out of 144) Cache efficiency : 100% > 2011-10-30 11:36:45,613 [DefaultQuartzScheduler_Worker-4] INFO > (BFile.java [printStatistics]:434) - values.dbx DATA Buffers > occupation : 100% (1796 out of 1797) Cache efficiency : 99% > 2011-10-30 11:36:45,613 [DefaultQuartzScheduler_Worker-4] DEBUG > (Journal.java [checkpoint]:328) - Checkpoint reached > 2011-10-30 11:36:45,613 [DefaultQuartzScheduler_Worker-4] DEBUG > (NotificationService.java [debug]:94) - Registered UpdateListeners: > 2011-10-30 11:37:42,704 [eXistThread-49] DEBUG > (HttpRequestWrapper.java [<init>]:129) - Retrieved 2 parameters. > 2011-10-30 11:37:42,706 [eXistThread-49] DEBUG (XQuery.java > [execute]:249) - Execution took 1 ms > 2011-10-30 11:37:42,706 [eXistThread-49] DEBUG (HTTPUtils.java > [addLastModifiedHeader]:61) - mostRecentDocumentTime: 0 > 2011-10-30 11:37:42,707 [eXistThread-49] DEBUG > (HttpRequestWrapper.java [<init>]:129) - Retrieved 2 parameters. > 2011-10-30 11:37:42,714 [eXistThread-49] ERROR (XQueryServlet.java > [process]:573) - Invalid URI: xmldb URI scheme has no instance name: > xmldb:.~lock.mlb-po-overall-2011-10-17.ods# > java.lang.IllegalArgumentException: Invalid URI: xmldb URI scheme has > no instance name: xmldb:.~lock.mlb-po-overall-2011-10-17.ods# > at org.exist.xmldb.XmldbURI.create(XmldbURI.java:181) > at org.exist.dom.BinaryDocument.read(BinaryDocument.java:105) > at org.exist.storage.NativeBroker$DocumentCallback.indexInfo(NativeBroker.java:3439) > at org.exist.storage.btree.BTree$BTreeNode.query(BTree.java:1689) > at org.exist.storage.btree.BTree$BTreeNode.query(BTree.java:1620) > at org.exist.storage.btree.BTree$BTreeNode.access$3(BTree.java:1591) > at org.exist.storage.btree.BTree.query(BTree.java:327) > at org.exist.storage.NativeBroker.getCollectionResources(NativeBroker.java:1872) > at org.exist.collections.Collection.read(Collection.java:731) > at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:755) > at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:724) > at org.exist.storage.NativeBroker.getCollection(NativeBroker.java:719) > at org.exist.xmldb.LocalCollection.getCollection(LocalCollection.java:213) > at org.exist.xmldb.LocalCollection.<init>(LocalCollection.java:154) > at org.exist.xmldb.LocalCollection.<init>(LocalCollection.java:123) > at org.exist.xquery.functions.xmldb.XMLDBAbstractCollectionManipulator.createLocalCollection(XMLDBAbstractCollectionManipulator.java:77) > at org.exist.xquery.functions.xmldb.XMLDBAbstractCollectionManipulator.eval(XMLDBAbstractCollectionManipulator.java:129) > at org.exist.xquery.BasicFunction.eval(BasicFunction.java:68) > at org.exist.xquery.InternalFunctionCall.eval(InternalFunctionCall.java:55) > at org.exist.xquery.LetExpr.eval(LetExpr.java:155) > at org.exist.xquery.LetExpr.eval(LetExpr.java:208) > at org.exist.xquery.ForExpr.eval(ForExpr.java:317) > at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156) > at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:135) > at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61) > at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:278) > at org.exist.xquery.FunctionCall.eval(FunctionCall.java:205) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.PathExpr.eval(PathExpr.java:243) > at org.exist.xquery.SequenceConstructor.eval(SequenceConstructor.java:83) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.PathExpr.eval(PathExpr.java:243) > at org.exist.xquery.EnclosedExpr.eval(EnclosedExpr.java:70) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.PathExpr.eval(PathExpr.java:243) > at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279) > at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:135) > at org.exist.xquery.DynamicCardinalityCheck.eval(DynamicCardinalityCheck.java:72) > at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61) > at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:278) > at org.exist.xquery.FunctionCall.eval(FunctionCall.java:205) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.PathExpr.eval(PathExpr.java:243) > at org.exist.xquery.EnclosedExpr.eval(EnclosedExpr.java:70) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.PathExpr.eval(PathExpr.java:243) > at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.PathExpr.eval(PathExpr.java:243) > at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279) > at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:56) > at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:63) > at org.exist.xquery.LetExpr.eval(LetExpr.java:210) > at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156) > at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:135) > at org.exist.xquery.DynamicCardinalityCheck.eval(DynamicCardinalityCheck.java:72) > at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61) > at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:278) > at org.exist.xquery.FunctionCall.eval(FunctionCall.java:205) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.PathExpr.eval(PathExpr.java:243) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.SwitchExpression.eval(SwitchExpression.java:110) > at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:56) > at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:63) > at org.exist.xquery.LetExpr.eval(LetExpr.java:210) > at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156) > at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:135) > at org.exist.xquery.DynamicCardinalityCheck.eval(DynamicCardinalityCheck.java:72) > at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61) > at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:278) > at org.exist.xquery.FunctionCall.eval(FunctionCall.java:205) > at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:100) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.PathExpr.eval(PathExpr.java:243) > at org.exist.xquery.EnclosedExpr.eval(EnclosedExpr.java:70) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.PathExpr.eval(PathExpr.java:243) > at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.PathExpr.eval(PathExpr.java:243) > at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.PathExpr.eval(PathExpr.java:243) > at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.PathExpr.eval(PathExpr.java:243) > at org.exist.xquery.SequenceConstructor.eval(SequenceConstructor.java:83) > at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:56) > at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:63) > at org.exist.xquery.LetExpr.eval(LetExpr.java:210) > at org.exist.xquery.LetExpr.eval(LetExpr.java:208) > at org.exist.xquery.LetExpr.eval(LetExpr.java:208) > at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.PathExpr.eval(PathExpr.java:243) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70) > at org.exist.xquery.XQuery.execute(XQuery.java:246) > at org.exist.xquery.XQuery.execute(XQuery.java:201) > at org.exist.http.servlets.XQueryServlet.process(XQueryServlet.java:512) > at org.exist.http.servlets.XQueryServlet.doGet(XQueryServlet.java:217) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:538) > at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:478) > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) > at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:517) > at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225) > at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937) > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406) > at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183) > at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871) > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) > at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:239) > at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:115) > at org.exist.http.urlrewrite.Forward.doRewrite(Forward.java:47) > at org.exist.http.urlrewrite.XQueryURLRewrite.doRewrite(XQueryURLRewrite.java:525) > at org.exist.http.urlrewrite.XQueryURLRewrite.doFilter(XQueryURLRewrite.java:344) > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1323) > at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:476) > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) > at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:480) > at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225) > at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937) > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406) > at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183) > at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871) > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) > at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) > at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110) > at org.eclipse.jetty.server.Server.handle(Server.java:346) > at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:589) > at org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048) > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:601) > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:214) > at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:411) > at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:531) > at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40) > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529) > at java.lang.Thread.run(Thread.java:636) > > I have tried to restore the problem directory from a backup, but I get > similar exceptions as well. > > I have shut down the server, removed the .~lock file from the > $EXIST_HOME/webapp/WEB-INF/data/fs structure, then restarted, also > without a change. > > Reindexing also hasn't worked. > > Do you have any recommendation of other things I can try? > > -- > Michael Westbay > Writer/System Administrator > http://www.japanesebaseball.com/ > > ------------------------------------------------------------------------------ > Get your Android app more play: Bring it to the BlackBerry PlayBook > in minutes. BlackBerry App World™ now supports Android™ Apps > for the BlackBerry® PlayBook™. Discover just how easy and simple > it is! http://p.sf.net/sfu/android-dev2dev > _______________________________________________ > Exist-development mailing list > Exi...@li... > https://lists.sourceforge.net/lists/listinfo/exist-development > -- Adam Retter eXist Developer { United Kingdom } ad...@ex... irc://irc.freenode.net/existdb |
|
From: Dmitriy S. <sha...@gm...> - 2011-11-02 05:56:43
|
For me this safe solution & fix. Transactions ALWAYS MUST be in
try-catch-final block.
BTW, transaction API can be improved by success()/fail() + finish() methods.
tx = db.startTransaction();
try {
....
if (....) {
throw new ...
}
....
tx.success();
} catch {
tx.fail();
} finally {
tx.finish();
}
On Wed, Nov 2, 2011 at 2:39 AM, Patrick Bosek <pat...@jo...>wrote:
> Hi everyone,
>
> I've found, and fixed, a bug in the xmlrpc.RpcConnection.java. I was going
> to simply commit it, but I thought it might be best to run it past the
> developers first since I'm only 99% sure of what I've done.
>
> The remove(..) function on like 2615 catches any Throwable on line 2643,
> but it doesn't abort it's transaction. So, if there is an exception (like
> the target file being locked), the transaction never gets committed and
> never gets aborted, which is bad (obviously).
>
> What I did to fix this was remove the aborts that sat above the exceptions
> the function threw it's self and added one into the catch(Throwable)
>
> See here:
>
>
> private boolean remove(XmldbURI docUri) throws EXistException,
> PermissionDeniedException {
> TransactionManager transact =
> factory.getBrokerPool().getTransactionManager();
> Txn transaction = transact.beginTransaction();
> DBBroker broker = null;
> Collection collection = null;
> try {
> broker = factory.getBrokerPool().get(user);
> collection = broker.openCollection(docUri.removeLastSegment(),
> Lock.WRITE_LOCK);
> if (collection == null) {
> *--------- START REMOVED -------------*
> // transact.abort(transaction);
> *--------- START REMOVED -------------*
> throw new EXistException("Collection " +
> docUri.removeLastSegment() + " not found");
> }
> // keep the write lock in the transaction
> transaction.registerLock(collection.getLock(),
> Lock.WRITE_LOCK);
>
> DocumentImpl doc = collection.getDocument(broker,
> docUri.lastSegment());
> if (doc == null) {
> *--------- START REMOVED -------------*
> // transact.abort(transaction);
> *--------- END REMOVED -------------*
> throw new EXistException("Document " + docUri + " not
> found");
> }
>
> if(doc.getResourceType() == DocumentImpl.BINARY_FILE)
> collection.removeBinaryResource(transaction, broker, doc);
> else
> collection.removeXMLResource(transaction, broker,
> docUri.lastSegment());
> transact.commit(transaction);
> return true;
>
> } catch (Throwable e) {
> *--------- START ADDED -------------*
> transact.abort(transaction);
> *--------- END ADDED -------------*
> handleException(e);
> return false;
>
> } finally {
> factory.getBrokerPool().release(broker);
> }
> }
>
>
>
> My only question about this is what happens in the case there is an
> exception thrown from the transaction.commit(..)? From looking at the
> function it wasn't immediately obvious to me if this was going to create an
> issue.
>
>
> Thoughts?
>
>
> --
> Patrick Bosek
> Jorsek Software
> Cell (585) 820 9634
> Office (877) 492 2960
> Jorsek.com
>
>
>
> ------------------------------------------------------------------------------
> RSA® Conference 2012
> Save $700 by Nov 18
> Register now!
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>
>
--
Dmitriy Shabanov
|
|
From: Patrick B. <pat...@jo...> - 2011-11-01 21:39:14
|
Hi everyone,
I've found, and fixed, a bug in the xmlrpc.RpcConnection.java. I was going
to simply commit it, but I thought it might be best to run it past the
developers first since I'm only 99% sure of what I've done.
The remove(..) function on like 2615 catches any Throwable on line 2643,
but it doesn't abort it's transaction. So, if there is an exception (like
the target file being locked), the transaction never gets committed and
never gets aborted, which is bad (obviously).
What I did to fix this was remove the aborts that sat above the exceptions
the function threw it's self and added one into the catch(Throwable)
See here:
private boolean remove(XmldbURI docUri) throws EXistException,
PermissionDeniedException {
TransactionManager transact =
factory.getBrokerPool().getTransactionManager();
Txn transaction = transact.beginTransaction();
DBBroker broker = null;
Collection collection = null;
try {
broker = factory.getBrokerPool().get(user);
collection = broker.openCollection(docUri.removeLastSegment(),
Lock.WRITE_LOCK);
if (collection == null) {
*--------- START REMOVED -------------*
// transact.abort(transaction);
*--------- START REMOVED -------------*
throw new EXistException("Collection " +
docUri.removeLastSegment() + " not found");
}
// keep the write lock in the transaction
transaction.registerLock(collection.getLock(), Lock.WRITE_LOCK);
DocumentImpl doc = collection.getDocument(broker,
docUri.lastSegment());
if (doc == null) {
*--------- START REMOVED -------------*
// transact.abort(transaction);
*--------- END REMOVED -------------*
throw new EXistException("Document " + docUri + " not
found");
}
if(doc.getResourceType() == DocumentImpl.BINARY_FILE)
collection.removeBinaryResource(transaction, broker, doc);
else
collection.removeXMLResource(transaction, broker,
docUri.lastSegment());
transact.commit(transaction);
return true;
} catch (Throwable e) {
*--------- START ADDED -------------*
transact.abort(transaction);
*--------- END ADDED -------------*
handleException(e);
return false;
} finally {
factory.getBrokerPool().release(broker);
}
}
My only question about this is what happens in the case there is an
exception thrown from the transaction.commit(..)? From looking at the
function it wasn't immediately obvious to me if this was going to create an
issue.
Thoughts?
--
Patrick Bosek
Jorsek Software
Cell (585) 820 9634
Office (877) 492 2960
Jorsek.com
|
|
From: Michael W. <wes...@ja...> - 2011-10-30 04:45:02
|
I had edited the file mlb-po-overall-2011-10-17.ods with LibraOffice
within the database via WebDAV. This generated a file
.~lock.mlb-po-overall-2011-10-17.ods#. I then use eXist to extract
the XML content and process/populate data in the database. At some
unknown time, well after I thought I had closed this file, I started
getting the following error:
Invalid URI: xmldb URI scheme has no instance name:
xmldb:.~lock.mlb-po-overall-2011-10-17.ods#
This error has since caused any WebDAV or admin client (web based and
application based) to fail with a permission not allowed error to the
collection *above* where this file resides. XPath/XQuery queries to
the data in adjacent collections work. The stacktrace that I get
trying to access the parent collection with the web based admin tool
follows:
2011-10-30 11:36:45,611 [DefaultQuartzScheduler_Worker-4] DEBUG
(NGramIndex.java [sync]:91) - SYNC NGRAM
2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO
(NativeBroker.java [sync]:3141) - Memory: 497,352K total; 748,352K
max; 31,171K free
2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO
(BTree.java [printStatistics]:2420) - dom.dbx INDEX Buffers occupation
: 100% (1093 out of 1093) Cache efficiency : 100%
2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO
(DOMFile.java [printStatistics]:1441) - dom.dbx DATA Buffers
occupation : 100% (255 out of 256) Cache efficiency : 94%
2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO
(BTree.java [printStatistics]:2420) - collections.dbx INDEX Buffers
occupation : 84% (54 out of 64) Cache efficiency : 100%
2011-10-30 11:36:45,612 [DefaultQuartzScheduler_Worker-4] INFO
(BFile.java [printStatistics]:434) - collections.dbx DATA Buffers
occupation : 99% (99 out of 100) Cache efficiency : 99%
2011-10-30 11:36:45,613 [DefaultQuartzScheduler_Worker-4] INFO
(BTree.java [printStatistics]:2420) - values.dbx INDEX Buffers
occupation : 100% (144 out of 144) Cache efficiency : 100%
2011-10-30 11:36:45,613 [DefaultQuartzScheduler_Worker-4] INFO
(BFile.java [printStatistics]:434) - values.dbx DATA Buffers
occupation : 100% (1796 out of 1797) Cache efficiency : 99%
2011-10-30 11:36:45,613 [DefaultQuartzScheduler_Worker-4] DEBUG
(Journal.java [checkpoint]:328) - Checkpoint reached
2011-10-30 11:36:45,613 [DefaultQuartzScheduler_Worker-4] DEBUG
(NotificationService.java [debug]:94) - Registered UpdateListeners:
2011-10-30 11:37:42,704 [eXistThread-49] DEBUG
(HttpRequestWrapper.java [<init>]:129) - Retrieved 2 parameters.
2011-10-30 11:37:42,706 [eXistThread-49] DEBUG (XQuery.java
[execute]:249) - Execution took 1 ms
2011-10-30 11:37:42,706 [eXistThread-49] DEBUG (HTTPUtils.java
[addLastModifiedHeader]:61) - mostRecentDocumentTime: 0
2011-10-30 11:37:42,707 [eXistThread-49] DEBUG
(HttpRequestWrapper.java [<init>]:129) - Retrieved 2 parameters.
2011-10-30 11:37:42,714 [eXistThread-49] ERROR (XQueryServlet.java
[process]:573) - Invalid URI: xmldb URI scheme has no instance name:
xmldb:.~lock.mlb-po-overall-2011-10-17.ods#
java.lang.IllegalArgumentException: Invalid URI: xmldb URI scheme has
no instance name: xmldb:.~lock.mlb-po-overall-2011-10-17.ods#
at org.exist.xmldb.XmldbURI.create(XmldbURI.java:181)
at org.exist.dom.BinaryDocument.read(BinaryDocument.java:105)
at org.exist.storage.NativeBroker$DocumentCallback.indexInfo(NativeBroker.java:3439)
at org.exist.storage.btree.BTree$BTreeNode.query(BTree.java:1689)
at org.exist.storage.btree.BTree$BTreeNode.query(BTree.java:1620)
at org.exist.storage.btree.BTree$BTreeNode.access$3(BTree.java:1591)
at org.exist.storage.btree.BTree.query(BTree.java:327)
at org.exist.storage.NativeBroker.getCollectionResources(NativeBroker.java:1872)
at org.exist.collections.Collection.read(Collection.java:731)
at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:755)
at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:724)
at org.exist.storage.NativeBroker.getCollection(NativeBroker.java:719)
at org.exist.xmldb.LocalCollection.getCollection(LocalCollection.java:213)
at org.exist.xmldb.LocalCollection.<init>(LocalCollection.java:154)
at org.exist.xmldb.LocalCollection.<init>(LocalCollection.java:123)
at org.exist.xquery.functions.xmldb.XMLDBAbstractCollectionManipulator.createLocalCollection(XMLDBAbstractCollectionManipulator.java:77)
at org.exist.xquery.functions.xmldb.XMLDBAbstractCollectionManipulator.eval(XMLDBAbstractCollectionManipulator.java:129)
at org.exist.xquery.BasicFunction.eval(BasicFunction.java:68)
at org.exist.xquery.InternalFunctionCall.eval(InternalFunctionCall.java:55)
at org.exist.xquery.LetExpr.eval(LetExpr.java:155)
at org.exist.xquery.LetExpr.eval(LetExpr.java:208)
at org.exist.xquery.ForExpr.eval(ForExpr.java:317)
at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156)
at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:135)
at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61)
at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:278)
at org.exist.xquery.FunctionCall.eval(FunctionCall.java:205)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.PathExpr.eval(PathExpr.java:243)
at org.exist.xquery.SequenceConstructor.eval(SequenceConstructor.java:83)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.PathExpr.eval(PathExpr.java:243)
at org.exist.xquery.EnclosedExpr.eval(EnclosedExpr.java:70)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.PathExpr.eval(PathExpr.java:243)
at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279)
at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:135)
at org.exist.xquery.DynamicCardinalityCheck.eval(DynamicCardinalityCheck.java:72)
at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61)
at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:278)
at org.exist.xquery.FunctionCall.eval(FunctionCall.java:205)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.PathExpr.eval(PathExpr.java:243)
at org.exist.xquery.EnclosedExpr.eval(EnclosedExpr.java:70)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.PathExpr.eval(PathExpr.java:243)
at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.PathExpr.eval(PathExpr.java:243)
at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279)
at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:56)
at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:63)
at org.exist.xquery.LetExpr.eval(LetExpr.java:210)
at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156)
at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:135)
at org.exist.xquery.DynamicCardinalityCheck.eval(DynamicCardinalityCheck.java:72)
at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61)
at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:278)
at org.exist.xquery.FunctionCall.eval(FunctionCall.java:205)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.PathExpr.eval(PathExpr.java:243)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.SwitchExpression.eval(SwitchExpression.java:110)
at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:56)
at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:63)
at org.exist.xquery.LetExpr.eval(LetExpr.java:210)
at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156)
at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:135)
at org.exist.xquery.DynamicCardinalityCheck.eval(DynamicCardinalityCheck.java:72)
at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61)
at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:278)
at org.exist.xquery.FunctionCall.eval(FunctionCall.java:205)
at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:100)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.PathExpr.eval(PathExpr.java:243)
at org.exist.xquery.EnclosedExpr.eval(EnclosedExpr.java:70)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.PathExpr.eval(PathExpr.java:243)
at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.PathExpr.eval(PathExpr.java:243)
at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.PathExpr.eval(PathExpr.java:243)
at org.exist.xquery.ElementConstructor.eval(ElementConstructor.java:279)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.PathExpr.eval(PathExpr.java:243)
at org.exist.xquery.SequenceConstructor.eval(SequenceConstructor.java:83)
at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:56)
at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:63)
at org.exist.xquery.LetExpr.eval(LetExpr.java:210)
at org.exist.xquery.LetExpr.eval(LetExpr.java:208)
at org.exist.xquery.LetExpr.eval(LetExpr.java:208)
at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.PathExpr.eval(PathExpr.java:243)
at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:70)
at org.exist.xquery.XQuery.execute(XQuery.java:246)
at org.exist.xquery.XQuery.execute(XQuery.java:201)
at org.exist.http.servlets.XQueryServlet.process(XQueryServlet.java:512)
at org.exist.http.servlets.XQueryServlet.doGet(XQueryServlet.java:217)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:538)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:478)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:517)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:239)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:115)
at org.exist.http.urlrewrite.Forward.doRewrite(Forward.java:47)
at org.exist.http.urlrewrite.XQueryURLRewrite.doRewrite(XQueryURLRewrite.java:525)
at org.exist.http.urlrewrite.XQueryURLRewrite.doFilter(XQueryURLRewrite.java:344)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1323)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:476)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:480)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
at org.eclipse.jetty.server.Server.handle(Server.java:346)
at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:589)
at org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:601)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:214)
at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:411)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:531)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
at java.lang.Thread.run(Thread.java:636)
I have tried to restore the problem directory from a backup, but I get
similar exceptions as well.
I have shut down the server, removed the .~lock file from the
$EXIST_HOME/webapp/WEB-INF/data/fs structure, then restarted, also
without a change.
Reindexing also hasn't worked.
Do you have any recommendation of other things I can try?
--
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/
|
|
From: Alain C. <ala...@ag...> - 2011-10-28 15:54:34
|
Hello, I have just integrated TinyMCE in XSLTForms to allow rich text editing with a large range of possible parameters. In XSLTForms SF repositories, I copied the latest version of TinyMCE in a sub-folder named "scripts/tinymce_3.4.6" and I wonder what to do within eXist repository. In eXist SF repository, is it better to add, instead, a "tinymce" sub-folder in the "webapp/scripts" folder? Thanks! -Alain |
|
From: Dmitriy S. <sha...@gm...> - 2011-10-21 02:55:09
|
On Thu, Oct 20, 2011 at 11:46 PM, Adam Retter <ad...@ex...> wrote: > Yes I have, but that doesnt help really!!! What are you saying? Most > of the file processing we do is XSLT based, Maven supports XSLT. The > other file processing is properties processing, Maven also supports > that. Then of course there is building a package of some sort > (WAR/IzPack etc), also Maven supports that. > Maven do support it through pluggins, that what I did say. Same as you did, but looks like you can type faster or have more spare time =) -- Dmitriy Shabanov |
|
From: Adam R. <ad...@ex...> - 2011-10-20 18:46:23
|
Yes I have, but that doesnt help really!!! What are you saying? Most
of the file processing we do is XSLT based, Maven supports XSLT. The
other file processing is properties processing, Maven also supports
that. Then of course there is building a package of some sort
(WAR/IzPack etc), also Maven supports that.
So what doesnt it support that we need exactly?
On 20 October 2011 18:18, Dmitriy Shabanov <sha...@gm...> wrote:
> see it in context....
>
> On Thu, Oct 20, 2011 at 5:44 PM, Adam Retter <ad...@ex...> wrote:
>>
>> On 20 October 2011 02:55, Dmitriy Shabanov <sha...@gm...> wrote:
>> > On Thu, Oct 20, 2011 at 12:55 AM, Dannes Wessels <di...@ex...>
>> > wrote:
>> >>
>> >> I am on the maven side, I am convinced in the Prague days. I'll always
>> >> be
>> >> healthy critical though on all used tooling, maven in special because
>> >> there
>> >> are too many fanboys that think maven is the golden tool.
>> >>
>> >> At the moment we switch to maven, we need to re-do lots of logic we
>> >> have
>> >> coded in ant right now. Maven does not provide all we need :) think of
>> >> file
>> >> processing, rewriting. And.... The current layout with conf.XML does
>> >> not
>> >> really map on the maven structure.....
>> >
>> > file processing, rewriting - pluggins???
>>
>> Its hard or impossible to understand what you are talking about from
>> just those few words. Please elaborate?
>
>
> --
> Dmitriy Shabanov
>
--
Adam Retter
eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb
|
|
From: Dmitriy S. <sha...@gm...> - 2011-10-20 17:18:24
|
see it in context.... On Thu, Oct 20, 2011 at 5:44 PM, Adam Retter <ad...@ex...> wrote: > On 20 October 2011 02:55, Dmitriy Shabanov <sha...@gm...> wrote: > > On Thu, Oct 20, 2011 at 12:55 AM, Dannes Wessels <di...@ex...> > > wrote: > >> > >> I am on the maven side, I am convinced in the Prague days. I'll always > be > >> healthy critical though on all used tooling, maven in special because > there > >> are too many fanboys that think maven is the golden tool. > >> > >> At the moment we switch to maven, we need to re-do lots of logic we have > >> coded in ant right now. Maven does not provide all we need :) *think of > file > >> processing, rewriting*. And.... The current layout with conf.XML does > not > >> really map on the maven structure..... > > > > file processing, rewriting - pluggins??? > > Its hard or impossible to understand what you are talking about from > just those few words. Please elaborate? > -- Dmitriy Shabanov |
|
From: Adam R. <ad...@ex...> - 2011-10-20 12:44:39
|
On 20 October 2011 02:55, Dmitriy Shabanov <sha...@gm...> wrote: > On Thu, Oct 20, 2011 at 12:55 AM, Dannes Wessels <di...@ex...> > wrote: >> >> I am on the maven side, I am convinced in the Prague days. I'll always be >> healthy critical though on all used tooling, maven in special because there >> are too many fanboys that think maven is the golden tool. >> >> At the moment we switch to maven, we need to re-do lots of logic we have >> coded in ant right now. Maven does not provide all we need :) think of file >> processing, rewriting. And.... The current layout with conf.XML does not >> really map on the maven structure..... > > -1 : configuration is standardized staff > http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html You dont have to use the standard directory layout, you can use any layout you want. However I think it would be good to use the standard directory layout as this adopted widely even in situations where the projects are not maven based. In fact switching eXist to a standard directory layout is very very easy, I have done it here. > > file processing, rewriting - pluggins??? Its hard or impossible to understand what you are talking about from just those few words. Please elaborate? > -- > Dmitriy Shabanov > -- Adam Retter eXist Developer { United Kingdom } ad...@ex... irc://irc.freenode.net/existdb |
|
From: Dmitriy S. <sha...@gm...> - 2011-10-20 09:28:05
|
Another solution is eXist as maven plugin (like jetty). -- Dmitriy Shabanov |
|
From: Dmitriy S. <sha...@gm...> - 2011-10-20 09:25:35
|
On Thu, Oct 20, 2011 at 1:19 PM, Dannes Wessels <di...@ex...> wrote: > On 20 okt. 2011, at 03:55, Dmitriy Shabanov <sha...@gm...> wrote: > > On Thu, Oct 20, 2011 at 12:55 AM, Dannes Wessels <di...@ex...>wrote: > >> >> I am on the maven side, I am convinced in the Prague days. I'll always be >> healthy critical though on all used tooling, maven in special because there >> are too many fanboys that think maven is the golden tool. >> >> At the moment we switch to maven, we need to re-do lots of logic we have >> coded in ant right now. Maven does not provide all we need :) think of file >> processing, rewriting. And.... The current layout with conf.XML does not >> really map on the maven structure..... >> > > -1 : configuration is standardized staff > > > I need to be convinced for this.... > Well, several solution: I do like one - move conf.xml to parent folder of data one. > file processing, rewriting - pluggins??? > > > Jups, this implies lots of work, unless proven differently. We can't > underestimate this amount of work.... > > agree... -- Dmitriy Shabanov |
|
From: Dannes W. <di...@ex...> - 2011-10-20 08:18:33
|
On 20 okt. 2011, at 03:55, Dmitriy Shabanov <sha...@gm...> wrote: > On Thu, Oct 20, 2011 at 12:55 AM, Dannes Wessels <di...@ex...> wrote: > > I am on the maven side, I am convinced in the Prague days. I'll always be healthy critical though on all used tooling, maven in special because there are too many fanboys that think maven is the golden tool. > > At the moment we switch to maven, we need to re-do lots of logic we have coded in ant right now. Maven does not provide all we need :) think of file processing, rewriting. And.... The current layout with conf.XML does not really map on the maven structure..... > > -1 : configuration is standardized staff I need to be convinced for this.... > http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html hmmm rather unclear and ambiguous piece of text. > file processing, rewriting - pluggins??? Jups, this implies lots of work, unless proven differently. We can't underestimate this amount of work.... D. |
|
From: Dmitriy S. <sha...@gm...> - 2011-10-20 01:55:15
|
On Thu, Oct 20, 2011 at 12:55 AM, Dannes Wessels <di...@ex...>wrote: > > I am on the maven side, I am convinced in the Prague days. I'll always be > healthy critical though on all used tooling, maven in special because there > are too many fanboys that think maven is the golden tool. > > At the moment we switch to maven, we need to re-do lots of logic we have > coded in ant right now. Maven does not provide all we need :) think of file > processing, rewriting. And.... The current layout with conf.XML does not > really map on the maven structure..... > -1 : configuration is standardized staff http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html file processing, rewriting - pluggins??? > -- Dmitriy Shabanov |
|
From: Adam R. <ad...@ex...> - 2011-10-19 20:32:13
|
How are you applying changes, what messages do you see in exist.log or urlrewrite.log etc? 2011/10/19 Василий Старцев <wst...@gm...>: > Hi, > I'm update my exist from trunk form rev15402 to latest rev15459. > svn info > Path: . > URL: https://exist.svn.sourceforge.net/svnroot/exist/trunk/eXist > Repository Root: https://exist.svn.sourceforge.net/svnroot/exist > Repository UUID: 8784284e-840f-0410-bd5a-a6a6012a0d1a > Revision: 15459 > Node Kind: directory > Schedule: normal > Last Changed Author: wolfgang_m > Last Changed Rev: 15459 > Last Changed Date: 2011-10-17 11:15:05 +0200 (Mon, 17 Oct 2011) > and try to change permissions through exist client. > And when I apply permission changes exist is no answer for any > requests from browser, oXygen (xmlrpc) or client. > Htop on server show zero loading in all cpu's. > -- > С наилучшими пожеланиями > Василий Старцев > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense. > http://p.sf.net/sfu/splunk-d2d-oct > _______________________________________________ > Exist-development mailing list > Exi...@li... > https://lists.sourceforge.net/lists/listinfo/exist-development > > -- Adam Retter eXist Developer { United Kingdom } ad...@ex... irc://irc.freenode.net/existdb |
|
From: Dannes W. <di...@ex...> - 2011-10-19 19:54:03
|
Ow yeas, I am on the maven side, I am convinced in the Prague days. I'll always be healthy critical though on all used tooling, maven in special because there are too many fanboys that think maven is the golden tool. At the moment we switch to maven, we need to re-do lots of logic we have coded in ant right now. Maven does not provide all we need :) think of file processing, rewriting. And.... The current layout with conf.XML does not really map on the maven structure..... Ps. I think pessimistic is a bad choice of words Ps2. For tooling we should go mainstream. Mainstream at this moment is ant, maven or make...... :) D. On 18 okt. 2011, at 15:12, Adam Retter <ad...@ex...> wrote: > Yes he was, because he had some concerns, which we discussed and I > think I addressed each of these to his satisfaction? |
|
From: Василий С. <wst...@gm...> - 2011-10-19 09:07:07
|
Hi, I'm update my exist from trunk form rev15402 to latest rev15459. *svn info* Path: . URL: https://exist.svn.sourceforge.net/svnroot/exist/trunk/eXist Repository Root: https://exist.svn.sourceforge.net/svnroot/exist Repository UUID: 8784284e-840f-0410-bd5a-a6a6012a0d1a Revision: 15459 Node Kind: directory Schedule: normal Last Changed Author: wolfgang_m Last Changed Rev: 15459 Last Changed Date: 2011-10-17 11:15:05 +0200 (Mon, 17 Oct 2011) and try to change permissions through exist client. And when I apply permission changes exist is no answer for any requests from browser, oXygen (xmlrpc) or client. Htop on server show zero loading in all cpu's. -- С наилучшими пожеланиями Василий Старцев |
|
From: Dmitriy S. <sha...@gm...> - 2011-10-18 16:16:21
|
On Tue, Oct 18, 2011 at 6:12 PM, Adam Retter <ad...@ex...> wrote: > However the 'proof is in the pudding' as we say, so that is why I > started on a branch to create a Mavenised build of eXist-db, to prove > this is possible and hopefully to demonstrate the improvements. If at > the end of development, I cannot demonstrate improvemnets then it is > indeed a failure and we should not use Maven. > Adam, can you summarize last mile problems that left? > -- Dmitriy Shabanov |
|
From: Chris T. <chr...@gm...> - 2011-10-18 15:52:00
|
Adam,
Thank you for the two emails. I think that will help all of manage expectations and plan for where we are going.
I'm looking at several interim solutions for our immediate needs.
Thanks again,
Chris
On Oct 18, 2011, at 9:17 PM, Adam Retter wrote:
> As for for the Security Roadmap -
>
> 0) Replace the Configurator with a better design.
>
> 1) Implement SetUID and SetGID
>
> 2) Implement Sticky bit.
>
> 3) Implement ACL permission inheritance
>
> 4) Move toward full NFSv4 ACL's
>
> 5) Replace the security manager core with an identified 3rd party library.
>
> 6) Implement a global Security Session Manager
>
> 7) Re-work the HTTP and REST security abstractions for consistency
> with the global Session Security Manager.
>
> Cheers Adam.
>
> On 18 October 2011 16:29, Adam Retter <ad...@ex...> wrote:
>> Security stuff status -
>>
>> Basic ACL support is complete.
>> Security permissions checks are now much faster.
>> LDAP support is done, as is Active Directory
>> OpenId and OAuth have also been added.
>>
>> Security stuff TODO for 1.6 -
>>
>> 1) Finish migrating from 'rwu' to 'rwx', style permissions, actually I
>> have that done here. But a lot of test cases broke because they were
>> wired around the existing permissions model.
>>
>> 2) Align the security requirements on operations to the Unix model.
>> That is to say, for example, when you delete a file, the permissions
>> you need to remove a file depend on the permissions of the parent
>> collection and not the file itself. The current model in eXist is
>> somewhat ad-hoc, sometimes it aligns with the Unix model and other
>> times it does not. We need a consistent approach, so we all know what
>> assigning these permissions mean.
>> In fact, im 1/2 way through this, but again the thing that is holding
>> me up is having to fix 100's of tests that have always relied on
>> rather relaxed permissions in eXist-db.
>>
>> 3) Fix a deadlock issue, this is a design problem with the
>> Configurator. This can be demonstrated through heavy use of the
>> SecuirtyManager. This will be tricky but not impossible, I have an
>> idea for the approach already in mind.
>>
>> 4) Document the new Security architecture.
>>
>> All in all this is about 1 weeks work if I can sit down and focus on
>> this for one week solid. My efforts on this are not sponsored and so I
>> have to fit this in amongst work that is sponsored (higher priority).
>> As I mentioned, it looks like I may have a clear week coming up in the
>> near future, and I hope to complete this then.
>>
>> On 18 October 2011 14:32, Chris Tomlinson <chr...@gm...> wrote:
>>> Adam,
>>>
>>> It would be a great help if we had a simple roadmap for the security rework. If you can point me to a thread that has the details or hopefully quickly sketch out where we are and where we're going that would be great. I've been following the security stuff and actually thought it had settled down in the early summer and assumed that it was now stabilizing.
>>>
>>> In fact what what other major items are on the to do list for the next stabilization - 1.6?
>>>
>>> Thanks,
>>> Chris
>>>
>>>
>>> On Oct 18, 2011, at 7:08 PM, Adam Retter wrote:
>>>
>>>> Yes I have read your emails. I also just responded. I am not deaf to
>>>> the requirement but please do not enable it right at the moment as it
>>>> will impact on outstanding changes I have. As this is all in flux
>>>> before the next release anyway, if you enable it now, it will most
>>>> likely be disabled again later in some way when it is replaced...
>>>>
>>>>
>>>>
>>>> On 18 October 2011 14:12, Wolfgang Meier <wol...@ex...> wrote:
>>>>> Adam,
>>>>>
>>>>>> Default permissions are no longer configurable, this was a security
>>>>>> design concern.
>>>>>
>>>>> Have you read our emails? I think there are many arguments why we
>>>>> still need default permissions and I definitely would like to
>>>>> re-enable them.
>>>>>
>>>>>> New ACL's will permit inheritance in the near future, so this would be
>>>>>> the mechanism for replace such use cases.
>>>>>
>>>>> ... at least until this mechanism is in place and has been tested.
>>>>>
>>>>> Wolfgang
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Adam Retter
>>>>
>>>> eXist Developer
>>>> { United Kingdom }
>>>> ad...@ex...
>>>> irc://irc.freenode.net/existdb
>>>>
>>>> ------------------------------------------------------------------------------
>>>> All the data continuously generated in your IT infrastructure contains a
>>>> definitive record of customers, application performance, security
>>>> threats, fraudulent activity and more. Splunk takes this data and makes
>>>> sense of it. Business sense. IT sense. Common sense.
>>>> http://p.sf.net/sfu/splunk-d2d-oct
>>>> _______________________________________________
>>>> Exist-development mailing list
>>>> Exi...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/exist-development
>>>
>>>
>>
>>
>>
>> --
>> Adam Retter
>>
>> eXist Developer
>> { United Kingdom }
>> ad...@ex...
>> irc://irc.freenode.net/existdb
>>
>
>
>
> --
> Adam Retter
>
> eXist Developer
> { United Kingdom }
> ad...@ex...
> irc://irc.freenode.net/existdb
|
|
From: Chris T. <chr...@gm...> - 2011-10-18 15:47:18
|
Wolfgang, Thank you for that summary. It confirms my anecdotal experience that the system seemed to be converging and that the major developments were coming to a close. The improvements in indexing and performance have been quite noticeable for us and greatly appreciated. I can work with the permissions issue I think - it just kind of caught me off guard. As you pointed out in another post there's a lot of stuff that does Chris On Oct 18, 2011, at 7:49 PM, Wolfgang Meier wrote: >> In fact what what other major items are on the to do list for the next stabilization - 1.6? > > There are not many issues which block a first beta release towards the > next stable version: > > * there's a deadlock situation in the security manager - or rather: > the Configuration class which reads account information. This is easy > to reproduce but not so easy to fix. The changes Adam mentioned should > be addressing this issue. > * we have to provide a faster method for backing up large databases. > This was discussed on this list a few days ago. > * a few changes on the surface: we started to move all demo code out > of trunk and into installable packages. This has to be finished and > the web site changed accordingly to explain people how to get to the > demos. > > Apart from this, I have one thing on my list to further improve > performance. It is not required for 1.6, but would be nice to have: > redesign the range index in the same way as the structural index. The > reorganization of the structural index has led to a major performance > boost for complex queries. Now it's the range index which is blocking > further speed ups and should be changed along the same lines. > > Wolfgang |
|
From: Adam R. <ad...@ex...> - 2011-10-18 15:32:36
|
As for for the Security Roadmap -
0) Replace the Configurator with a better design.
1) Implement SetUID and SetGID
2) Implement Sticky bit.
3) Implement ACL permission inheritance
4) Move toward full NFSv4 ACL's
5) Replace the security manager core with an identified 3rd party library.
6) Implement a global Security Session Manager
7) Re-work the HTTP and REST security abstractions for consistency
with the global Session Security Manager.
Cheers Adam.
On 18 October 2011 16:29, Adam Retter <ad...@ex...> wrote:
> Security stuff status -
>
> Basic ACL support is complete.
> Security permissions checks are now much faster.
> LDAP support is done, as is Active Directory
> OpenId and OAuth have also been added.
>
> Security stuff TODO for 1.6 -
>
> 1) Finish migrating from 'rwu' to 'rwx', style permissions, actually I
> have that done here. But a lot of test cases broke because they were
> wired around the existing permissions model.
>
> 2) Align the security requirements on operations to the Unix model.
> That is to say, for example, when you delete a file, the permissions
> you need to remove a file depend on the permissions of the parent
> collection and not the file itself. The current model in eXist is
> somewhat ad-hoc, sometimes it aligns with the Unix model and other
> times it does not. We need a consistent approach, so we all know what
> assigning these permissions mean.
> In fact, im 1/2 way through this, but again the thing that is holding
> me up is having to fix 100's of tests that have always relied on
> rather relaxed permissions in eXist-db.
>
> 3) Fix a deadlock issue, this is a design problem with the
> Configurator. This can be demonstrated through heavy use of the
> SecuirtyManager. This will be tricky but not impossible, I have an
> idea for the approach already in mind.
>
> 4) Document the new Security architecture.
>
> All in all this is about 1 weeks work if I can sit down and focus on
> this for one week solid. My efforts on this are not sponsored and so I
> have to fit this in amongst work that is sponsored (higher priority).
> As I mentioned, it looks like I may have a clear week coming up in the
> near future, and I hope to complete this then.
>
> On 18 October 2011 14:32, Chris Tomlinson <chr...@gm...> wrote:
>> Adam,
>>
>> It would be a great help if we had a simple roadmap for the security rework. If you can point me to a thread that has the details or hopefully quickly sketch out where we are and where we're going that would be great. I've been following the security stuff and actually thought it had settled down in the early summer and assumed that it was now stabilizing.
>>
>> In fact what what other major items are on the to do list for the next stabilization - 1.6?
>>
>> Thanks,
>> Chris
>>
>>
>> On Oct 18, 2011, at 7:08 PM, Adam Retter wrote:
>>
>>> Yes I have read your emails. I also just responded. I am not deaf to
>>> the requirement but please do not enable it right at the moment as it
>>> will impact on outstanding changes I have. As this is all in flux
>>> before the next release anyway, if you enable it now, it will most
>>> likely be disabled again later in some way when it is replaced...
>>>
>>>
>>>
>>> On 18 October 2011 14:12, Wolfgang Meier <wol...@ex...> wrote:
>>>> Adam,
>>>>
>>>>> Default permissions are no longer configurable, this was a security
>>>>> design concern.
>>>>
>>>> Have you read our emails? I think there are many arguments why we
>>>> still need default permissions and I definitely would like to
>>>> re-enable them.
>>>>
>>>>> New ACL's will permit inheritance in the near future, so this would be
>>>>> the mechanism for replace such use cases.
>>>>
>>>> ... at least until this mechanism is in place and has been tested.
>>>>
>>>> Wolfgang
>>>>
>>>
>>>
>>>
>>> --
>>> Adam Retter
>>>
>>> eXist Developer
>>> { United Kingdom }
>>> ad...@ex...
>>> irc://irc.freenode.net/existdb
>>>
>>> ------------------------------------------------------------------------------
>>> All the data continuously generated in your IT infrastructure contains a
>>> definitive record of customers, application performance, security
>>> threats, fraudulent activity and more. Splunk takes this data and makes
>>> sense of it. Business sense. IT sense. Common sense.
>>> http://p.sf.net/sfu/splunk-d2d-oct
>>> _______________________________________________
>>> Exist-development mailing list
>>> Exi...@li...
>>> https://lists.sourceforge.net/lists/listinfo/exist-development
>>
>>
>
>
>
> --
> Adam Retter
>
> eXist Developer
> { United Kingdom }
> ad...@ex...
> irc://irc.freenode.net/existdb
>
--
Adam Retter
eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb
|
|
From: Adam R. <ad...@ex...> - 2011-10-18 15:29:25
|
Security stuff status -
Basic ACL support is complete.
Security permissions checks are now much faster.
LDAP support is done, as is Active Directory
OpenId and OAuth have also been added.
Security stuff TODO for 1.6 -
1) Finish migrating from 'rwu' to 'rwx', style permissions, actually I
have that done here. But a lot of test cases broke because they were
wired around the existing permissions model.
2) Align the security requirements on operations to the Unix model.
That is to say, for example, when you delete a file, the permissions
you need to remove a file depend on the permissions of the parent
collection and not the file itself. The current model in eXist is
somewhat ad-hoc, sometimes it aligns with the Unix model and other
times it does not. We need a consistent approach, so we all know what
assigning these permissions mean.
In fact, im 1/2 way through this, but again the thing that is holding
me up is having to fix 100's of tests that have always relied on
rather relaxed permissions in eXist-db.
3) Fix a deadlock issue, this is a design problem with the
Configurator. This can be demonstrated through heavy use of the
SecuirtyManager. This will be tricky but not impossible, I have an
idea for the approach already in mind.
4) Document the new Security architecture.
All in all this is about 1 weeks work if I can sit down and focus on
this for one week solid. My efforts on this are not sponsored and so I
have to fit this in amongst work that is sponsored (higher priority).
As I mentioned, it looks like I may have a clear week coming up in the
near future, and I hope to complete this then.
On 18 October 2011 14:32, Chris Tomlinson <chr...@gm...> wrote:
> Adam,
>
> It would be a great help if we had a simple roadmap for the security rework. If you can point me to a thread that has the details or hopefully quickly sketch out where we are and where we're going that would be great. I've been following the security stuff and actually thought it had settled down in the early summer and assumed that it was now stabilizing.
>
> In fact what what other major items are on the to do list for the next stabilization - 1.6?
>
> Thanks,
> Chris
>
>
> On Oct 18, 2011, at 7:08 PM, Adam Retter wrote:
>
>> Yes I have read your emails. I also just responded. I am not deaf to
>> the requirement but please do not enable it right at the moment as it
>> will impact on outstanding changes I have. As this is all in flux
>> before the next release anyway, if you enable it now, it will most
>> likely be disabled again later in some way when it is replaced...
>>
>>
>>
>> On 18 October 2011 14:12, Wolfgang Meier <wol...@ex...> wrote:
>>> Adam,
>>>
>>>> Default permissions are no longer configurable, this was a security
>>>> design concern.
>>>
>>> Have you read our emails? I think there are many arguments why we
>>> still need default permissions and I definitely would like to
>>> re-enable them.
>>>
>>>> New ACL's will permit inheritance in the near future, so this would be
>>>> the mechanism for replace such use cases.
>>>
>>> ... at least until this mechanism is in place and has been tested.
>>>
>>> Wolfgang
>>>
>>
>>
>>
>> --
>> Adam Retter
>>
>> eXist Developer
>> { United Kingdom }
>> ad...@ex...
>> irc://irc.freenode.net/existdb
>>
>> ------------------------------------------------------------------------------
>> All the data continuously generated in your IT infrastructure contains a
>> definitive record of customers, application performance, security
>> threats, fraudulent activity and more. Splunk takes this data and makes
>> sense of it. Business sense. IT sense. Common sense.
>> http://p.sf.net/sfu/splunk-d2d-oct
>> _______________________________________________
>> Exist-development mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-development
>
>
--
Adam Retter
eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb
|
|
From: Wolfgang M. <wol...@ex...> - 2011-10-18 14:04:45
|
> In fact what what other major items are on the to do list for the next stabilization - 1.6? There are not many issues which block a first beta release towards the next stable version: * there's a deadlock situation in the security manager - or rather: the Configuration class which reads account information. This is easy to reproduce but not so easy to fix. The changes Adam mentioned should be addressing this issue. * we have to provide a faster method for backing up large databases. This was discussed on this list a few days ago. * a few changes on the surface: we started to move all demo code out of trunk and into installable packages. This has to be finished and the web site changed accordingly to explain people how to get to the demos. Apart from this, I have one thing on my list to further improve performance. It is not required for 1.6, but would be nice to have: redesign the range index in the same way as the structural index. The reorganization of the structural index has led to a major performance boost for complex queries. Now it's the range index which is blocking further speed ups and should be changed along the same lines. Wolfgang |
64 messages has been excluded from this view by a project administrator.
