exist-development — eXist Developer's List

Re: [Exist-development] Pushing error message xquery line numbers into type comparisons

[Wolfgang M. <wol...@ex...>]

> This is just a simple example, see the attached diff, but I am
> wondering if there is value in, expanding on this concept and pushing
> xquery line context information into the datatype system for better
> errors.

It somehow doesn't feel right to add caller information to the
returned data values. Our approach so far has been to catch the thrown
exception on a higher level and add the line information there. See
class FunctionCall for an example. I think this concept could be
extended, though we won't be able to cover all cases where a type
error could occur.
But for your example, the if expression could certainly catch and
rethrow the exception.

Wolfgang

[Exist-development] Pushing error message xquery line numbers into type comparisons

[Adam R. <ad...@ex...>]

I had a problem with a type data conversion, in this case with
effectiveBooleanValue being called on AbstractDateTimeValue.

I could not find the cause of the error, and so I 'hacked' the code to
push the calling expression into the type comparisson, so that I could
then extract the line and column numbers of the calling XQuery to
annotate the XPathException that was thrown.

This is just a simple example, see the attached diff, but I am
wondering if there is value in, expanding on this concept and pushing
xquery line context information into the datatype system for better
errors.

A cleaner approach might be to change the exceptions thrown by the
types, then catch them in the expression, add the expression
information and then throw them as XPathExceptions?

The following xquery -

if(current-dateTime())then
()
else()

Will cause the error I was seeing, with my patch applied you will also
get the line number of where the error in the XQuery occurred.

Thoughts please :-)


-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

[Exist-development] re JSON and REST Server

[James F. <jam...@gm...>]

when I wrote this

http://xmltoday.org/content/turning-exist-db-json-server-part-i

I intended to augment eXist to churn out json ... but after hearing at
the user group that the REST server may get a refactor perhaps this is
the right place after all for this type of data transformation to
happen; so I wont be doing anything else with this for the time being.

J

[Exist-development] Fwd: [Exist-commits] SF.net SVN: exist:[11442] trunk/eXist/extensions

[Evgeny G. <gaz...@gm...>]

Hi!

I suggest to make a common webstart interface for eXist.
So we could have many client applications or applets instead
one "Admin client" as just  And every can create self own
applet/applicationfor eXist.

So I'd committed new webstart stuff, look webapp/webstart.
You can crate an any dynamic jnlp on xquery and use any
eXist's jar or any jar from core, optional,. extensions or user lib.
For this I had committed a new LibModule with functions
lib:get-lib() and lib:get-lib-info().
(You can rename one as you want).

As you can see this is extension, like fluent.
and eXist could be compiled without one.
So your suggestion is not correct, sorry.

This extension is a new kind of REST client for
manage documents stored in the eXist's DB on
native applications, like Open Office, MS Office,
or any another application (depend from your OS and
applications which you can use).

Just I'm working on client applet which will help
create/manage/edit/store docs in any desktop
application directly from browser via REST.

Idea is give users a HTML interface with links to any docs.
After clicking by so link user will get doc opened in
his desktop application and could edit one and store then.
After storing doc, applet will store doc back into server.
So user can open, edit, store any kind of
doc, stored in DB, directly from browser without webdavfs
or any kind of shares (like smb+davfs) and without using
download/upload files process (via html forms).
Think this can make IS based on eXist more friendly for
users, not for geeks or programmers, and adding unique
features.

---------
Evgeny

Re: [Exist-development] reposting some user group meeting tasks

[James F. <jam...@gm...>]

>> * testers should be on  #existdb IRC channel to facilitate comms and
>> ensure everyone is in sync

channel is #existdb

in sync basically means that people will typically go through a few
attempts feeding back information in real time ... posting bug reports
are good, but as the test cycle typically occurs over 4-5 days it can
help to have real time comms via irc to punch out the simple stuff.

As for release management ... I think the knowledge of what this
entails needs to be shared out e.g. perhaps Dannes would be willing to
mentor a volunteer(s) for a few release cycles ? Thoughts ?

J

Re: [Exist-development] reposting some user group meeting tasks

[Stefan M. <ste...@un...>]

Thanks for posting the tasks. And by the way thanks to everyone involved 
in the eXUG mtg on Friday.

> * that we nominate and communicate a release manager as 'hub' between
> testers and developers
>   

great idea.

> * testers should be on  #existdb IRC channel to facilitate comms and
> ensure everyone is in sync
>   

could arrange for that; on which network is the channel, btw? Haven't 
used IRC since the early 2000s, but I'll manage. Anyway, what do you 
mean by "in sync"? Is there a need to do the testing simultaneously? Or 
do you think in terms of having all the same version of eXist and the 
installer, such that things don't get mixed up with several revisions?

> * that results are captured as bugs in bug tracker (or is this too cumbersome?)
>   
No, I think this is feasible. I don't know how sf.net deals with bugs, 
but with bugzilla based bugtracking it is common practice to open a 
release bug entry and make it depend on a bug for each test that has to 
be performed. This way it is pretty simple to track when all tests are 
completed, that is, when all bugs, this release-specific bug depends on, 
are closed. I assume there would be some similar mechanism on sf. I like 
the fact that the process and the results are then well documented. I 
could imagine, that this reduces the strain on the main developers of 
eXist. If there were a release manager, he could open the necessary 
bugs. If not, the person doing the test, could do so. Then, only the 
release specific bug-report would have to be reported.

Stefan

[Exist-development] Changes lost in rev 11428

[Wolfgang M. <wol...@ex...>]

Hi Evgeny,

you rolled back some of my changes to the Lucene index in rev 11428.
It cost me 45 minutes to find out how to undo this.

Please check every commit twice before you press enter.

Wolfgang

[Exist-development] reposting some user group meeting tasks

[James F. <jam...@gm...>]

these are the items I captured from fridays usergroup;

I) next release will have the following testers to help out release
team (who will contact them):

      Henry (Dan's mate): windows vista

      Dan: windows xp

      Adam: opensolaris

      Jim: mac osx

      Lief: linux

      Stefan M: ubuntu

      Andy Bunce:windows

This test cycle e.g. will ask people to install, capture comments,
then iterate that may run over a day or so.
We need to manage this process to reduce churn as well as capture
things consistently. I would recommend
that testing consists of

* that we nominate and communicate a release manager as 'hub' between
testers and developers

* testers should be on  #existdb IRC channel to facilitate comms and
ensure everyone is in sync

* that results are captured as bugs in bug tracker (or is this too cumbersome?)


II) It was suggested to expose more of the team city build results ...
Lief to investigate public Team city page ?


III) I had various discussions about integrating a unit test suite ...
Jim (e.g. me) will follow up with integrating into trunk with
documentation

Re: [Exist-development] file module and DBA role

[Dannes W. <da...@ex...>]

Hi,

On Tue, Mar 9, 2010 at 7:48 PM, Evgeny Gazdovsky <gaz...@gm...>wrote:

> I'm suggest to change access to read/list functioons in file module
> for everyone instead DBA just.
> What do you think?
>

That is what I consider as a very bad idea. We changed it recently to the
status it is at this moment. The File module introduces a not-acceptable
security risk when regular users can use the functions.....

greets

Dannnes

-- 
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

[Exist-development] file module and DBA role

[Evgeny G. <gaz...@gm...>]

I'm suggest to change access to read/list functioons in file module
for everyone instead DBA just.
What do you think?

--------------
Evgeny

Re: [Exist-development] Recording OpenID users in the system collection

[Dmitriy S. <sha...@gm...>]

On Wed, Mar 3, 2010 at 6:30 PM, Loren Cahlander
<lor...@gm...>wrote:

>
> For our purposes, we would need a script that would add an entry into
> /system/openid/OpenIDUsers.xml if the user has not already been added to the
> system.
>

Can we use /system/OpenIDUsers.xml or /system/openid/users.xml (I prefer
last one)?


>
> <SessionLogs>
>

namespace should make it more readable <logs xmlns="
http://exist.org/security/session">

<?xml version="1.0" encoding="UTF-8"?>
<Logs xmlns="http://exist.org/security/session">
    <LogStartDateTime>2010-02-16T01:00:00.45+00:00</LogStartDateTime> <!--
why do we need this? it can be in log events-->
    <LogEndDateTime>2010-02-17T01:00:00.00+00:00</LogEndDateTime> <!-- &
this? -->
    <Session id="bla">
        <User id=http://john-q-public.myopenid.com>
            <Type>OpenID</Type>
        </user>
//            <SessionEvents> <!-- I don't feel comfotable with this, why we
can use /session/event
        <Event timestamp="">
             <Type>login</Type>
        </Event>


Well, we should disscuss this in detail at Prague.

-- 
Dmitriy Shabanov

Re: [Exist-development] Recording OpenID users in the system collection

[Dannes W. <da...@ex...>]

The code structure looks roughly like

try {
    broker = pool.get(user)

    XQuery xquery = broker.getXQueryService();

    XQueryContext context = xquery.newContext(AccessContext.XXXX);

    CompiledXQuery compiled = xquery.compile(context, source);

    Sequence result = xquery.execute(compiled, null, outputProperties);

....

} finally {
    pool.release(broker);
}

Code inspired by the RESTserver.java implementation.

cheers

Dannes

-- 
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

[Exist-development] Using chunked encoding in eXist

[Thomas W. <tho...@gm...>]

When we have a long running query and we produce the content incrementally,
it will be great to flush the content available so far down the wire ( so
the user will see the content as it comes ).  The solution is using "chunked
encoding". Bellow is a quote borrowed from
http://www.httpwatch.com/httpgallery/chunked/

"Chunked encoding is useful when a large amount of data is being returned to
the client and the total size of the response may not be known until the
request has been fully processed. An example of this is generating an HTML
table of results from a database query. If you wanted to use the *
Content-Length* header you would have to buffer the whole result set before
calculating the total content size. However, with chunked encoding you could
just write the data one row at a time and write a zero sized chunk when the
end of the query was reached."

Is there a way to implement "chunked encoding" in eXist?

Thomas

------

Thomas White

Mobile:+44 7711 922 966
Skype: thomaswhite
gTalk: thomas.0007
Linked-In:http://www.linkedin.com/in/thomaswhite0007
facebook: http://www.facebook.com/thomas.0007

Re: [Exist-development] Ant upgraded to 1.8.0 in Trunk...

[Dannes W. <da...@ex...>]

Ok,

I hesitiated to do it myself, but I guess it is okay :-)

We need to be sure that the build servers (our and some external) keep
working.....

D.

On Thu, Mar 4, 2010 at 1:07 AM, Andrzej Jan Taramina
<an...@ch...> wrote:
> Guys:
>
> I just updated Ant to the latest 1.8.0 release (which I have been using since early this year).  Clean builds and full
> test suite seem to be fine.
>
> This is in preparation for some packaging of standalone restore/backup distribution zips.
>
> I also included the ant-contrib.jar, which is very handy for doing if/then/else things in your ant scripts and other
> advanced techniques.
>
> Keep an eye out for any other problems.....with things like NetBeans, since I don't use it and can't test it.
>
>
> --
> Andrzej Taramina
> Chaeron Corporation: Enterprise System Solutions
> http://www.chaeron.com
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>



-- 
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

[Exist-development] Ant upgraded to 1.8.0 in Trunk...

[Andrzej J. T. <an...@ch...>]

Guys:

I just updated Ant to the latest 1.8.0 release (which I have been using since early this year).  Clean builds and full
test suite seem to be fine.

This is in preparation for some packaging of standalone restore/backup distribution zips.

I also included the ant-contrib.jar, which is very handy for doing if/then/else things in your ant scripts and other
advanced techniques.

Keep an eye out for any other problems.....with things like NetBeans, since I don't use it and can't test it.


-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

[Exist-development] Recording OpenID users in the system collection

[Loren C. <lor...@gm...>]

Hello Wolfgang and Dmitriy,

The implementation of OpenID in eXist is great.  I am working on adding the recording of the OpenID users and a general purpose session logger for eXist.  I am thinking that these two events should be performed with XQuery scripts.  The selection of the scripts should be configurable.

i.e. When an OpenID user is verified, the org.exist.security.openid.SessionAuthentication.java should check to see if a configuration entry org.exist.security.openid.verify_logging_script exists.  The entry can be either on the filesystem or in the database.  The script should be run as an XQuery and as the admin user.

For our purposes, we would need a script that would add an entry into /system/openid/OpenIDUsers.xml if the user has not already been added to the system.  The format of the file would be something like:


<?xml version="1.0" encoding="UTF-8"?>
<OpenIDUsers>
    <OpenIDUser>
        <id>http://john-q-public.myopenid.com</id>
        <CreatedDateTime>2010-02-16T19:20:30.45+00:00</CreatedDateTime>
        <attributes>
            <attribute>
                <name>fullname</name>
                <value>John Q. Public</value>
            </attribute>
            <attribute>
                <name>email</name>
                <value>joh...@ex...</value>
            </attribute>
            <attribute>
                <name>country</name>
                <value>US</value>
            </attribute>
            <attribute>
                <name>language</name>
                <value>EN</value>
            </attribute>
        </attributes>
        <groups>
            <group>guest</group>
        </groups>
    </OpenIDUser>
</OpenIDUsers>

There would also be a session log for logging all user (other than guest) accesses to the system.  When a user logs in through either OpenID, eXist internal or LDAP, then a configuration entry org.exist.security.session.logger_script would point to a script the would add an entry to a log file similar to:


<?xml version="1.0" encoding="UTF-8"?>
<SessionLogs>
    <SessionLogStartDateTime>2010-02-16T01:00:00.45+00:00</SessionLogStartDateTime>
    <SessionLogEndDateTime>2010-02-17T01:00:00.00+00:00</SessionLogEndDateTime>
    <SessionLog>
        <SessionID>00129485</SessionID>
        <SessionUser>
            <SessionUserType>OpenID</SessionUserType>
            <SessionUserID>http://john-q-public.myopenid.com</SessionUserID>
            <SessionEvents>
                <SessionEvent>
                    <SessionEventType>login</SessionEventType>
                    <SessionEventDateTime>2010-02-16T19:20:30.45+00:00</SessionEventDateTime>
                    <SessionEventMessage>User John Q. Public logged in from myopenid.com</SessionEventMessage>
                </SessionEvent>
                <SessionEvent>
                    <SessionEventType>timeout</SessionEventType>
                    <SessionEventDateTime>2010-02-16T19:21:00.45+00:00</SessionEventDateTime>
                    <SessionEventMessage>User has been inactive for 15 minutes.  Session has timed out.</SessionEventMessage>
                </SessionEvent>
            </SessionEvents>
        </SessionUser>
    </SessionLog>
    <SessionLog>
        <SessionID>00129486</SessionID>
        <SessionUser>
            <SessionUserType>OpenID</SessionUserType>
            <SessionUserID>http://john-q-public.myopenid.com</SessionUserID>
            <SessionEvents>
                <SessionEvent>
                    <SessionEventType>login</SessionEventType>
                    <SessionEventDateTime>2010-02-16T22:20:30.45+00:00</SessionEventDateTime>
                    <SessionEventMessage>User John Q. Public logged in from myopenid.com</SessionEventMessage>
                </SessionEvent>
                <SessionEvent>
                    <SessionEventType>logout</SessionEventType>
                    <SessionEventDateTime>2010-02-16T23:21:00.45+00:00</SessionEventDateTime>
                    <SessionEventMessage>User has logged out of the session.</SessionEventMessage>
                </SessionEvent>
            </SessionEvents>
        </SessionUser>
    </SessionLog>
</SessionLogs>

The current idea is to have the session logging entries in /system/log/{year}/SessionLog-{year}-{month}.xml, but it would be up to the XQuery script implementation as to where the log entries go (in the database, in the filesystem, on another server, etc.).

These things are necessary for proper security metrics for our site.  I just need a little help with a code snippet that I can add to org.exist.security.openid.SessionAuthentication.java to run an XQuery script from a reference as the admin user.


Thank you,

Loren

[Exist-development] Awarding of the prestigious Codestud award...

[Andrzej J. T. <an...@ch...>]

I am happy to announce that the prestigious award and designation of Codestud has been conferred on Wolfgang!

Wolfgang qualified for this special award through all his hard work on the new structural index, followed by an intense
week and a half of performance optimizations to ensure that the new index didn't have any negative performance impact on
production sites.

And after all that hard code slinging, he then fixed a long-standing range index bug dealing with in-memory fragments
within a few hours of receiving a test harness that replicated the problem.

Bravo Wolfgang!

Many thanks for all your ongoing work in helping to make eXist such a powerful, stable and useful tool.

You are now an official Codestud!  Wear the label proudly. ;-)

-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] error during the running of the benchmark

[Leif-Jöran O. <lj...@ex...>]

Loren, I have updated the imdi-snapshot data. So please download it 
again. It was due to a missing referenced file not affecting the 
benchmark so it could easily be removed. I don't know how this passed by 
unnoticed on the various systems I tested it on.

Sorry for the inconvenience,
Leif-Jöran

Den 2010-02-27 22:47, Loren Cahlander skrev:
> Hello folks,
>
> I received the following errors while trying to run the benchmark.  Does anyone know how to fix this?
>
> Thanks,
>
> Loren
>
>
> run-benchmark:
...
>       [java] org.xmldb.api.base.XMLDBException: java.io.FileNotFoundException:
/Users/lcahlander/Documents/workspace/exist-trunk/test/benchmark/data/imdi/Annotations-corpus.cfg 
(No such file or directory)

[Exist-development] error during the running of the benchmark

[Loren C. <lor...@gm...>]

Hello folks,

I received the following errors while trying to run the benchmark.  Does anyone know how to fix this?

Thanks,

Loren


run-benchmark:
    [mkdir] Created dir: /Users/lcahlander/Documents/workspace/exist-trunk/test/benchmark/temp
     [echo] Running test file: test/src/org/exist/performance/test.xml
     [java] ARG: -d test/benchmark
     [java] ARG: -f test/src/org/exist/performance/test.xml
     [java] ARG: -r
     [java] ARG: large-documents
     [java] ARG: ft-tests
     [java] ARG: imdi-tests
     [java] [Deprecated] Xalan: org.apache.xalan.res.XSLTErrorResources_en_US
     [java] Running setup ...
     [java] Setup done ...
     [java] Loading mime table from file /Users/lcahlander/Documents/workspace/exist-trunk/mime-types.xml
     [java] Running setup ...
     [java] Setup done ...
     [java] Tearing down ...
     [java] Done.
     [java] Running setup ...
     [java] Setup done ...
     [java] org.xmldb.api.base.XMLDBException: java.io.FileNotFoundException: /Users/lcahlander/Documents/workspace/exist-trunk/test/benchmark/data/imdi/Annotations-corpus.cfg (No such file or directory)
     [java] 	at org.exist.xmldb.LocalCollection.storeXMLResource(LocalCollection.java:764)
     [java] 	at org.exist.xmldb.LocalCollection.storeResource(LocalCollection.java:651)
     [java] 	at org.exist.xmldb.LocalCollection.storeResource(LocalCollection.java:642)
     [java] 	at org.exist.performance.actions.StoreFromFile.execute(StoreFromFile.java:97)
     [java] 	at org.exist.performance.ActionSequence.execute(ActionSequence.java:83)
     [java] 	at org.exist.performance.ActionThread.run(ActionThread.java:50)
     [java] 	at org.exist.performance.Group.run(Group.java:83)
     [java] 	at org.exist.performance.Runner.run(Runner.java:100)
     [java] 	at org.exist.performance.Main.main(Main.java:118)
     [java] Caused by: org.exist.EXistException: java.io.FileNotFoundException: /Users/lcahlander/Documents/workspace/exist-trunk/test/benchmark/data/imdi/Annotations-corpus.cfg (No such file or directory)
     [java] 	at org.exist.collections.Collection$4.run(Collection.java:1140)
     [java] 	at org.exist.collections.Collection.validateXMLResourceInternal(Collection.java:1217)
     [java] 	at org.exist.collections.Collection.validateXMLResource(Collection.java:1130)
     [java] 	at org.exist.xmldb.LocalCollection.storeXMLResource(LocalCollection.java:734)
     [java] 	... 8 more
     [java] Caused by: java.io.FileNotFoundException: /Users/lcahlander/Documents/workspace/exist-trunk/test/benchmark/data/imdi/Annotations-corpus.cfg (No such file or directory)
     [java] 	at java.io.FileInputStream.open(Native Method)
     [java] 	at java.io.FileInputStream.<init>(FileInputStream.java:106)
     [java] 	at java.io.FileInputStream.<init>(FileInputStream.java:66)
     [java] 	at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:70)
     [java] 	at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:161)
     [java] 	at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLEntityManager.startEntity(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLEntityManager.startEntity(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDTDScannerImpl.startPE(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDTDScannerImpl.skipSeparator(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDTDScannerImpl.scanDecls(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDTDScannerImpl.scanDTDInternalSubset(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDocumentScannerImpl$DTDDispatcher.dispatch(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
     [java] 	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
     [java] 	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
     [java] 	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
     [java] 	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
     [java] 	at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
     [java] 	at org.exist.collections.Collection$4.run(Collection.java:1136)
     [java] 	... 11 more
     [java] [B@18182506org.exist.EXistException: java.io.FileNotFoundException: /Users/lcahlander/Documents/workspace/exist-trunk/test/benchmark/data/imdi/Annotations-corpus.cfg (No such file or directory)
     [java] 	at org.exist.collections.Collection$4.run(Collection.java:1140)
     [java] 	at org.exist.collections.Collection.validateXMLResourceInternal(Collection.java:1217)
     [java] 	at org.exist.collections.Collection.validateXMLResource(Collection.java:1130)
     [java] 	at org.exist.xmldb.LocalCollection.storeXMLResource(LocalCollection.java:734)
     [java] 	at org.exist.xmldb.LocalCollection.storeResource(LocalCollection.java:651)
     [java] 	at org.exist.xmldb.LocalCollection.storeResource(LocalCollection.java:642)
     [java] 	at org.exist.performance.actions.StoreFromFile.execute(StoreFromFile.java:97)
     [java] 	at org.exist.performance.ActionSequence.execute(ActionSequence.java:83)
     [java] 	at org.exist.performance.ActionThread.run(ActionThread.java:50)
     [java] 	at org.exist.performance.Group.run(Group.java:83)
     [java] 	at org.exist.performance.Runner.run(Runner.java:100)
     [java] 	at org.exist.performance.Main.main(Main.java:118)
     [java] Caused by: java.io.FileNotFoundException: /Users/lcahlander/Documents/workspace/exist-trunk/test/benchmark/data/imdi/Annotations-corpus.cfg (No such file or directory)
     [java] 	at java.io.FileInputStream.open(Native Method)
     [java] 	at java.io.FileInputStream.<init>(FileInputStream.java:106)
     [java] 	at java.io.FileInputStream.<init>(FileInputStream.java:66)
     [java] 	at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:70)
     [java] 	at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:161)
     [java] 	at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLEntityManager.startEntity(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLEntityManager.startEntity(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDTDScannerImpl.startPE(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDTDScannerImpl.skipSeparator(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDTDScannerImpl.scanDecls(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDTDScannerImpl.scanDTDInternalSubset(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDocumentScannerImpl$DTDDispatcher.dispatch(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
     [java] 	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
     [java] 	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
     [java] 	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
     [java] 	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
     [java] 	at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
     [java] 	at org.exist.collections.Collection$4.run(Collection.java:1136)
     [java] 	... 11 more
     [java] Caused by: org.exist.EXistException: java.io.FileNotFoundException: /Users/lcahlander/Documents/workspace/exist-trunk/test/benchmark/data/imdi/Annotations-corpus.cfg (No such file or directory)
     [java] 	at org.exist.collections.Collection$4.run(Collection.java:1140)
     [java] 	at org.exist.collections.Collection.validateXMLResourceInternal(Collection.java:1217)
     [java] 	at org.exist.collections.Collection.validateXMLResource(Collection.java:1130)
     [java] 	at org.exist.xmldb.LocalCollection.storeXMLResource(LocalCollection.java:734)
     [java] 	at org.exist.xmldb.LocalCollection.storeResource(LocalCollection.java:651)
     [java] 	at org.exist.xmldb.LocalCollection.storeResource(LocalCollection.java:642)
     [java] 	at org.exist.performance.actions.StoreFromFile.execute(StoreFromFile.java:97)
     [java] 	at org.exist.performance.ActionSequence.execute(ActionSequence.java:83)
     [java] 	at org.exist.performance.ActionThread.run(ActionThread.java:50)
     [java] 	at org.exist.performance.Group.run(Group.java:83)
     [java] 	at org.exist.performance.Runner.run(Runner.java:100)
     [java] 	at org.exist.performance.Main.main(Main.java:118)
     [java] Caused by: java.io.FileNotFoundException: /Users/lcahlander/Documents/workspace/exist-trunk/test/benchmark/data/imdi/Annotations-corpus.cfg (No such file or directory)
     [java] 	at java.io.FileInputStream.open(Native Method)
     [java] 	at java.io.FileInputStream.<init>(FileInputStream.java:106)
     [java] 	at java.io.FileInputStream.<init>(FileInputStream.java:66)
     [java] 	at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:70)
     [java] 	at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:161)
     [java] 	at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLEntityManager.startEntity(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLEntityManager.startEntity(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDTDScannerImpl.startPE(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDTDScannerImpl.skipSeparator(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDTDScannerImpl.scanDecls(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDTDScannerImpl.scanDTDInternalSubset(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDocumentScannerImpl$DTDDispatcher.dispatch(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
     [java] 	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
     [java] 	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
     [java] 	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
     [java] 	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
     [java] 	at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
     [java] 	at org.exist.collections.Collection$4.run(Collection.java:1136)
     [java] 	... 11 more
     [java] org.exist.EXistException: collection /db/test not found
     [java] 	at org.exist.performance.actions.XQuery.execute(XQuery.java:77)
     [java] 	at org.exist.performance.ActionSequence.execute(ActionSequence.java:83)
     [java] 	at org.exist.performance.ActionThread.run(ActionThread.java:50)
     [java] 	at java.lang.Thread.run(Thread.java:637)

benchmark:

BUILD SUCCESSFUL
Total time: 66 minutes 36 seconds
x-160-94-88-100:exist-trunk lcahlander$ 

Re: [Exist-development] Sorting problem

[Adam R. <ad...@ex...>]

> We need a stable one!

I was not disagreeing with that. But we dont want to loose performance
if avoidable, sorting has always been a slow operation.

After checking the source code it would seem that the JDK 6
implementation of Arrays.sort is based on a quick sort as well.


> On Mon, Feb 22, 2010 at 3:05 PM, Adam Retter <ad...@ex...> wrote:
>>
>> How does the performance of Arrays.sort compare to Quicksort?
>>
>> If it is not favourable, perhaps we need to identify a better
>> alternative algorithm?
>>
>> On 21 February 2010 08:05, Dmitriy Shabanov <sha...@gm...> wrote:
>> > Hello,
>> >
>> > We use Quicksort algorithm for sorting. It's not stable
>> > (
>> > http://en.wikipedia.org/wiki/Sorting_algorithm#Comparison_of_algorithms )
>> >
>> > Example:
>> >
>> > http://sourceforge.net/tracker/?func=detail&aid=2903818&group_id=17691&atid=117691
>> >
>> > I think better to switch to Arrays.sort .
>> >
>> > --
>> > Cheers,
>> >
>> > Dmitriy Shabanov
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Download Intel&#174; Parallel Studio Eval
>> > Try the new software tools for yourself. Speed compiling, find bugs
>> > proactively, and fine-tune applications for parallel performance.
>> > See why Intel Parallel Studio got high marks during beta.
>> > http://p.sf.net/sfu/intel-sw-dev
>> > _______________________________________________
>> > Exist-development mailing list
>> > Exi...@li...
>> > https://lists.sourceforge.net/lists/listinfo/exist-development
>> >
>> >
>>
>>
>>
>> --
>> Adam Retter
>>
>> eXist Developer
>> { United Kingdom }
>> ad...@ex...
>> irc://irc.freenode.net/existdb
>
>
>
> --
> Dmitriy Shabanov
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] Sorting problem

[Dmitriy S. <sha...@gm...>]

We need a stable one!

On Mon, Feb 22, 2010 at 3:05 PM, Adam Retter <ad...@ex...> wrote:

> How does the performance of Arrays.sort compare to Quicksort?
>
> If it is not favourable, perhaps we need to identify a better
> alternative algorithm?
>
> On 21 February 2010 08:05, Dmitriy Shabanov <sha...@gm...> wrote:
> > Hello,
> >
> > We use Quicksort algorithm for sorting. It's not stable
> > (
> http://en.wikipedia.org/wiki/Sorting_algorithm#Comparison_of_algorithms )
> >
> > Example:
> >
> http://sourceforge.net/tracker/?func=detail&aid=2903818&group_id=17691&atid=117691
> >
> > I think better to switch to Arrays.sort .
> >
> > --
> > Cheers,
> >
> > Dmitriy Shabanov
> >
> >
> ------------------------------------------------------------------------------
> > Download Intel&#174; Parallel Studio Eval
> > Try the new software tools for yourself. Speed compiling, find bugs
> > proactively, and fine-tune applications for parallel performance.
> > See why Intel Parallel Studio got high marks during beta.
> > http://p.sf.net/sfu/intel-sw-dev
> > _______________________________________________
> > Exist-development mailing list
> > Exi...@li...
> > https://lists.sourceforge.net/lists/listinfo/exist-development
> >
> >
>
>
>
> --
> Adam Retter
>
> eXist Developer
> { United Kingdom }
> ad...@ex...
> irc://irc.freenode.net/existdb
>



-- 
Dmitriy Shabanov

Re: [Exist-development] Sorting problem

[Adam R. <ad...@ex...>]

How does the performance of Arrays.sort compare to Quicksort?

If it is not favourable, perhaps we need to identify a better
alternative algorithm?

On 21 February 2010 08:05, Dmitriy Shabanov <sha...@gm...> wrote:
> Hello,
>
> We use Quicksort algorithm for sorting. It's not stable
> ( http://en.wikipedia.org/wiki/Sorting_algorithm#Comparison_of_algorithms )
>
> Example:
> http://sourceforge.net/tracker/?func=detail&aid=2903818&group_id=17691&atid=117691
>
> I think better to switch to Arrays.sort .
>
> --
> Cheers,
>
> Dmitriy Shabanov
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] Memory usage

[Wolfgang M. <wol...@ex...>]

> Should I decrease the 48M despite the fact that the JVM is allocated 512M ?

Mmmh, no. Looks like it's not a caching issue then. We have to see
what happens once I committed my other fixes.

Wolfgang

Re: [Exist-development] Memory usage

[Pierrick B. <pie...@fr...>]

Hi again,

Le 21/02/2010 10:28, Wolfgang Meier a écrit :

> Sorry for that. There are some issues with the cache management, which
> were revealed by the new indexing code. I'm working on this. Trying
> more conservative cache settings should help.

My current config for my tests is the default one :

<db-connection cacheSize="48M" collectionCache="24M" database="native"
         files="webapp/WEB-INF/data" pageSize="4096" nodesBuffer="-1" 
cacheShrinkThreshold="10000">

Should I decrease the 48M despite the fact that the JVM is allocated 512M ?

Cheers,

p.b.

Re: [Exist-development] Memory usage

[Wolfgang M. <wol...@ex...>]

Hi Pierrick,

> Since the last commits about indexing, I can't successfully run the
> spatial tests.

Sorry for that. There are some issues with the cache management, which
were revealed by the new indexing code. I'm working on this. Trying
more conservative cache settings should help.

Wolfgang