exist-development — eXist Developer's List

Re: [Exist-development] jetty 7

[Dmitriy S. <sha...@gm...>]

On Tue, 2010-01-12 at 19:25 +0100, Dannes Wessels wrote:
> Dmitriy,
> 
> On 12 Jan 2010, at 17:54 , Dmitriy Shabanov wrote:
> 
> > Are we ready for jetty 7?
> > 
> > http://exist.svn.sourceforge.net/viewvc/exist/branches/shabanovd/jetty7/ 
> 
> What is the benefit for upgrading? I see 7 is considered as stable,
> that is good. Servlet 2.5 is supported..... so compatible with
> tomcat6? Pity is the relation with eclipse.org (personal opinion :) )
> 

primary:

 * http://wiki.eclipse.org/Jetty/Feature/JAAS 
 * http://wiki.eclipse.org/Jetty/Feature/Continuations 
	(asynchronous servlets (updated continuations))
 * faster (personal feeling, but it looks very fast)

 * preparation for servlets 3.0 
 * step forward =)

-- 
Cheers,

Dmitriy Shabanov

Re: [Exist-development] jetty 7

[Dannes W. <di...@ex...>]

Dmitriy,

On 12 Jan 2010, at 17:54 , Dmitriy Shabanov wrote:

> Are we ready for jetty 7?
> 
> http://exist.svn.sourceforge.net/viewvc/exist/branches/shabanovd/jetty7/ 

What is the benefit for upgrading? I see 7 is considered as stable, that is good. Servlet 2.5 is supported..... so compatible with tomcat6? Pity is the relation with eclipse.org (personal opinion :) )

Kind regards

Dannes

--
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

[Exist-development] jetty 7

[Dmitriy S. <sha...@gm...>]

Hello,

Are we ready for jetty 7?

http://exist.svn.sourceforge.net/viewvc/exist/branches/shabanovd/jetty7/ 

-- 
Cheers,

Dmitriy Shabanov

Re: [Exist-development] The authentication service

[Dmitriy S. <sha...@gm...>]

On Fri, 2010-01-08 at 18:43 +0100, Dannes Wessels wrote:
> hi,
> 
> On 8 Jan 2010, at 17:35 , Dmitriy Shabanov wrote:
> 
> > > what will not work anymore? 
> > 
> > jetty's authentication service
> > $url/j_security_check?j_username=.&ej_password=.
> 
> 
> I don't recognize it... is this part of jetty??? probably not......

yes, it's jetty internal staff.

> I think I just do not understand what you are referring to......

so, that should mean that you don't use it ;)

-- 
Cheers,

Dmitriy Shabanov

Re: [Exist-development] The authentication service

[Dannes W. <di...@ex...>]

hi,

On 8 Jan 2010, at 17:35 , Dmitriy Shabanov wrote:

>> what will not work anymore? 
> 
> jetty's authentication service
> $url/j_security_check?j_username=.&ej_password=.


I don't recognize it... is this part of jetty??? probably not......

I think I just do not understand what you are referring to......

Kind regards

Dannes

--
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

Re: [Exist-development] The authentication service

[Dmitriy S. <sha...@gm...>]

On Fri, 2010-01-08 at 16:44 +0100, Dannes Wessels wrote:
> Heho,.
> 
> passwords in an URL is a bad design... should probably be done via
> POST (urlencoded or multipart) for bia Basic Authentication (or like
> technology_ ?

yes, it'll process POST too.

> what are the immediate implication of this change? 

1. make API simple:

<form action="$url/j_security_check" method="post">
 Username: <input name="exist_username" type="text"/><br>
 Password: <input name="exist_password" type="password"/><br>
 <input type="submit"/>
</form>

2. "secure urls" - only some users have access to it.
3. resolve issue with long sessions
4. integrate pluggable security modules (openid)
5. login-form (url) + errors-form (url)
6. ...

> what will not work anymore? 

jetty's authentication service
$url/j_security_check?j_username=.&ej_password=.

-- 
Cheers,

Dmitriy Shabanov

PS most all can be found @
http://exist.svn.sourceforge.net/viewvc/exist/branches/shabanovd/access_control/

Re: [Exist-development] The authentication service

[Dannes W. <di...@ex...>]

Heho,.

passwords in an URL is a bad design... should probably be done via POST
(urlencoded or multipart) for bia Basic Authentication (or like technology_
?

what are the immediate implication of this change? what will not work
anymore?

regards

Dannes

On Fri, Jan 8, 2010 at 4:07 PM, Dmitriy Shabanov <sha...@gm...>wrote:

> Hello,
>
> I want to replace (this sunday) jetty's authentication service by
> eXist's one.
>
> It will handle authentication by
> $url/j_security_check?eXist_username=.&eXist_password=. & redirect back
> to $url with user as eXist_username if authenticated successfully or
> "guest" if failed.
>
> Later, OpenID add-on will handle $url/j_security_check?eXist_openID=. or
> $url/j_security_check?eXist_openIDprovider=.
>
> After that, custom login-form url.
>
> Any objections?
>
> --
> Cheers,
>
> Dmitriy Shabanov
>
>
>
> ------------------------------------------------------------------------------
> This SF.Net email is sponsored by the Verizon Developer Community
> Take advantage of Verizon's best-in-class app development support
> A streamlined, 14 day to market process makes app distribution fast and
> easy
> Join now and get one step closer to millions of Verizon customers
> http://p.sf.net/sfu/verizon-dev2dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>



-- 
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

[Exist-development] The authentication service

[Dmitriy S. <sha...@gm...>]

Hello,

I want to replace (this sunday) jetty's authentication service by
eXist's one. 

It will handle authentication by
$url/j_security_check?eXist_username=.&eXist_password=. & redirect back
to $url with user as eXist_username if authenticated successfully or
"guest" if failed.

Later, OpenID add-on will handle $url/j_security_check?eXist_openID=. or
$url/j_security_check?eXist_openIDprovider=.

After that, custom login-form url.

Any objections?

-- 
Cheers,

Dmitriy Shabanov

Re: [Exist-development] Licensing for XQuery grammar (ljo branch)

[Dmitriy S. <sha...@gm...>]

On Wed, 2010-01-06 at 21:42 +0100, Leif-Jöran Olsson wrote:
> 3> Another option may be to ask the copyright holder(s) on the
> xqgrammar
> > software (Nikolay Ognyanov, et al.?) to release their code under a
> dual
> > license including LGPL 2.1.
> 
> We could always start with investigating 3 if noone objects.

I'm in contact with Nikolay Ognyanov & we did exchange some ideas. I do
believe that license will not be big issue (other world will be
resolved). All we need is tree parse to prove that it's right way.

By the way, I vote for LGPL v3, because we do have Apache License
Version 2.0 staff (example: commons-*.jar & so on).
 
-- 
Cheers,

Dmitriy Shabanov

Re: [Exist-development] Licensing for XQuery grammar (ljo branch)

[Leif-Jöran O. <lj...@ex...>]

Den 2010-01-06 21:17, Wolfgang Meier skrev:
>> I'm concerned about some files which were recently committed to the ljo
>> branch:
>> http://exist.svn.sourceforge.net/exist/?rev=10949&view=rev
>
> This is an experimental branch and personally I'm not yet sure what
> benefits and risks it would have to replace our own grammar. It is a
> major change which has to be considered carefully.

Yes I have been turning this question back and forwards, outside and in 
for the last two days since Dmitriy added the xpath grammar part and 
wanted us to use the full xquery version.

> The main problem we
> faced during the past year was to migrate our own lexer/grammar from
> ANTLR 2 to 3.   Among other things, ANTLR 3 would allow us to provide
> better error messages (e.g. to finally fix the annoying "expected xyz
> found null" messages). Obviously Nikolay managed to use ANTLR 3 to
> cover all of XQuery. That's an advantage. But instead of reusing his
> grammar, we could also try to learn from what he did.

I have been leaning more and more towards this today while working up 
some steam for getting on with the tree parser for this xqgrammar 
lexer/parser. The antlr3 inmaturity has bitten us hardest in the tree 
parser and that is still where most of our needed action is located. The 
general idea and direction of our own rewrite and Nikolaj's is not that 
different.

> Anyway, the grammar alone doesn't help us much: we would need to
> extend it to generate an abstract syntax tree as input for our own
> tree parser (which then generates the executable XQuery expression
> tree).

> We cannot just reuse the code: we need to extend and annotate
> it. This makes the license issue more problematic.

I totally agree. This will get quite messy with our needed extensions.

> Well, I guess we should just get into contact with Nikolay (if this
> hasn't happened already) and ask him about his opinion.

Agreed.

Leif-Jöran

Re: [Exist-development] Licensing for XQuery grammar (ljo branch)

[Leif-Jöran O. <lj...@ex...>]

Hi Luigi,
I agree with your analysis. This has to be discussed. I numbered the 
options below for simpler reference.

Den 2010-01-06 20:32, fi...@us... skrev:
> I'm concerned about some files which were recently committed to the ljo
> branch:
> http://exist.svn.sourceforge.net/exist/?rev=10949&view=rev
>
> The files were released under the Apache 2.0 license, which is not compatible
> with the LGPL v2 (or 2.1). The rest of the code is licensed under LGPL v2 (or
> later).
>
> I am not a lawyer, so this is not a legal analysis, but my impression is that

1> changing the license for eXist to LGPL v3 (or later) is an option 
because v3
> is compatible with Apache 2.0.

2> Or, a more difficult approach would be to
> release eXist-ljo under LGPLv2.1 or later, and eXist+ljo under LGPL v3 or
> later.

3> Another option may be to ask the copyright holder(s) on the xqgrammar
> software (Nikolay Ognyanov, et al.?) to release their code under a dual
> license including LGPL 2.1.

We could always start with investigating 3 if noone objects.

> And I guess yet another option would be to find or write another compatible
> xqgrammar.

Yes, I was on my way in the original antlr3 branch.

Cheers,
Leif-Jöran

Re: [Exist-development] Licensing for XQuery grammar (ljo branch)

[Wolfgang M. <wol...@ex...>]

> I'm concerned about some files which were recently committed to the ljo
> branch:
> http://exist.svn.sourceforge.net/exist/?rev=10949&view=rev

This is an experimental branch and personally I'm not yet sure what
benefits and risks it would have to replace our own grammar. It is a
major change which has to be considered carefully. The main problem we
faced during the past year was to migrate our own lexer/grammar from
ANTLR 2 to 3. Among other things, ANTLR 3 would allow us to provide
better error messages (e.g. to finally fix the annoying "expected xyz
found null" messages). Obviously Nikolay managed to use ANTLR 3 to
cover all of XQuery. That's an advantage. But instead of reusing his
grammar, we could also try to learn from what he did.

Anyway, the grammar alone doesn't help us much: we would need to
extend it to generate an abstract syntax tree as input for our own
tree parser (which then generates the executable XQuery expression
tree). We cannot just reuse the code: we need to extend and annotate
it. This makes the license issue more problematic.

Well, I guess we should just get into contact with Nikolay (if this
hasn't happened already) and ask him about his opinion.

Wolfgang

[Exist-development] Licensing for XQuery grammar (ljo branch)

[<fi...@us...>]

I'm concerned about some files which were recently committed to the ljo 
branch:
http://exist.svn.sourceforge.net/exist/?rev=10949&view=rev

The files were released under the Apache 2.0 license, which is not compatible 
with the LGPL v2 (or 2.1). The rest of the code is licensed under LGPL v2 (or 
later).

I am not a lawyer, so this is not a legal analysis, but my impression is that 
changing the license for eXist to LGPL v3 (or later) is an option because v3 
is compatible with Apache 2.0. Or, a more difficult approach would be to 
release eXist-ljo under LGPLv2.1 or later, and eXist+ljo under LGPL v3 or 
later. Another option may be to ask the copyright holder(s) on the xqgrammar 
software (Nikolay Ognyanov, et al.?) to release their code under a dual 
license including LGPL 2.1.

And I guess yet another option would be to find or write another compatible 
xqgrammar.

Does this make sense?
Luigi

Re: [Exist-development] Java Based Function Modules

[Adam R. <ad...@ex...>]

2010/1/5 Loren Cahlander <lor...@gm...>:
> Hello Wolfgang,
>
> I think that we should create a strategy to move the java based function module documentation out into XQDoc files.  I also think that the text within the description tag in XQDoc be allowed to contain DocBook style tags to format the text within the descriptions.

Sounds sensible

> This is now a lower priority, but should be discussed.
>
> Loren
>
> On Jun 10, 2009, at 03:43 PM, Wolfgang wrote:
>
>> Hi Loren,
>>
>> I agree with your suggestions, but would like to make an addition. We were actually thinking about moving away from embedding function documentation within the Java code. Some functions can be pretty hard to explain, but writing longer paragraphs in a Java string is just not possible.
>>
>> Ideally I would still like to keep the documentation together with the documented module, but as a separate XML document. Maybe a good compromise could be: we extend the current FunctionSignature and SequenceType to include variable names as suggested by Loren.
>>
>> However, as an alternative to providing a documentation string in the Java class, the developer can also specify an id which links into an XML document residing in the same package as the module. The XML document contains a list of function descriptors with a documentation element whose content is based on the docbook schema. The parameters could be described in the descriptor as well.
>>
>> The documentation generator could then combine the signature from the class with the description in the XML to create a self-contained output document.
>>
>> What do you think?
>>
>> Wolfgang
>
>
> ------------------------------------------------------------------------------
> This SF.Net email is sponsored by the Verizon Developer Community
> Take advantage of Verizon's best-in-class app development support
> A streamlined, 14 day to market process makes app distribution fast and easy
> Join now and get one step closer to millions of Verizon customers
> http://p.sf.net/sfu/verizon-dev2dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

[Exist-development] Java Based Function Modules

[Loren C. <lor...@gm...>]

Hello Wolfgang,

I think that we should create a strategy to move the java based function module documentation out into XQDoc files.  I also think that the text within the description tag in XQDoc be allowed to contain DocBook style tags to format the text within the descriptions.

This is now a lower priority, but should be discussed.

Loren

On Jun 10, 2009, at 03:43 PM, Wolfgang wrote:

> Hi Loren,
> 
> I agree with your suggestions, but would like to make an addition. We were actually thinking about moving away from embedding function documentation within the Java code. Some functions can be pretty hard to explain, but writing longer paragraphs in a Java string is just not possible.
> 
> Ideally I would still like to keep the documentation together with the documented module, but as a separate XML document. Maybe a good compromise could be: we extend the current FunctionSignature and SequenceType to include variable names as suggested by Loren.
> 
> However, as an alternative to providing a documentation string in the Java class, the developer can also specify an id which links into an XML document residing in the same package as the module. The XML document contains a list of function descriptors with a documentation element whose content is based on the docbook schema. The parameters could be described in the descriptor as well.
> 
> The documentation generator could then combine the signature from the class with the description in the XML to create a self-contained output document.
> 
> What do you think?
> 
> Wolfgang

Re: [Exist-development] [Exist-builds] eXist REST

[Adam R. <ad...@ex...>]

2009/12/22 Lukas Lewandowski <luk...@un...>:
> Hi eXist team,
>
> I'm new in web services and i'm interested especially in RESTful WS.
>
> eXist offers REST, XML-RPC and SOAP as possible HTTP layer. Which of
> them is the most used within eXist and it's HTTP users? Are RESTful WS
> becoming a competitive to SOAP WS?
>
> Since last year, October 2008, a JSR-311 Java specification - JAX-RS -
> occurred.  JAX-RS specifies the way RESTful WS have to be in Java. Did
> you considered this specification or it's reference implementation
> Jersey for eXist?

Lukas, I am somewhat familiar with Jersey outside of an eXist context.
At this years XML Summer School I attended a presentation by Marc
Hadley and managed to also have a chat with him about Jersey and how
we might be able to integrate it into eXist.

I previously undertook a piece of work which is in the
tools/SOAPServer folder of eXist, which allows you to automagically
serve up individual XQuery functions from a function module as SOAP
Web Services. It has always been on my mind to develop this work
further and add support for REST as well. After talking with Marc I
did a bit of research and discovered Apache CXF, this single framework
offers both SOAP and REST support and it was my intention to integrate
this with eXist as a replacement for the SOAP Server. This would allow
us to easily serve up XQuery functions as both SOAP and REST Web
Services.

Unfortunately I have not had a great deal of time, and really only
started looking through the Apache CXF code to understand where I
needed to place the hooks.


Is this something you are interested in?

p.s. the exist-development list, is a better forum for this than the
exist-builds list.



> Thank you in advance for answering my questions.
>
> Best regards & merry Xmas
> Lukas
>
>
> Lukas Lewandowski
> Distributed Systems Group
> Universität Konstanz
>
>
> ------------------------------------------------------------------------------
> This SF.Net email is sponsored by the Verizon Developer Community
> Take advantage of Verizon's best-in-class app development support
> A streamlined, 14 day to market process makes app distribution fast and easy
> Join now and get one step closer to millions of Verizon customers
> http://p.sf.net/sfu/verizon-dev2dev
> _______________________________________________
> Exist-builds mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-builds
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] Standalone server

[Wolfgang M. <wol...@ex...>]

P.S.: for demonstration, to install the admin and sandbox apps into
the standalone server, you can do the following:

* start the "full" eXist with bin/startup.sh
* log into the admin web app and go to the "Install Tools" panel,
install the admin and sandbox app there
* stop eXist and launch the standalone server with bin/server.sh
* browse to

http://localhost:8088/admin/admin.xql

Wolfgang

[Exist-development] Standalone server

[Wolfgang M. <wol...@ex...>]

Hi,

as promised, I changed the way in which the minimal, "standalone"
server is configured. Instead of hardcoding the configuration in Java,
we now use a special Jetty configuration file in
tools/jetty/etc/standalone.xml, which creates a plain, minimal web app
with just REST, XMLRPC, WebDAV and Atom, as well as support for
XQueryURLRewrite (see config in
tools/jetty/etc/standalone/WEB-INF/controller-config.xml).

This setup is easier to customize. The new configuration works as
before: all URLs starting with /db will be passed to the REST servlet,
while /xmlrpc, /webdav and /atom are handled by the corresponding
servlets. Nothing has changed in this respect. However, all other URLs
will be served from the db collection /db/www by default (see
controller-config.xml).

Calling bin/server.sh will now launch this new configuration instead
of the old Java class StandaloneServer. The same applies to all unit
tests which rely on a running server. I hope this will fix some of the
issues we had with the test suite recently. I'm currently checking the
test suite on various machines. Please report if it doesn't work for
you.

TODO: update the documentation.

Wolfgang

Re: [Exist-development] Early version of JQuery Tree Editor Admin Tool

[Wolfgang M. <wol...@ex...>]

Hi Dan and Alex,

> I just wanted to give the dev team a quick update on the new eXist admin
> tools.

This is great!

I never really worked with jquery before, but this seems to be a
reason to consider it. The Javascript code looks very clean and
understandable.

Wolfgang

Re: [Exist-development] Early version of JQuery Tree Editor Admin Tool

[Dmitriy S. <sha...@gm...>]

I like it!

-- 
Cheers,

Dmitriy Shabanov

On Sat, 2009-12-19 at 11:15 -0600, Dan McCreary wrote:
> Hello all,
> 
> I just wanted to give the dev team a quick update on the new eXist
> admin tools.
> 
> Alex B. (soon to be back in London) and I have been working on a
> drag-and-drop version of the eXist admin tool that does basic
> operations on collections.
> 
> The first version is just a proof-of-concept that presents the users
> with two "tree views" of the eXist collection tree.  On the left is a
> source and on the right is a destination.
> 
> Here is an example:
> 
> http://demo.syntactica.com/exist/rest/db/cust/exist-admin/apps/treeeditor-v.05/index.xq
> 
> These will not "really" work on our demo server unless you have admin
> rights.
> 
> We have the basic move (default), copy (hold the control key down),
> create, delete and rename working for both collections and resources.
> Note you have to RIGHT CLICK over the node to see the operations.
> 
> If you are interested in seeing the source here is the actual JQuery
> controller file:
> 
> http://demo.syntactica.com/exist/rest/db/cust/exist-admin/apps/treeeditor-v.05/js/controller.js
> 
> For each operation we need a simple RESTful action script.
> 
> Let us know what you think.  Anyone that knows JavaScript and JQuery
> and would like to help please let us know.
> 
>   - Dan and Alex
> ------------------------------------------------------------------------------
> This SF.Net email is sponsored by the Verizon Developer Community
> Take advantage of Verizon's best-in-class app development support
> A streamlined, 14 day to market process makes app distribution fast and easy
> Join now and get one step closer to millions of Verizon customers
> http://p.sf.net/sfu/verizon-dev2dev 
> _______________________________________________ Exist-development mailing list Exi...@li... https://lists.sourceforge.net/lists/listinfo/exist-development

[Exist-development] Early version of JQuery Tree Editor Admin Tool

[Dan M. <dan...@gm...>]

Hello all,

I just wanted to give the dev team a quick update on the new eXist admin
tools.

Alex B. (soon to be back in London) and I have been working on a
drag-and-drop version of the eXist admin tool that does basic operations on
collections.

The first version is just a proof-of-concept that presents the users with
two "tree views" of the eXist collection tree.  On the left is a source and
on the right is a destination.

Here is an example:

http://demo.syntactica.com/exist/rest/db/cust/exist-admin/apps/treeeditor-v.05/index.xq

These will not "really" work on our demo server unless you have admin
rights.

We have the basic move (default), copy (hold the control key down), create,
delete and rename working for both collections and resources.  Note you have
to RIGHT CLICK over the node to see the operations.

If you are interested in seeing the source here is the actual JQuery
controller file:

http://demo.syntactica.com/exist/rest/db/cust/exist-admin/apps/treeeditor-v.05/js/controller.js

For each operation we need a simple RESTful action script.

Let us know what you think.  Anyone that knows JavaScript and JQuery and
would like to help please let us know.

  - Dan and Alex

Re: [Exist-development] servlet.jar

[Dmitriy S. <sha...@gm...>]

On Thu, 2009-12-17 at 09:42 +0100, Wolfgang Meier wrote:
> I understand the build works in eclipse. BUT I'm not using eclipse.
> I'm always building and launching eXist from the shell. Why? I want to
> make sure that eXist can be used by the average user, who may not have
> a Java IDE. The Ant build is the baseline. You should not rely on
> eclipse for testing.

I want a table (aka plan) "things to work". Please, add missing one.

 - tests = TC do cover all platforms, may be we need TC on different
platforms? (windows xp, mac, debian, ???)
 - bin/startup.sh = to run with out errors (rest should be covered by
tests)
 - bin/shutdown.sh = should stop db

what else?

-- 
Cheers,

Dmitriy Shabanov

Re: [Exist-development] servlet.jar

[Wolfgang M. <wol...@ex...>]

Dmitryi,

right now eXist does not start up properly anymore (due to missing jsp
dependencies) and it has been like that for more than 36 hours. This
is not acceptable! If you break the trunk, you should try to repair it
as soon as you can and stop all other work.

I understand the build works in eclipse. BUT I'm not using eclipse.
I'm always building and launching eXist from the shell. Why? I want to
make sure that eXist can be used by the average user, who may not have
a Java IDE. The Ant build is the baseline. You should not rely on
eclipse for testing.

Wolfgang

Re: [Exist-development] optimizing positional predicates for fast paging

[Wolfgang M. <wol...@ex...>]

> A great topic!  I'd certainly welcome optimizations like this.  I
> wonder:  If a FLWOR expression has an 'order by' clause, is it even
> possible to optimize the expression's positional predicate (or
> subsequence)?  Or does 'order by' make optimizations impossible?

The "order by" has to be processed to determine the position of each
item in the result sequence. It would be an option to use the range
index as Mike suggested to speed up the sorting.

>> Note that using subsequence instead of the predicate can be a lot
>> faster sometimes.
>
> I didn't realize that.  Can you generalize about when subsequence
> would be faster?  And would you say it's always at least as fast as
> the predicate?

Mike used position() within the predicate: [position() = (1 to 10)].
This will indeed be slower than a corresponding subsequence($input, 1,
10). Reason: the predicate has to be evaluated once for every item in
the context sequence. This doesn't make a big difference if you have a
simple expression like $a[position() = (1 to 10)], but it could result
in a substantial performance loss for more complex expressions
involving a path expression like a//*/b[position() = (1 to 10)].

To be on the safe side, I would recommend subsequence() if you need to
select *more than one* item.

Wolfgang

Re: [Exist-development] optimizing positional predicates for fast paging

[Michael S. <so...@if...>]

Thanks for your answer, Wolfgang - that's just the sort of thing I was
wondering about.

I walked through the logic in selectFollowing: I can see it would be
possible, but complicated to implement something similar for other axes and
for a somewhat more complex predicate (compare position against a constant
sequence).  Maybe that's worth doing - I'm not sure though.

Frankly the main cases I am interested in are not really axis selections,
but ordered sequences of nodes from different documents where the ordering
is determined either by a value (that may be indexed) or by the lucene
score.  These correspond to searches in our applications where the results
are ordered alphabetically by title, or by relevance.  So I asked the other
question mostly just to try and understand better what kind of optimization
strategies you are using.

I'll keep looking and let you know if I have anything helpful to offer!

I'm also curious about the answer to Joe's question re: the difference
between subsequence and predicate: if you have a moment could you elucidate?

-Mike

Also: my e-mails to this list seem to need moderation.  If you'd rather not
be bothered about that, I could move this discussion over to the exist-open
list?

> -----Original Message-----
> From: Wolfgang Meier [mailto:wol...@ex...] 
> Sent: Monday, December 14, 2009 3:37 AM
> To: Michael Sokolov
> Cc: exi...@li...
> Subject: Re: [Exist-development] optimizing positional 
> predicates for fast paging
> 
> > I wonder whether eXist is currently optimizing xquery 
> evaluations that 
> > use positional predicates, or similar.
> 
> I did implement shortcuts for the expensive axes: following 
> and preceding. For example, LocationStep.getFollowing checks 
> the property hasPositionalPredicate. If set to true, the 
> method evaluates the predicate in advance to get its numeric 
> value. This is then forwarded to the NodeSet method which 
> calculates the join:
> 
> return currentSet.selectFollowing(contextSet, position, contextId);
> 
> We could apply similar optimizations to other axes, though 
> their execution paths are more complex as you have more 
> possibilities to choose from.
> 
> However, my positional predicate optimization would only work 
> for //elem[1], not //elem[position() = 1]. But I guess it 
> would be possible to add a rewrite rule here to automatically 
> simplify the expression.
> 
> > collection("/")/ (//elem) [position() = (1 to 10)]
> 
> Note that using subsequence instead of the predicate can be a 
> lot faster sometimes.
> 
> Wolfgang
>