exist-development — eXist Developer's List

[Exist-development] Problem with new permissions stuff...

[Andrzej J. T. <an...@ch...>]

Adam:

Why did I know this was going to break things for us? ;-)

If you try to use your newest permissions code base to access a remote db (which is still running an
older version of eXist without the new permissions stuff) then you get the following NPE:

/home/andrzej/coalese/dev/raven/build.xml:656: java.lang.NullPointerException
	at java.util.Arrays$ArrayList.<init>(Arrays.java:3357)
	at java.util.Arrays.asList(Arrays.java:3343)
	at org.exist.xmldb.RemoteCollection.readCollection(RemoteCollection.java:432)
	at org.exist.xmldb.RemoteCollection.getChildCollection(RemoteCollection.java:159)
	at org.exist.xmldb.RemoteCollection.getChildCollection(RemoteCollection.java:153)
	at org.exist.xmldb.DatabaseImpl.readCollection(DatabaseImpl.java:248)
	at org.exist.xmldb.DatabaseImpl.getRemoteCollection(DatabaseImpl.java:225)
	at org.exist.xmldb.DatabaseImpl.getCollection(DatabaseImpl.java:158)
	at org.exist.xmldb.DatabaseImpl.getCollection(DatabaseImpl.java:151)
	at org.xmldb.api.DatabaseManager.getCollection(Unknown Source)
	at org.exist.ant.AbstractXMLDBTask.mkcol(AbstractXMLDBTask.java:141)
	at org.exist.ant.XMLDBStoreTask.execute(XMLDBStoreTask.java:105)

This is something we do all the time....we use ant tasks that point to a more or less current SVN
copy of eXist to access remote databases to deploy various items (xqueries, create collections,
etc.) in the remote DB.

Sure would be nice if this was backward compatible in this particular scenario, since this has
worked for about 4 years now....but the latest permissions code has broken compatibility.

Thanks!

-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] Problem with new permissions stuff...

[Andrzej J. T. <an...@ch...>]

Adam:

> The XML-RPC API may have changed, personally I am not that concerned
> about backwards compatibility here because I would rather suggest that
> you use the same version of the client and the server together.

I would love to....but the attribute replication bug I reported over a month ago is still present,
and is precluding us from upgrading our remote systems to a more recent release.  I had hoped that
Wolfgangs recent fixes for the optimizer would fix this bug, but unfortunately they haven't.

So that means we now need to keep multiple versions of the exist distribution around and select the
correct one based on the version of the server code we are accessing remotely.  That is a real PITA
from a management perspective.

> Of course, you are welcome to make suggestions for fixes to the code
> which will allow the current and previous behaviours to co-exist.

Very helpful...thanks. ;-(


-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] Problem with new permissions stuff...

[Andrzej J. T. <an...@ch...>]

Adam:

> Its not that I am being unhelpful here, its that we have different
> perspectives and responsibilities. My perspective is that I am not
> concerned with backwards compatibility in the XML-RPC API and believe
> that that will be fine for the majority of users. Your perspective is
> that of a user that would prefer to see XML-RPC backward compatibility
> because its difficult inside your project.

Yup...that sums it up pretty well.

> Also we are both quite busy people, so I tend to focus on what is
> important to me. Maintaing backwards compatibility in the XML-RPC API
> is not important to me personally, I am not personally interested in
> it and no one is paying me to do that. Conversely, I am sure you focus
> on what is important to you... So if this is very important to you,
> then I also know that you have the necessary programming skills to
> look into this yourself and propose or contribute a fix (if one is
> possible).

I would love to....but in this particular case, the new Unix permissions stuff touched nearly a
hundred source files.  As you point out, there's not enough time for me to try and understand all
the intricacies of that code at this time, due to it's breadth and impact on the main code base.

What spare time I have, I'm going to try and devote this week to delving into the "duplicate
attribute" bug, since that has been tough to replicate with a simple test case, so I'll have to dive
into the code to figure out what is causing this.  This bug has serious implications on data and
code integrity, even though it's obviously a very specific set of circumstances (as yet unknown)
that cause it to manifest.  Plus fixing that bug means I can probably update our remote instances to
the latest SVN version which would resolve the XML-RPC API issue.

Because of instabilities, security re-architecture (and bugs), database structure/version changes
and the like, we've been unable to upgrade since last July, which is almost a year ago now.

The instability of the code base and lack of a 1.5 production capable release has been a concern to
me for some time now, since in the past everything was pretty much rock solid. Massive changes that
touch the whole code base, such as the recent permissions stuff, worry me since there's a good
chance that they will take a while to stabilize/debug (not to mention the testing required and
potential impact on existing applications, which I suspect will be great) and thus the code base
continues to be unstable.

The only reason I had updated our R&D instances to the latest SVN revisions was to be able to test
the impact of the permissions stuff on our apps.  I've reverted to a revision just prior to the
permissions stuff to get our deployment stuff to work again, since that was the easiest path for us.
I'll look into managing multiple versions of the eXist code base when we're ready to test the
permissions stuff. Given how extensive our app is and our history of finding issues in new code as a
result, I suspect that we'll chat again soon. ;-)

> Dont get me wrong, I would love to have the resources to take every
> sensible suggestion on-board and spend time working on everything for
> free for everyone, but I just dont have the resources at my disposal.

Same for me....though I think some time will free up soon for me to do more work on the eXist code
base, which I have not been able to do for the past few months.

-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] Problem with new permissions stuff...

[Wolfgang M. <wol...@ex...>]

>> /home/andrzej/coalese/dev/raven/build.xml:656: java.lang.NullPointerException
>>        at java.util.Arrays$ArrayList.<init>(Arrays.java:3357)
>>        at java.util.Arrays.asList(Arrays.java:3343)
>>        at org.exist.xmldb.RemoteCollection.readCollection(RemoteCollection.java:432)

I guess fixing this is just a matter of checking for acl == null after

final Object[] acl = (Object[])collection.get("acl");

in line 431 of RemoteCollection.

Wolfgang

Re: [Exist-development] HEADS UP - Revision 14670 Permissions Changes

[Anthony F. <an...@re...>]

Very needed and welcome changes. I'm looking forward to getting them
in place. Thanks Adam!

On Fri, Jun 10, 2011 at 6:57 PM, Adam Retter <ad...@ex...> wrote:
> Heads up - The permissions model in eXist-db (as a first step) has
> been tightened up, default permissions on new collections (including
> /db) and resources are now 0644 by default, this now aligns closely
> with the Unix model.
>
> The facility for Access Control List permissions has now also been
> added to eXist-db, however this functionality has yet to be documented
> and will not be endorsed just yet, as their are a few more security
> concepts that need to be completed (e.g. SetUID, SetGID and moving
> from rwu to rwx) before the documentation is ready to be finalised.
>
> Whilst we try to preserve backwards compatibility, if you have been relying on -
>
> 1) Guest Write/Update access of resources in either /db or newly
> created collections - this will no longer work, however you `really`
> should not have been relying on these lax permissions.
>
> 2) Execute permission of XQuery resources without thinking about the
> user or permissions. This facility will soon be tightened up so that
> execute permissions are required on queries.
>
> Any questions... ;-)
>
> Thanks Adam.
>
> --
> Adam Retter
>
> eXist Developer
> { United Kingdom }
> ad...@ex...
> irc://irc.freenode.net/existdb
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>

Re: [Exist-development] Problem with new permissions stuff...

[Adam R. <ad...@ex...>]

>> Its not that I am being unhelpful here, its that we have different
>> perspectives and responsibilities. My perspective is that I am not
>> concerned with backwards compatibility in the XML-RPC API and believe
>> that that will be fine for the majority of users. Your perspective is
>> that of a user that would prefer to see XML-RPC backward compatibility
>> because its difficult inside your project.
>
> Yup...that sums it up pretty well.
>
>> Also we are both quite busy people, so I tend to focus on what is
>> important to me. Maintaing backwards compatibility in the XML-RPC API
>> is not important to me personally, I am not personally interested in
>> it and no one is paying me to do that. Conversely, I am sure you focus
>> on what is important to you... So if this is very important to you,
>> then I also know that you have the necessary programming skills to
>> look into this yourself and propose or contribute a fix (if one is
>> possible).
>
> I would love to....but in this particular case, the new Unix permissions stuff touched nearly a
> hundred source files.  As you point out, there's not enough time for me to try and understand all
> the intricacies of that code at this time, due to it's breadth and impact on the main code base.

Yes, there were a LOT of changes involved, but you should only have to
focus on the classes in org.exist.xmlrpc and org.exist.xmldb - maybe 4
files in total I think.

> What spare time I have, I'm going to try and devote this week to delving into the "duplicate
> attribute" bug, since that has been tough to replicate with a simple test case, so I'll have to dive
> into the code to figure out what is causing this.  This bug has serious implications on data and
> code integrity, even though it's obviously a very specific set of circumstances (as yet unknown)
> that cause it to manifest.  Plus fixing that bug means I can probably update our remote instances to
> the latest SVN version which would resolve the XML-RPC API issue.

Okay cool, it would be great to get a reproducible test case for that one.

> Because of instabilities, security re-architecture (and bugs), database structure/version changes
> and the like, we've been unable to upgrade since last July, which is almost a year ago now.

Yes trunk has seen a lot of churn in the last year. Growing pains ;-)

However, it is trunk which is stable for developers, but we dont
guarantee stable for production. 1.4 is our production version and
1.4.1 is on the doorstep of being released, a new preview of 1.4.1 is
out and in a few weeks if we dont have any reported issues then it
will hit the streets.

I would expect shortly after 1.4.1 to see a release of 1.6 our new
Next Gen version based on 1.5 dev code.

> The instability of the code base and lack of a 1.5 production capable release has been a concern to
> me for some time now, since in the past everything was pretty much rock solid. Massive changes that
> touch the whole code base, such as the recent permissions stuff, worry me since there's a good
> chance that they will take a while to stabilize/debug (not to mention the testing required and
> potential impact on existing applications, which I suspect will be great) and thus the code base
> continues to be unstable.

Actually the ACL changes were relatively straight forward and apart
from all the glue that had to be modified the core changes are minimal
and if you choose not to use the ACL stuff, but stick with the default
Unix Style permissions then you should not experience any differences
in the core.

> The only reason I had updated our R&D instances to the latest SVN revisions was to be able to test
> the impact of the permissions stuff on our apps.  I've reverted to a revision just prior to the
> permissions stuff to get our deployment stuff to work again, since that was the easiest path for us.
> I'll look into managing multiple versions of the eXist code base when we're ready to test the
> permissions stuff. Given how extensive our app is and our history of finding issues in new code as a
> result, I suspect that we'll chat again soon. ;-)

Having testers like yourselves is incredibly valuable to us, and we
appreciate the feedback you provide, but I am sure you can imagine,
that without reproducible test cases life can be very hard for us.

>> Dont get me wrong, I would love to have the resources to take every
>> sensible suggestion on-board and spend time working on everything for
>> free for everyone, but I just dont have the resources at my disposal.
>
> Same for me....though I think some time will free up soon for me to do more work on the eXist code
> base, which I have not been able to do for the past few months.
>
> --
> Andrzej Taramina
> Chaeron Corporation: Enterprise System Solutions
> http://www.chaeron.com
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] Problem with new permissions stuff...

[Adam R. <ad...@ex...>]

On 13 June 2011 16:24, Andrzej Jan Taramina <an...@ch...> wrote:
> Adam:
>
>> The XML-RPC API may have changed, personally I am not that concerned
>> about backwards compatibility here because I would rather suggest that
>> you use the same version of the client and the server together.
>
> I would love to....but the attribute replication bug I reported over a month ago is still present,
> and is precluding us from upgrading our remote systems to a more recent release.  I had hoped that
> Wolfgangs recent fixes for the optimizer would fix this bug, but unfortunately they haven't.
>
> So that means we now need to keep multiple versions of the exist distribution around and select the
> correct one based on the version of the server code we are accessing remotely.  That is a real PITA
> from a management perspective.
>
>> Of course, you are welcome to make suggestions for fixes to the code
>> which will allow the current and previous behaviours to co-exist.
>
> Very helpful...thanks. ;-(

Its not that I am being unhelpful here, its that we have different
perspectives and responsibilities. My perspective is that I am not
concerned with backwards compatibility in the XML-RPC API and believe
that that will be fine for the majority of users. Your perspective is
that of a user that would prefer to see XML-RPC backward compatibility
because its difficult inside your project.

Also we are both quite busy people, so I tend to focus on what is
important to me. Maintaing backwards compatibility in the XML-RPC API
is not important to me personally, I am not personally interested in
it and no one is paying me to do that. Conversely, I am sure you focus
on what is important to you... So if this is very important to you,
then I also know that you have the necessary programming skills to
look into this yourself and propose or contribute a fix (if one is
possible).

Dont get me wrong, I would love to have the resources to take every
sensible suggestion on-board and spend time working on everything for
free for everyone, but I just dont have the resources at my disposal.

>
> --
> Andrzej Taramina
> Chaeron Corporation: Enterprise System Solutions
> http://www.chaeron.com
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] Problem with new permissions stuff...

[Adam R. <ad...@ex...>]

The XML-RPC API may have changed, personally I am not that concerned
about backwards compatibility here because I would rather suggest that
you use the same version of the client and the server together.

Of course, you are welcome to make suggestions for fixes to the code
which will allow the current and previous behaviours to co-exist.

On 13 June 2011 16:15, Andrzej Jan Taramina <an...@ch...> wrote:
> Adam:
>
> Why did I know this was going to break things for us? ;-)
>
> If you try to use your newest permissions code base to access a remote db (which is still running an
> older version of eXist without the new permissions stuff) then you get the following NPE:
>
> /home/andrzej/coalese/dev/raven/build.xml:656: java.lang.NullPointerException
>        at java.util.Arrays$ArrayList.<init>(Arrays.java:3357)
>        at java.util.Arrays.asList(Arrays.java:3343)
>        at org.exist.xmldb.RemoteCollection.readCollection(RemoteCollection.java:432)
>        at org.exist.xmldb.RemoteCollection.getChildCollection(RemoteCollection.java:159)
>        at org.exist.xmldb.RemoteCollection.getChildCollection(RemoteCollection.java:153)
>        at org.exist.xmldb.DatabaseImpl.readCollection(DatabaseImpl.java:248)
>        at org.exist.xmldb.DatabaseImpl.getRemoteCollection(DatabaseImpl.java:225)
>        at org.exist.xmldb.DatabaseImpl.getCollection(DatabaseImpl.java:158)
>        at org.exist.xmldb.DatabaseImpl.getCollection(DatabaseImpl.java:151)
>        at org.xmldb.api.DatabaseManager.getCollection(Unknown Source)
>        at org.exist.ant.AbstractXMLDBTask.mkcol(AbstractXMLDBTask.java:141)
>        at org.exist.ant.XMLDBStoreTask.execute(XMLDBStoreTask.java:105)
>
> This is something we do all the time....we use ant tasks that point to a more or less current SVN
> copy of eXist to access remote databases to deploy various items (xqueries, create collections,
> etc.) in the remote DB.
>
> Sure would be nice if this was backward compatible in this particular scenario, since this has
> worked for about 4 years now....but the latest permissions code has broken compatibility.
>
> Thanks!
>
> --
> Andrzej Taramina
> Chaeron Corporation: Enterprise System Solutions
> http://www.chaeron.com
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

[Exist-development] Pre-release 1.4.1 rev14669

[Dannes W. <da...@ex...>]

All,

The development team is happy to announce the availability of a new pre-release version of eXist-db release 1.4.1. This version contains a number of important stability bugfixes since rev14490. The revision contains the following (selected) items:

	• (14469) Fixed concurrency issue: db could deadlock under very high load.
	• (14566) Update replace with a document node causes corruption.
	• (14561) REST interface can disallow collection browsing and evaluation of special parameters like _query.

A detailed changelog is available. The pre-release can be downloaded from sourceforge:

	• Installers: .jar (cross-platform) and .exe for Windows
	• For Tomcat: .war

Please help us by testing the software! By proving your feedback (via mailinglist) we'll be able to release the final version soon.

Find more on http://atomic.exist-db.org/blogs/eXist/Prerelease141

Kind regards

Dannes

--
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

[Exist-development] HEADS UP - Revision 14670 Permissions Changes

[Adam R. <ad...@ex...>]

Heads up - The permissions model in eXist-db (as a first step) has
been tightened up, default permissions on new collections (including
/db) and resources are now 0644 by default, this now aligns closely
with the Unix model.

The facility for Access Control List permissions has now also been
added to eXist-db, however this functionality has yet to be documented
and will not be endorsed just yet, as their are a few more security
concepts that need to be completed (e.g. SetUID, SetGID and moving
from rwu to rwx) before the documentation is ready to be finalised.

Whilst we try to preserve backwards compatibility, if you have been relying on -

1) Guest Write/Update access of resources in either /db or newly
created collections - this will no longer work, however you `really`
should not have been relying on these lax permissions.

2) Execute permission of XQuery resources without thinking about the
user or permissions. This facility will soon be tightened up so that
execute permissions are required on queries.

Any questions... ;-)

Thanks Adam.

-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] SVN Lock 16:00 CEST Today

[Adam R. <ad...@ex...>]

Apologies for the delay. The Subversion repository has now been
successfully upgraded, is unlocked and available for your pleasure
again :-)

Thanks for bearing with us.

Cheers Adam.

On 9 June 2011 17:56, Adam Retter <ad...@ex...> wrote:
> Unfortunately there is a hiccup in the restore process, SourceForge
> dont allocate enough memory to your shell account for you to be able
> to restore large SVN backups.
>
> I have talked to a SF support worker and subsequently opened a ticket,
> which they will look at ASAP -
> https://sourceforge.net/apps/trac/sourceforge/ticket/19862
>
> Done Panic! You can see that others have also has this issue in the
> past and it has been resolved by SF support staff -
> https://sourceforge.net/apps/trac/sourceforge/ticket/5955
>
> I will post another update once this is resolved, and the repo is
> available to all again (hopefully very soon).
>
> Thanks Adam.
>
> On 9 June 2011 11:44, Adam Retter <ad...@ex...> wrote:
>> Developers,
>>
>> I will be locking the Subversion repository today at 16.00 Central
>> European Summer Time.
>>
>> The purpose of this is to upgrade the format of our Subversion
>> repository to Subversion 1.6. This should not take longer than 1 hour
>> I hope.
>>
>> I will announce again when the upgrade is complete. The reason for
>> this upgrade is to enable support for the --reintegrate option when
>> merging with Subversion 1.6, this makes it much easier to merge
>> complex changes from a branch back to trunk. For further information
>> here is a good starting point -
>> http://blogs.collab.net/subversion/2008/07/subversion-merg/
>>
>> You should not have to take any action locally, however you will
>> subsequently have to have version 1.6 of the Subversion Client to
>> operate on the new repository. This should not be an issue as 1.6 is
>> now rather well established.
>>
>> Thanks Adam.
>>
>> --
>> Adam Retter
>>
>> eXist Developer
>> { United Kingdom }
>> ad...@ex...
>> irc://irc.freenode.net/existdb
>>
>
>
>
> --
> Adam Retter
>
> eXist Developer
> { United Kingdom }
> ad...@ex...
> irc://irc.freenode.net/existdb
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] SVN Lock 16:00 CEST Today

[Adam R. <ad...@ex...>]

Unfortunately there is a hiccup in the restore process, SourceForge
dont allocate enough memory to your shell account for you to be able
to restore large SVN backups.

I have talked to a SF support worker and subsequently opened a ticket,
which they will look at ASAP -
https://sourceforge.net/apps/trac/sourceforge/ticket/19862

Done Panic! You can see that others have also has this issue in the
past and it has been resolved by SF support staff -
https://sourceforge.net/apps/trac/sourceforge/ticket/5955

I will post another update once this is resolved, and the repo is
available to all again (hopefully very soon).

Thanks Adam.

On 9 June 2011 11:44, Adam Retter <ad...@ex...> wrote:
> Developers,
>
> I will be locking the Subversion repository today at 16.00 Central
> European Summer Time.
>
> The purpose of this is to upgrade the format of our Subversion
> repository to Subversion 1.6. This should not take longer than 1 hour
> I hope.
>
> I will announce again when the upgrade is complete. The reason for
> this upgrade is to enable support for the --reintegrate option when
> merging with Subversion 1.6, this makes it much easier to merge
> complex changes from a branch back to trunk. For further information
> here is a good starting point -
> http://blogs.collab.net/subversion/2008/07/subversion-merg/
>
> You should not have to take any action locally, however you will
> subsequently have to have version 1.6 of the Subversion Client to
> operate on the new repository. This should not be an issue as 1.6 is
> now rather well established.
>
> Thanks Adam.
>
> --
> Adam Retter
>
> eXist Developer
> { United Kingdom }
> ad...@ex...
> irc://irc.freenode.net/existdb
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] SVN Lock 16:00 CEST Today

[Dannes W. <da...@ex...>]

Adam,

On Thu, Jun 9, 2011 at 11:44 AM, Adam Retter <ad...@ex...> wrote:
> The purpose of this is to upgrade the format of our Subversion
> repository to Subversion 1.6. This should not take longer than 1 hour
> I hope.

This is really great news. In the past I wasted literally many
evenings with the 'broken' merge function (see earlier messages).
This will save us lot's of time and effort.

Thnx for picking this up!

cheers

Dannes

-- 
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

[Exist-development] SVN Lock 16:00 CEST Today

[Adam R. <ad...@ex...>]

Developers,

I will be locking the Subversion repository today at 16.00 Central
European Summer Time.

The purpose of this is to upgrade the format of our Subversion
repository to Subversion 1.6. This should not take longer than 1 hour
I hope.

I will announce again when the upgrade is complete. The reason for
this upgrade is to enable support for the --reintegrate option when
merging with Subversion 1.6, this makes it much easier to merge
complex changes from a branch back to trunk. For further information
here is a good starting point -
http://blogs.collab.net/subversion/2008/07/subversion-merg/

You should not have to take any action locally, however you will
subsequently have to have version 1.6 of the Subversion Client to
operate on the new repository. This should not be an issue as 1.6 is
now rather well established.

Thanks Adam.

-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] betterFORM-extension: switched to HTTP Components

[Tobi K. <tob...@be...>]

On Monday 30 May 2011 11:09:59 Adam Retter wrote:
> Nope, absolutely no problem. Actually we do use both side-by-side
> already when the EXPath httpclient is enabled ;-)
Okay, great!
> 
> On 30 May 2011 11:04, Tobi Krebs <tob...@be...> wrote:
> > Hi All,
> > 
> > we are working on a new Release-Candidate of betterFORM and would be
> > happy to add it to the eXist Subversion repository as soon as it ready.
> > 
> > Because we switched from Commons HTTP Client to HTTP Components in the
> > meantime we would like to ask if anybody has any concerns in having the
> > HTTP Client and HTTP Components libraries side-by-side.
> > 
> > The HTTP Components are a complete rewrite with an distinct "package-
> > namespace" so there should be no conflicts between the two libraries.
> > 
> > We would be happy to hear your thoughts.
> > 
> > Cheers
> > 
> > Tobi
> > 
> > 
> > 
> > 
> > 
> > 
> > -------------------------------------------------------------------------
> > ----- vRanger cuts backup time in half-while increasing security.
> > With the market-leading solution for virtual backup and recovery,
> > you get blazing-fast, flexible, and affordable data protection.
> > Download your free trial now.
> > http://p.sf.net/sfu/quest-d2dcopy1
> > _______________________________________________
> > Exist-development mailing list
> > Exi...@li...
> > https://lists.sourceforge.net/lists/listinfo/exist-development

Re: [Exist-development] betterFORM-extension: switched to HTTP Components

[Adam R. <ad...@ex...>]

Nope, absolutely no problem. Actually we do use both side-by-side
already when the EXPath httpclient is enabled ;-)

On 30 May 2011 11:04, Tobi Krebs <tob...@be...> wrote:
> Hi All,
>
> we are working on a new Release-Candidate of betterFORM and would be happy to
> add it to the eXist Subversion repository as soon as it ready.
>
> Because we switched from Commons HTTP Client to HTTP Components in the
> meantime we would like to ask if anybody has any concerns in having the HTTP
> Client and HTTP Components libraries side-by-side.
>
> The HTTP Components are a complete rewrite with an distinct "package-
> namespace" so there should be no conflicts between the two libraries.
>
> We would be happy to hear your thoughts.
>
> Cheers
>
> Tobi
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> vRanger cuts backup time in half-while increasing security.
> With the market-leading solution for virtual backup and recovery,
> you get blazing-fast, flexible, and affordable data protection.
> Download your free trial now.
> http://p.sf.net/sfu/quest-d2dcopy1
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

[Exist-development] betterFORM-extension: switched to HTTP Components

[Tobi K. <tob...@be...>]

Hi All,

we are working on a new Release-Candidate of betterFORM and would be happy to 
add it to the eXist Subversion repository as soon as it ready. 

Because we switched from Commons HTTP Client to HTTP Components in the 
meantime we would like to ask if anybody has any concerns in having the HTTP 
Client and HTTP Components libraries side-by-side.

The HTTP Components are a complete rewrite with an distinct "package-
namespace" so there should be no conflicts between the two libraries.

We would be happy to hear your thoughts.

Cheers

Tobi

Re: [Exist-development] Strange PathExpr warnings in log....

[Andrzej J. T. <an...@ch...>]

Wolfgang:

> the warnings are coming from the XQuery optimizer and can be safely
> ignored. They point to an optimizer issue though: I recently tried to
> make the compiled XQuery expression tree more lightweight and removed
> a few intermediate objects. Obviously I did too much and broke some
> optimizations.

I suspect that the same changes also are causing the repeated attribute expression values bug that
I've reported, but as of yet have not been able to create a simple test case for.

> I still need to look into this.

Much appreciated!  I'll keep an eye out for checkins pertaining to the optimizer, and will test both
the warnings and the repeated attribute expression values bug when you check in some fixes.

Thanks!

....A

> 
> 2011/5/18 Andrzej Jan Taramina <an...@ch...>:
>> Wolfgang:
>>
>> I'm testing latest trunk.  Getting a bunch of warnings like the following:
>>
>> 2011-05-18 12:30:28,671 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
>> [replaceExpression]:119) - Expression not found: $matchDate[child::v3:code/attribute::codeSystem =
>> $syscode]; in: exists($syscode)
>> 2011-05-18 12:30:28,671 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
>> [replaceExpression]:119) - Expression not found: $matchDate[child::v3:code/attribute::codeSystemName
>> = $sysname]; in: exists($sysname)
>> 2011-05-18 12:30:28,671 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
>> [replaceExpression]:119) - Expression not found:
>> $matchValCode[text:matches-regex(child::v3:statusCode/attribute::code, $pattern)]; in:
>> $caseSensitive = "true"
>> 2011-05-18 12:30:28,673 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
>> [replaceExpression]:119) - Expression not found: $encounters[contains(child::v3:text, $pattern)];
>> in: $caseSensitive = "true"
>> 2011-05-18 12:30:28,673 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
>> [replaceExpression]:119) - Expression not found: $observations[contains(child::v3:value, $pattern)];
>> in: $caseSensitive = "true"
>> 2011-05-18 12:30:28,673 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
>> [replaceExpression]:119) - Expression not found: $observations[text:matches-regex(child::v3:value,
>> $pattern)]; in: $caseSensitive = "true"
>> 2011-05-18 12:30:28,674 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
>> [replaceExpression]:119) - Expression not found:
>> $observations[contains(child::v3:value/attribute::value, $pattern)]; in: $caseSensitive = "true"
>> 2011-05-18 12:30:28,674 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
>> [replaceExpression]:119) - Expression not found:
>> $observations[text:matches-regex(child::v3:value/attribute::value, $pattern)]; in: $caseSensitive =
>> "true"
>> 2011-05-18 12:30:28,675 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
>> [replaceExpression]:119) - Expression not found:
>> common:get-sections-with-data($patientRecord)[common:get-section-id(self::node()) = $id]; in:
>> common:get-section-by-id(item()*, item()*) item()*
>>
>> Which just look very suspicious, since I don't see how the new and old expressions might be
>> related....and I've never seen these warnings in the past.
>>
>> I wonder if it might be related to the repeated attribute expression values bug that I've been
>> seeing (copied that post below for reference) since both relate to expression handling in XQuery?
>>
>> I still haven't managed to create a simple test case (I've been trying!) to replicate the repeated
>> attribute bug....though in my production XQuery code (which is very complex) it happens consistently.
>>
>> Help!
>>
>> Thanks.
>>
>> .....Andrzej
>>
>>> More testing on this issue:
>>>
>>> I changed the test function to not use the Binary stuff, so it looks like this:
>>>
>>> declare function common:test()
>>> {
>>>     let $line := <property name="java.naming.security.credentials"
>>>                          value="{ xs:string( 'dGVzdA==' ) }"/>
>>>
>>>     let $debug2 := util:log-system-out( concat( "Credentials XML String: ",
>>>                      util:serialize(  $line, "method=xml media-type=text/xml indent=yes" ) ) )
>>>
>>>     return $line
>>> };
>>>
>>> And I still get the erroneous result, with 8 repeated values:
>>>
>>> (Line: 494) Credentials XML String: <property name="java.naming.security.credentials"
>>>             value="dGVzdA== dGVzdA== dGVzdA== dGVzdA== dGVzdA== dGVzdA== dGVzdA== dGVzdA=="/>
>>>
>>>
>>> So that means your Binary stuff is off the hook, Adam!  The problem is somewhere else....maybe in
>>> the query parser or execution engine.
>>>
>>> Changing the line that creates the xml snippet to:
>>>
>>>    let $line := <property name="java.naming.security.credentials"
>>>                         value="{ let $tmp := 'dGVzdA==' return $tmp }"/>
>>>
>>> shows the same issue....8 repeated strings, delimited with a space for the @value attribute.
>>>
>>> But this variant does NOT repeat the values!
>>>
>>>       let $tmp := 'dGVzdA=='
>>>
>>>       let $line := <property name="java.naming.security.credentials" value="{ $tmp }"/>
>>>
>>> and neither does this one:
>>>
>>>       let $tmp := 'dGVzdA=='
>>>
>>>       let $line := <property name="java.naming.security.credentials"
>>>                              value="{ xs:string( 'dGVzdA==' ) }"/>
>>>
>>> Which leads me to believe that the bug is partially position sensitive.  If the creation of the xml
>>> snippet is the first line of code in the function, then the values get replicated 8 times.  If there
>>> is any code prior to that line, then it does not replicate.
>>>
>>> I then changed the test function to take a parameter (the string being replicated) and changed the
>>> calling function to pass in the string:
>>>
>>> So this version:
>>>
>>> declare function common:test( $tmp )
>>> {
>>>     let $line := <property name="java.naming.security.credentials" value="{ xs:string( $tmp ) }"/>
>>>
>>>     .....
>>> };
>>>
>>> replicates the attribute value 8 times (showing the bug), but this one:
>>>
>>> declare function common:test( $tmp )
>>> {
>>>     let $line := <property name="java.naming.security.credentials" value="{ $tmp }"/>
>>>
>>>     .....
>>> };
>>>
>>> does not replicate (no bug!).
>>>
>>> This leads me to believe that the bug is somewhere in how code fragments inside attribute values are
>>> evaluated.
>>>
>>> Not being very familiar with the xquery parser nor evaluation engine, I can't easily dive into the
>>> code (which is very complex) to figure out what is going on. I'm hoping that all of these different
>>> test cases might twig something with you, Wolfgang, since creating a simple test case to replicate
>>> this is proving difficult.
>>>
>>> .....Andrzej
>>>
>>>
>>>>> On 30 April 2011 00:55, Andrzej Jan Taramina <an...@ch...> wrote:
>>>>>>> Wolfgang:
>>>>>>>
>>>>>>> I've hit a very weird error in trunk, that could potentially be very serious.
>>>>>>>
>>>>>>> I have some code in a module that looks like this:
>>>>>>>
>>>>>>> declare function common:test()
>>>>>>> {
>>>>>>>    let $line := <property name="java.naming.security.credentials"
>>>>>>>                           value="{ util:binary-to-string( xs:base64Binary( 'dGVzdAoK' ) ) }"/>
>>>>>>>
>>>>>>>    let $debug2 := util:log-system-out( concat( "Credentials XML: ",
>>>>>>>                        util:serialize(  $line, "method=xml media-type=text/xml indent=yes" ) ) )
>>>>>>>
>>>>>>>    return $line
>>>>>>> };
>>>>>>>
>>>>>>> declare function common:get-ldap-connection()  as xs:integer
>>>>>>> {
>>>>>>>    let $test := common:test()
>>>>>>>
>>>>>>>    let $contextProps :=
>>>>>>>    <properties>
>>>>>>>       <property name="java.naming.factory.initial" value="com.sun.jndi.ldap.LdapCtxFactory"/>
>>>>>>>       <property name="java.naming.provider.url"
>>>>>>>                 value="{ common:get-config()/config/ldap/url/text() }"/>
>>>>>>>       <property name="java.naming.security.authentication" value="simple"/>
>>>>>>>       <property name="java.naming.security.principal"
>>>>>>>                 value="{ common:get-config()/config/ldap/adminUser/text() }"/>
>>>>>>>       <property name="java.naming.security.credentials"
>>>>>>>                 value="{ util:binary-to-string( xs:base64Binary(
>>>>>>>                                common:get-config()/config/ldap/adminPswd/text() ) ) }"/>
>>>>>>>    </properties>
>>>>>>>
>>>>>>>    let $ctx := jndi:get-dir-context( $contextProps )
>>>>>>>
>>>>>>>    return $ctx
>>>>>>> };
>>>>>>>
>>>>>>> I then have a xquery that includes this module, and does something like this:
>>>>>>>
>>>>>>>        let $cnx := common:get-ldap-connection()
>>>>>>>
>>>>>>> All works fine if I call this from the sandbox and in the console log I will see the following line
>>>>>>> produced by the common:test() function:
>>>>>>>
>>>>>>> (Line: 494) Credentials XML: <property name="java.naming.security.credentials"
>>>>>>>                                       value="test&#xA;&#xA;"/>
>>>>>>>
>>>>>>>
>>>>>>> The problem I run into is that when I run this xquery from the scheduler (which is how it's intended
>>>>>>> to run...it's a batch reporting job), cause then I get the following output in the console log:
>>>>>>>
>>>>>>> (Line: 494) Credentials XML: <property name="java.naming.security.credentials" value="test&#xA;&#xA;
>>>>>>> test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA;
>>>>>>> test&#xA;&#xA;"/>
>>>>>>>
>>>>>>> I get 8 concatenated decoded values in the value attribute!!!!
>>>>>>>
>>>>>>> This should never happen!!!!  Of course, in that situation, I fail to get an LDAP connection since
>>>>>>> the password (the credentials) is a concatenation of 8 copies of the actual password, and so fails,
>>>>>>> if I remove the call to the test function.  If I leave in the call to the test function, then the
>>>>>>> correct (non-replicated) password is inserted into the credentials element of the $contextProps
>>>>>>> variable. If I remove the test function call, then I get the same replication in $contextProps.
>>>>>>>
>>>>>>> What is weird is that the error only shows in the one place and is sensitive to order. If I put the
>>>>>>> call to the test function after the construction of the $contextProps value, then $contextProps gets
>>>>>>> the repeated value and the test function returns a correct, unrepeated value.
>>>>>>>
>>>>>>> I've been trying to create a simple test case to replicate this, but have not yet been successful.
>>>>>>> It seems I need to run the big batch job, where the included module is 3600 lines long, and the
>>>>>>> actual xquery is very long and complex as well.
>>>>>>>
>>>>>>> But the code snippet that causes this bug is static code in the common:test() function, and should
>>>>>>> always return the correct result.
>>>>>>>
>>>>>>> I'm not sure what is causing this.  Could it be the new Binary Value caching/handling under weird
>>>>>>> conditions that only happen when running under a scheduled task?  I've never managed to reproduce
>>>>>>> this when calling xqueries directly (the ldap connection call is made all over the place in our web
>>>>>>> app for authentication purposes).  Maybe util:binary-to-string() is at fault, but that seems
>>>>>>> unlikely.  Maybe some of your path wrapper stuff?
>>>>>>>
>>>>>>> This exact same code runs just fine as a scheduled task in an older version of trunk (from late
>>>>>>> summer last year), so it's definitely something new in trunk that has manifested since then.
>>>>>>>
>>>>>>> Any ideas on what might cause something this strange?  I has me very worried since this is pretty
>>>>>>> basic stuff....and thus might cause other things like DB corruption, or incorrect values being
>>>>>>> written or wrong logic paths being followed.
>>>>>>>
>>>>>>> I'll keep trying to create a simple test case that illustrates this, but I think it may not be
>>>>>>> possible to do so.
>>>>>>>
>>
>> --
>> Andrzej Taramina
>> Chaeron Corporation: Enterprise System Solutions
>> http://www.chaeron.com
>>
> 


-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] LDAP integration

[Casey J. <cas...@jo...>]

Kasper,

Thanks so much. I ran into these problems a while ago and didn't have time
to look into it. I'll probably attempt to get LDAP working agian with your
fixes in the next few months. I will let you know how it goes.

Cheers,

Casey

On Wed, May 25, 2011 at 9:25 AM, Kasper Laudrup <kl...@vi...> wrote:

> Hi fellow eXist developers and users.
>
> In my opinion, the LDAP integration in eXist, at least in the stable
> version, has had quite a few issues. Most important for us, it didn't
> support membership of more than one group.
>
> I have rewritten the LDAP module and attached it to this post in hope that
> some one else might find it useful.
>
> The major changes are:
>
> - Support for multigroup membership with the memberUid LDAP property for
> the posixGroup object.
> - Removal of broken caching functionality which caused exist to use
> possibly outdated information when authenticating users.
> - Support for creating users through the eXist client interface.
>
> We have been using this in our production environment for a while and it
> seems to be working fine, at least better than what was in eXist before.
> Note that we do not use the eXist client interface for managing users, so
> that has not been thoroughly tested.
>
> I am, of course, willing to provide help with using this module and or
> fixing bugs. If someone thinks they might want to use it, I might even
> document it. :-)
>
> Best regards,
>
> Kasper Laudrup
>
> ViewWorld ApS.
>
>
>
> ------------------------------------------------------------------------------
> vRanger cuts backup time in half-while increasing security.
> With the market-leading solution for virtual backup and recovery,
> you get blazing-fast, flexible, and affordable data protection.
> Download your free trial now.
> http://p.sf.net/sfu/quest-d2dcopy1
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>
>


-- 
--
Casey Jordan
easyDITA a product of Jorsek LLC
"CaseyDJordan" on LinkedIn, Twitter & Facebook
(585) 348 7399
easydita.com


This message is intended only for the use of the Addressee(s) and may
contain information that is privileged, confidential, and/or exempt from
disclosure under applicable law.  If you are not the intended recipient,
please be advised that any disclosure  copying, distribution, or use of
the information contained herein is prohibited.  If you have received
this communication in error, please destroy all copies of the message,
whether in electronic or hard copy format, as well as attachments, and
immediately contact the sender by replying to this e-mail or by phone.
Thank you.

[Exist-development] LDAP integration

[Kasper L. <kl...@vi...>]

Hi fellow eXist developers and users.

In my opinion, the LDAP integration in eXist, at least in the stable
version, has had quite a few issues. Most important for us, it didn't
support membership of more than one group.

I have rewritten the LDAP module and attached it to this post in hope that
some one else might find it useful.

The major changes are:

- Support for multigroup membership with the memberUid LDAP property for the
posixGroup object.
- Removal of broken caching functionality which caused exist to use possibly
outdated information when authenticating users.
- Support for creating users through the eXist client interface.

We have been using this in our production environment for a while and it
seems to be working fine, at least better than what was in eXist before.
Note that we do not use the eXist client interface for managing users, so
that has not been thoroughly tested.

I am, of course, willing to provide help with using this module and or
fixing bugs. If someone thinks they might want to use it, I might even
document it. :-)

Best regards,

Kasper Laudrup

ViewWorld ApS.

Re: [Exist-development] Strange PathExpr warnings in log....

[Wolfgang M. <wol...@ex...>]

Andrzej,

the warnings are coming from the XQuery optimizer and can be safely
ignored. They point to an optimizer issue though: I recently tried to
make the compiled XQuery expression tree more lightweight and removed
a few intermediate objects. Obviously I did too much and broke some
optimizations.

I still need to look into this.

Wolfgang


2011/5/18 Andrzej Jan Taramina <an...@ch...>:
> Wolfgang:
>
> I'm testing latest trunk.  Getting a bunch of warnings like the following:
>
> 2011-05-18 12:30:28,671 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
> [replaceExpression]:119) - Expression not found: $matchDate[child::v3:code/attribute::codeSystem =
> $syscode]; in: exists($syscode)
> 2011-05-18 12:30:28,671 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
> [replaceExpression]:119) - Expression not found: $matchDate[child::v3:code/attribute::codeSystemName
> = $sysname]; in: exists($sysname)
> 2011-05-18 12:30:28,671 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
> [replaceExpression]:119) - Expression not found:
> $matchValCode[text:matches-regex(child::v3:statusCode/attribute::code, $pattern)]; in:
> $caseSensitive = "true"
> 2011-05-18 12:30:28,673 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
> [replaceExpression]:119) - Expression not found: $encounters[contains(child::v3:text, $pattern)];
> in: $caseSensitive = "true"
> 2011-05-18 12:30:28,673 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
> [replaceExpression]:119) - Expression not found: $observations[contains(child::v3:value, $pattern)];
> in: $caseSensitive = "true"
> 2011-05-18 12:30:28,673 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
> [replaceExpression]:119) - Expression not found: $observations[text:matches-regex(child::v3:value,
> $pattern)]; in: $caseSensitive = "true"
> 2011-05-18 12:30:28,674 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
> [replaceExpression]:119) - Expression not found:
> $observations[contains(child::v3:value/attribute::value, $pattern)]; in: $caseSensitive = "true"
> 2011-05-18 12:30:28,674 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
> [replaceExpression]:119) - Expression not found:
> $observations[text:matches-regex(child::v3:value/attribute::value, $pattern)]; in: $caseSensitive =
> "true"
> 2011-05-18 12:30:28,675 [DefaultQuartzScheduler_Worker-4] WARN  (PathExpr.java
> [replaceExpression]:119) - Expression not found:
> common:get-sections-with-data($patientRecord)[common:get-section-id(self::node()) = $id]; in:
> common:get-section-by-id(item()*, item()*) item()*
>
> Which just look very suspicious, since I don't see how the new and old expressions might be
> related....and I've never seen these warnings in the past.
>
> I wonder if it might be related to the repeated attribute expression values bug that I've been
> seeing (copied that post below for reference) since both relate to expression handling in XQuery?
>
> I still haven't managed to create a simple test case (I've been trying!) to replicate the repeated
> attribute bug....though in my production XQuery code (which is very complex) it happens consistently.
>
> Help!
>
> Thanks.
>
> .....Andrzej
>
>> More testing on this issue:
>>
>> I changed the test function to not use the Binary stuff, so it looks like this:
>>
>> declare function common:test()
>> {
>>     let $line := <property name="java.naming.security.credentials"
>>                          value="{ xs:string( 'dGVzdA==' ) }"/>
>>
>>     let $debug2 := util:log-system-out( concat( "Credentials XML String: ",
>>                      util:serialize(  $line, "method=xml media-type=text/xml indent=yes" ) ) )
>>
>>     return $line
>> };
>>
>> And I still get the erroneous result, with 8 repeated values:
>>
>> (Line: 494) Credentials XML String: <property name="java.naming.security.credentials"
>>             value="dGVzdA== dGVzdA== dGVzdA== dGVzdA== dGVzdA== dGVzdA== dGVzdA== dGVzdA=="/>
>>
>>
>> So that means your Binary stuff is off the hook, Adam!  The problem is somewhere else....maybe in
>> the query parser or execution engine.
>>
>> Changing the line that creates the xml snippet to:
>>
>>    let $line := <property name="java.naming.security.credentials"
>>                         value="{ let $tmp := 'dGVzdA==' return $tmp }"/>
>>
>> shows the same issue....8 repeated strings, delimited with a space for the @value attribute.
>>
>> But this variant does NOT repeat the values!
>>
>>       let $tmp := 'dGVzdA=='
>>
>>       let $line := <property name="java.naming.security.credentials" value="{ $tmp }"/>
>>
>> and neither does this one:
>>
>>       let $tmp := 'dGVzdA=='
>>
>>       let $line := <property name="java.naming.security.credentials"
>>                              value="{ xs:string( 'dGVzdA==' ) }"/>
>>
>> Which leads me to believe that the bug is partially position sensitive.  If the creation of the xml
>> snippet is the first line of code in the function, then the values get replicated 8 times.  If there
>> is any code prior to that line, then it does not replicate.
>>
>> I then changed the test function to take a parameter (the string being replicated) and changed the
>> calling function to pass in the string:
>>
>> So this version:
>>
>> declare function common:test( $tmp )
>> {
>>     let $line := <property name="java.naming.security.credentials" value="{ xs:string( $tmp ) }"/>
>>
>>     .....
>> };
>>
>> replicates the attribute value 8 times (showing the bug), but this one:
>>
>> declare function common:test( $tmp )
>> {
>>     let $line := <property name="java.naming.security.credentials" value="{ $tmp }"/>
>>
>>     .....
>> };
>>
>> does not replicate (no bug!).
>>
>> This leads me to believe that the bug is somewhere in how code fragments inside attribute values are
>> evaluated.
>>
>> Not being very familiar with the xquery parser nor evaluation engine, I can't easily dive into the
>> code (which is very complex) to figure out what is going on. I'm hoping that all of these different
>> test cases might twig something with you, Wolfgang, since creating a simple test case to replicate
>> this is proving difficult.
>>
>> .....Andrzej
>>
>>
>>> > On 30 April 2011 00:55, Andrzej Jan Taramina <an...@ch...> wrote:
>>>> >> Wolfgang:
>>>> >>
>>>> >> I've hit a very weird error in trunk, that could potentially be very serious.
>>>> >>
>>>> >> I have some code in a module that looks like this:
>>>> >>
>>>> >> declare function common:test()
>>>> >> {
>>>> >>    let $line := <property name="java.naming.security.credentials"
>>>> >>                           value="{ util:binary-to-string( xs:base64Binary( 'dGVzdAoK' ) ) }"/>
>>>> >>
>>>> >>    let $debug2 := util:log-system-out( concat( "Credentials XML: ",
>>>> >>                        util:serialize(  $line, "method=xml media-type=text/xml indent=yes" ) ) )
>>>> >>
>>>> >>    return $line
>>>> >> };
>>>> >>
>>>> >> declare function common:get-ldap-connection()  as xs:integer
>>>> >> {
>>>> >>    let $test := common:test()
>>>> >>
>>>> >>    let $contextProps :=
>>>> >>    <properties>
>>>> >>       <property name="java.naming.factory.initial" value="com.sun.jndi.ldap.LdapCtxFactory"/>
>>>> >>       <property name="java.naming.provider.url"
>>>> >>                 value="{ common:get-config()/config/ldap/url/text() }"/>
>>>> >>       <property name="java.naming.security.authentication" value="simple"/>
>>>> >>       <property name="java.naming.security.principal"
>>>> >>                 value="{ common:get-config()/config/ldap/adminUser/text() }"/>
>>>> >>       <property name="java.naming.security.credentials"
>>>> >>                 value="{ util:binary-to-string( xs:base64Binary(
>>>> >>                                common:get-config()/config/ldap/adminPswd/text() ) ) }"/>
>>>> >>    </properties>
>>>> >>
>>>> >>    let $ctx := jndi:get-dir-context( $contextProps )
>>>> >>
>>>> >>    return $ctx
>>>> >> };
>>>> >>
>>>> >> I then have a xquery that includes this module, and does something like this:
>>>> >>
>>>> >>        let $cnx := common:get-ldap-connection()
>>>> >>
>>>> >> All works fine if I call this from the sandbox and in the console log I will see the following line
>>>> >> produced by the common:test() function:
>>>> >>
>>>> >> (Line: 494) Credentials XML: <property name="java.naming.security.credentials"
>>>> >>                                       value="test&#xA;&#xA;"/>
>>>> >>
>>>> >>
>>>> >> The problem I run into is that when I run this xquery from the scheduler (which is how it's intended
>>>> >> to run...it's a batch reporting job), cause then I get the following output in the console log:
>>>> >>
>>>> >> (Line: 494) Credentials XML: <property name="java.naming.security.credentials" value="test&#xA;&#xA;
>>>> >> test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA;
>>>> >> test&#xA;&#xA;"/>
>>>> >>
>>>> >> I get 8 concatenated decoded values in the value attribute!!!!
>>>> >>
>>>> >> This should never happen!!!!  Of course, in that situation, I fail to get an LDAP connection since
>>>> >> the password (the credentials) is a concatenation of 8 copies of the actual password, and so fails,
>>>> >> if I remove the call to the test function.  If I leave in the call to the test function, then the
>>>> >> correct (non-replicated) password is inserted into the credentials element of the $contextProps
>>>> >> variable. If I remove the test function call, then I get the same replication in $contextProps.
>>>> >>
>>>> >> What is weird is that the error only shows in the one place and is sensitive to order. If I put the
>>>> >> call to the test function after the construction of the $contextProps value, then $contextProps gets
>>>> >> the repeated value and the test function returns a correct, unrepeated value.
>>>> >>
>>>> >> I've been trying to create a simple test case to replicate this, but have not yet been successful.
>>>> >> It seems I need to run the big batch job, where the included module is 3600 lines long, and the
>>>> >> actual xquery is very long and complex as well.
>>>> >>
>>>> >> But the code snippet that causes this bug is static code in the common:test() function, and should
>>>> >> always return the correct result.
>>>> >>
>>>> >> I'm not sure what is causing this.  Could it be the new Binary Value caching/handling under weird
>>>> >> conditions that only happen when running under a scheduled task?  I've never managed to reproduce
>>>> >> this when calling xqueries directly (the ldap connection call is made all over the place in our web
>>>> >> app for authentication purposes).  Maybe util:binary-to-string() is at fault, but that seems
>>>> >> unlikely.  Maybe some of your path wrapper stuff?
>>>> >>
>>>> >> This exact same code runs just fine as a scheduled task in an older version of trunk (from late
>>>> >> summer last year), so it's definitely something new in trunk that has manifested since then.
>>>> >>
>>>> >> Any ideas on what might cause something this strange?  I has me very worried since this is pretty
>>>> >> basic stuff....and thus might cause other things like DB corruption, or incorrect values being
>>>> >> written or wrong logic paths being followed.
>>>> >>
>>>> >> I'll keep trying to create a simple test case that illustrates this, but I think it may not be
>>>> >> possible to do so.
>>>> >>
>
> --
> Andrzej Taramina
> Chaeron Corporation: Enterprise System Solutions
> http://www.chaeron.com
>

Re: [Exist-development] xquery: xslt in recursive function

[Adam R. <ad...@ex...>]

I think you need to provide a 'LOT' more detail to get the answer to
your question. For example, what are you doing and how to we repeat
this?

2011/5/19 Василий Старцев <wst...@gm...>:
> hi!
> i try to transform document with xslt in recursive function like:
> declare function local:clean-document(node, xslt){
> (: transform node with xslt :)
> (: check transformation result :)
> (: if need do it again :)
> local:clean-document(document, xslt)
> (: else return node :)
> }
> and this transformation except with memory allocation error.
> i want to clearly understand: xq copy node in each call function?
> how to make recursive function to work with one node in each iteration?
>
> --
> С наилучшими пожеланиями
> Василий Старцев
>
> ------------------------------------------------------------------------------
> What Every C/C++ and Fortran developer Should Know!
> Read this article and learn how Intel has extended the reach of its
> next-generation tools to help Windows* and Linux* C/C++ and Fortran
> developers boost performance applications - including clusters.
> http://p.sf.net/sfu/intel-dev2devmay
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] URLs and redirects on demo.exist-db.org

[Dannes W. <di...@ex...>]

It should be fixed right now......

On 20 mei 2011, at 18:06, Dannes Wessels <di...@ex...> wrote:

> 
> On 20 mei 2011, at 14:25, Stefan Majewski <ste...@un...> wrote:
> 
>> I just used the googled for "exist-db function modules" (not in quotes)
>> and was directed to this URL:
>> 
>> http://demo.exist-db.org/functions/
>> 
>> While the page loads, most links appear to be broken in the menu (e.g.
>> eXide -> http://exist/eXide/)
> 
> Thank you Stefan for reporting the issue. I was already aware of the issue, it is in trunk as well, but I have no clue yet how to fix it. It was only introduced recently,.....
> 
> Thnx for monitoring this!
> 
> Cheers
> 
> Dannes 

Re: [Exist-development] URLs and redirects on demo.exist-db.org

[Dannes W. <di...@ex...>]

On 20 mei 2011, at 14:25, Stefan Majewski <ste...@un...> wrote:

> I just used the googled for "exist-db function modules" (not in quotes)
> and was directed to this URL:
> 
> http://demo.exist-db.org/functions/
> 
> While the page loads, most links appear to be broken in the menu (e.g.
> eXide -> http://exist/eXide/)

Thank you Stefan for reporting the issue. I was already aware of the issue, it is in trunk as well, but I have no clue yet how to fix it. It was only introduced recently,.....

Thnx for monitoring this!

Cheers

Dannes 

[Exist-development] URLs and redirects on demo.exist-db.org

[Stefan M. <ste...@un...>]

I just used the googled for "exist-db function modules" (not in quotes)
and was directed to this URL:

http://demo.exist-db.org/functions/

While the page loads, most links appear to be broken in the menu (e.g.
eXide -> http://exist/eXide/)

when visiting

http://demo.exist-db.org/

I get redirected to -> http://demo.exist-db.org/exist/index.xml

and the links appear to be correct. That is, people who come from google
might get to a broken website. If possible, I'd suggest to redirect
people to the /exist/... prefixed URLs when people arrive at a path not
starting with /exist/... . The same behaviour is seen with urls like
http://demo.exist-db.org/facts.xml

I don't know if this is related to the recent update. I would not find
it problematic if google would point to the correct pages. For people
who don't know eXist yet, and thus know that the website works, this
might be strange.

cheers,
Stefan