exist-development — eXist Developer's List

Re: [Exist-development] Very weird and serious error in trunk!

[Adam R. <ad...@ex...>]

I guess its not so serious ;-)

As its running from the scheduler, and you see the value repeat itself
8 times, could we hazard a guess that your scheduled task has been
called 8 times?

The new BinaryValues stuff,if it is not cleaned up just results in a
memory-leak, it is safely re-useable. So you can re-read the same
InputStream multiple times!

I would guess that somehow, something is appended to a session
variable, and this is not being cleaned up.

If you could remove the ldap stuff, and create some sort of
reproducible test case I would be happy to look into this.

Also for eXist-db related issues, could I ask that you use my
ad...@ex... email address. And if you have eXist Solutions
enquiries, or are one of our customers then use my
ad...@ex... email address. It helps me to keep these
things organised.

Thanks Adam.

On 30 April 2011 00:55, Andrzej Jan Taramina <an...@ch...> wrote:
> Wolfgang:
>
> I've hit a very weird error in trunk, that could potentially be very serious.
>
> I have some code in a module that looks like this:
>
> declare function common:test()
> {
>    let $line := <property name="java.naming.security.credentials"
>                           value="{ util:binary-to-string( xs:base64Binary( 'dGVzdAoK' ) ) }"/>
>
>    let $debug2 := util:log-system-out( concat( "Credentials XML: ",
>                        util:serialize(  $line, "method=xml media-type=text/xml indent=yes" ) ) )
>
>    return $line
> };
>
> declare function common:get-ldap-connection()  as xs:integer
> {
>    let $test := common:test()
>
>    let $contextProps :=
>    <properties>
>       <property name="java.naming.factory.initial" value="com.sun.jndi.ldap.LdapCtxFactory"/>
>       <property name="java.naming.provider.url"
>                 value="{ common:get-config()/config/ldap/url/text() }"/>
>       <property name="java.naming.security.authentication" value="simple"/>
>       <property name="java.naming.security.principal"
>                 value="{ common:get-config()/config/ldap/adminUser/text() }"/>
>       <property name="java.naming.security.credentials"
>                 value="{ util:binary-to-string( xs:base64Binary(
>                                common:get-config()/config/ldap/adminPswd/text() ) ) }"/>
>    </properties>
>
>    let $ctx := jndi:get-dir-context( $contextProps )
>
>    return $ctx
> };
>
> I then have a xquery that includes this module, and does something like this:
>
>        let $cnx := common:get-ldap-connection()
>
> All works fine if I call this from the sandbox and in the console log I will see the following line
> produced by the common:test() function:
>
> (Line: 494) Credentials XML: <property name="java.naming.security.credentials"
>                                       value="test&#xA;&#xA;"/>
>
>
> The problem I run into is that when I run this xquery from the scheduler (which is how it's intended
> to run...it's a batch reporting job), cause then I get the following output in the console log:
>
> (Line: 494) Credentials XML: <property name="java.naming.security.credentials" value="test&#xA;&#xA;
> test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA;
> test&#xA;&#xA;"/>
>
> I get 8 concatenated decoded values in the value attribute!!!!
>
> This should never happen!!!!  Of course, in that situation, I fail to get an LDAP connection since
> the password (the credentials) is a concatenation of 8 copies of the actual password, and so fails,
> if I remove the call to the test function.  If I leave in the call to the test function, then the
> correct (non-replicated) password is inserted into the credentials element of the $contextProps
> variable. If I remove the test function call, then I get the same replication in $contextProps.
>
> What is weird is that the error only shows in the one place and is sensitive to order. If I put the
> call to the test function after the construction of the $contextProps value, then $contextProps gets
> the repeated value and the test function returns a correct, unrepeated value.
>
> I've been trying to create a simple test case to replicate this, but have not yet been successful.
> It seems I need to run the big batch job, where the included module is 3600 lines long, and the
> actual xquery is very long and complex as well.
>
> But the code snippet that causes this bug is static code in the common:test() function, and should
> always return the correct result.
>
> I'm not sure what is causing this.  Could it be the new Binary Value caching/handling under weird
> conditions that only happen when running under a scheduled task?  I've never managed to reproduce
> this when calling xqueries directly (the ldap connection call is made all over the place in our web
> app for authentication purposes).  Maybe util:binary-to-string() is at fault, but that seems
> unlikely.  Maybe some of your path wrapper stuff?
>
> This exact same code runs just fine as a scheduled task in an older version of trunk (from late
> summer last year), so it's definitely something new in trunk that has manifested since then.
>
> Any ideas on what might cause something this strange?  I has me very worried since this is pretty
> basic stuff....and thus might cause other things like DB corruption, or incorrect values being
> written or wrong logic paths being followed.
>
> I'll keep trying to create a simple test case that illustrates this, but I think it may not be
> possible to do so.
>
> Help!
>
> Thanks!
>
> --
> Andrzej Taramina
> Chaeron Corporation: Enterprise System Solutions
> http://www.chaeron.com
>



-- 
Adam Retter
Director, eXist Solutions GmbH

e: ad...@ex...
w: http://www.existsolutions.com
t: +44 20 3239 7236
m: +44 7714 330069
Skype: adam.retter

Company Registration: HRB 89454, Amtsgericht Darmstadt

Re: [Exist-development] Very weird and serious error in trunk!

[Wolfgang M. <wol...@ex...>]

Assuming that the XQuery is really as simple as this:

<property name="java.naming.security.credentials"
                          value="{ util:binary-to-string(
xs:base64Binary( 'dGVzdAoK' ) ) }"/>

I would guess that the base64Binary is cleaned up when the query runs
stand-alone, but not when it is called through the scheduler. Maybe a
missing reset on the context? If it is, it should be reproducible.

Wolfgang

Re: [Exist-development] And yet another security/Ant issue....addendum

[Dmitriy S. <sha...@gm...>]

On Sat, Apr 30, 2011 at 7:43 AM, Andrzej Jan Taramina
<an...@ch...>wrote:

> Dmitry:
>
> > empty password "disable" account, it must have some chars.
>
> You don't seem to understand how the adduser Ant task works.  The
> user/password attributes are for
> the user that is performing the add.  On a new database that would be the
> admin user, which has no
> password set, so you provide an empty password.
>
> The user being added is specified by name/secret, in the example below
> myuser and the value of the
> Ant variable ${password}. The password is not empty!
>
> The issue is that the myuser is added, with the correct password.  You can
> log in as that user no
> problem.  Both the web and java admin clients show myuser as being in the
> "dba" group.
>
> But myuser does not have dba priviledges for some reason which is a big and
> a rather serious one in
> my estimation.  If I'm logged in as myuser, I cannot do a
> file:directory-list( $dir, "*.xml" ) as it
> fails on a permission problem.
>
> This is a bug in the security implementation that needs fixing.
>

That is not a security issue if you see it in DBA group. Did you check user
from running script? (let's say one line before file:directory-list)

-- 
Dmitriy Shabanov

Re: [Exist-development] And yet another security/Ant issue....addendum

[Andrzej J. T. <an...@ch...>]

Dmitry:

> empty password "disable" account, it must have some chars.

You don't seem to understand how the adduser Ant task works.  The user/password attributes are for
the user that is performing the add.  On a new database that would be the admin user, which has no
password set, so you provide an empty password.

The user being added is specified by name/secret, in the example below myuser and the value of the
Ant variable ${password}. The password is not empty!

The issue is that the myuser is added, with the correct password.  You can log in as that user no
problem.  Both the web and java admin clients show myuser as being in the "dba" group.

But myuser does not have dba priviledges for some reason which is a big and a rather serious one in
my estimation.  If I'm logged in as myuser, I cannot do a file:directory-list( $dir, "*.xml" ) as it
fails on a permission problem.

This is a bug in the security implementation that needs fixing.

....A

> On Fri, Apr 29, 2011 at 8:03 PM, Andrzej Jan Taramina <an...@ch...
> <mailto:an...@ch...>> wrote:
> 
>     BTW, I create the myuser like this (before I change the admin password from empty):
> 
>     <exist:adduser failonerror="false" uri="${exist.url}/xmlrpc/db" user="admin" password=""
>     name="myuser" secret="${password}" home="/db" primaryGroup="dba"/>
> 
>     That should make myuser a DBA, right?
> 
>     Logging into the admin web page shows that the user is not a DBA, since it can't do a database
>     shutdown.  But the user list (both in the admin web page and in the Java client) shows myuser as a
>     member of the dba group.
> 
>     ....A
> 
>     > I use an ant task to run an xquery as follows:
>     >
>     > <exist:xquery queryfile="${temp.dir}/importData.xql" outputproperty="importResults"
>     > uri="${exist.url}/xmlrpc/db" user="myuser" password="${password}"> </exist:xquery>
>     >
>     > but it the fails with the following:
>     >
>     > BUILD FAILED
>     > /home/andrzej/build.xml:1747: XMLDB exception caught while executing query: Permission denied,
>     > calling user 'myuser' must be a DBA to call this function. [at line 75, column 23, source: String]
>     >
>     > The line in the xquery that is failing is:
>     >
>     >       let $list   := file:directory-list( $dir, "*.xml" )
>     >
>     > which requires DBA permissions as expected.
>     >
>     > The problem is that the ant task was initiated using user "myuser" which is part of the dba
>     > group...and thus should be a DBA, right?
>     >
>     > This used to work fine....but is now broken with the new security stuff.
>     >
>     > Any idea why myuser isn't being seen as having dba permissions when executing an xquery using the
>     > ant task?
> 
> 
>     --
>     Andrzej Taramina
>     Chaeron Corporation: Enterprise System Solutions
>     http://www.chaeron.com
> 
> 
> 
> 
> -- 
> Dmitriy Shabanov


-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] And yet another security/Ant issue....

[Andrzej J. T. <an...@ch...>]

Dmitry:

> Can you make sure that 'myuser' in DBA group and check user at script by print out current user.

I did all that before I reported the problem. The user is in the dba group and is defined.  So why
the user is not considered to have dba permissions is a problem.

....A

> On Fri, Apr 29, 2011 at 7:57 PM, Andrzej Jan Taramina <an...@ch...
> <mailto:an...@ch...>> wrote:
> 
>     I use an ant task to run an xquery as follows:
> 
>     <exist:xquery queryfile="${temp.dir}/importData.xql" outputproperty="importResults"
>     uri="${exist.url}/xmlrpc/db" user="myuser" password="${password}"> </exist:xquery>
> 
>     but it the fails with the following:
> 
>     BUILD FAILED
>     /home/andrzej/build.xml:1747: XMLDB exception caught while executing query: Permission denied,
>     calling user 'myuser' must be a DBA to call this function. [at line 75, column 23, source: String]
> 
>     The line in the xquery that is failing is:
> 
>            let $list   := file:directory-list( $dir, "*.xml" )
> 
>     which requires DBA permissions as expected.
> 
>     The problem is that the ant task was initiated using user "myuser" which is part of the dba
>     group...and thus should be a DBA, right?
> 
>     This used to work fine....but is now broken with the new security stuff.
> 
>     Any idea why myuser isn't being seen as having dba permissions when executing an xquery using the
>     ant task?
> 
> 
> -- 
> Dmitriy Shabanov


-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] And yet another security/Ant issue....addendum

[Dmitriy S. <sha...@gm...>]

empty password "disable" account, it must have some chars.

On Fri, Apr 29, 2011 at 8:03 PM, Andrzej Jan Taramina
<an...@ch...>wrote:

> BTW, I create the myuser like this (before I change the admin password from
> empty):
>
> <exist:adduser failonerror="false" uri="${exist.url}/xmlrpc/db"
> user="admin" password=""
> name="myuser" secret="${password}" home="/db" primaryGroup="dba"/>
>
> That should make myuser a DBA, right?
>
> Logging into the admin web page shows that the user is not a DBA, since it
> can't do a database
> shutdown.  But the user list (both in the admin web page and in the Java
> client) shows myuser as a
> member of the dba group.
>
> ....A
>
> > I use an ant task to run an xquery as follows:
> >
> > <exist:xquery queryfile="${temp.dir}/importData.xql"
> outputproperty="importResults"
> > uri="${exist.url}/xmlrpc/db" user="myuser" password="${password}">
> </exist:xquery>
> >
> > but it the fails with the following:
> >
> > BUILD FAILED
> > /home/andrzej/build.xml:1747: XMLDB exception caught while executing
> query: Permission denied,
> > calling user 'myuser' must be a DBA to call this function. [at line 75,
> column 23, source: String]
> >
> > The line in the xquery that is failing is:
> >
> >       let $list   := file:directory-list( $dir, "*.xml" )
> >
> > which requires DBA permissions as expected.
> >
> > The problem is that the ant task was initiated using user "myuser" which
> is part of the dba
> > group...and thus should be a DBA, right?
> >
> > This used to work fine....but is now broken with the new security stuff.
> >
> > Any idea why myuser isn't being seen as having dba permissions when
> executing an xquery using the
> > ant task?
>
>
> --
> Andrzej Taramina
> Chaeron Corporation: Enterprise System Solutions
> http://www.chaeron.com
>



-- 
Dmitriy Shabanov

Re: [Exist-development] And yet another security/Ant issue....

[Dmitriy S. <sha...@gm...>]

Can you make sure that 'myuser' in DBA group and check user at script by
print out current user.

On Fri, Apr 29, 2011 at 7:57 PM, Andrzej Jan Taramina
<an...@ch...>wrote:

> I use an ant task to run an xquery as follows:
>
> <exist:xquery queryfile="${temp.dir}/importData.xql"
> outputproperty="importResults"
> uri="${exist.url}/xmlrpc/db" user="myuser" password="${password}">
> </exist:xquery>
>
> but it the fails with the following:
>
> BUILD FAILED
> /home/andrzej/build.xml:1747: XMLDB exception caught while executing query:
> Permission denied,
> calling user 'myuser' must be a DBA to call this function. [at line 75,
> column 23, source: String]
>
> The line in the xquery that is failing is:
>
>        let $list   := file:directory-list( $dir, "*.xml" )
>
> which requires DBA permissions as expected.
>
> The problem is that the ant task was initiated using user "myuser" which is
> part of the dba
> group...and thus should be a DBA, right?
>
> This used to work fine....but is now broken with the new security stuff.
>
> Any idea why myuser isn't being seen as having dba permissions when
> executing an xquery using the
> ant task?
>

-- 
Dmitriy Shabanov

Re: [Exist-development] Another Ant-related security problem....

[Dmitriy S. <sha...@gm...>]

fixed by http://exist.svn.sourceforge.net/exist/?rev=14339&view=rev

On Fri, Apr 29, 2011 at 7:43 PM, Andrzej Jan Taramina
<an...@ch...>wrote:

> Dmitry:
>
> If you try to create a user a 2nd time, using the ant task, then you get
> the following exception:
>
> [exist:adduser] XMLDB exception caught: Failed to invoke method addAccount
> in class
> org.exist.xmlrpc.RpcConnection: Configuration can't be used by <account
> name="myuser"
> id="9"></user>, because allready in use by <account name="myuser" id="6"
> home="/db"></user>
>
> Which is fine.  But then the password on the userid gets screwed up and you
> can't use that user any
> more!  Which is not so fine.  Incidentally, the second addition is being
> added with the same
> password as the first one, so nothing should change.
>
> I think that adding a user that already exists shouldn't cause this type of
> a problem.
>
> --
> Andrzej Taramina
> Chaeron Corporation: Enterprise System Solutions
> http://www.chaeron.com
>



-- 
Dmitriy Shabanov

[Exist-development] Very weird and serious error in trunk!

[Andrzej J. T. <an...@ch...>]

Wolfgang:

I've hit a very weird error in trunk, that could potentially be very serious.

I have some code in a module that looks like this:

declare function common:test()
{
    let $line := <property name="java.naming.security.credentials"
			   value="{ util:binary-to-string( xs:base64Binary( 'dGVzdAoK' ) ) }"/>

    let $debug2 := util:log-system-out( concat( "Credentials XML: ",
			util:serialize(  $line, "method=xml media-type=text/xml indent=yes" ) ) )

    return $line
};

declare function common:get-ldap-connection()  as xs:integer
{		
    let $test := common:test()

    let $contextProps :=
    <properties>
       <property name="java.naming.factory.initial" value="com.sun.jndi.ldap.LdapCtxFactory"/>
       <property name="java.naming.provider.url"
		 value="{ common:get-config()/config/ldap/url/text() }"/>
       <property name="java.naming.security.authentication" value="simple"/>
       <property name="java.naming.security.principal"
		 value="{ common:get-config()/config/ldap/adminUser/text() }"/>
       <property name="java.naming.security.credentials"
		 value="{ util:binary-to-string( xs:base64Binary(
				common:get-config()/config/ldap/adminPswd/text() ) ) }"/>
    </properties>

    let $ctx := jndi:get-dir-context( $contextProps )
	
    return $ctx
};

I then have a xquery that includes this module, and does something like this:

	let $cnx := common:get-ldap-connection()

All works fine if I call this from the sandbox and in the console log I will see the following line
produced by the common:test() function:

(Line: 494) Credentials XML: <property name="java.naming.security.credentials"
                                       value="test&#xA;&#xA;"/>


The problem I run into is that when I run this xquery from the scheduler (which is how it's intended
to run...it's a batch reporting job), cause then I get the following output in the console log:

(Line: 494) Credentials XML: <property name="java.naming.security.credentials" value="test&#xA;&#xA;
test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA;
test&#xA;&#xA;"/>

I get 8 concatenated decoded values in the value attribute!!!!

This should never happen!!!!  Of course, in that situation, I fail to get an LDAP connection since
the password (the credentials) is a concatenation of 8 copies of the actual password, and so fails,
if I remove the call to the test function.  If I leave in the call to the test function, then the
correct (non-replicated) password is inserted into the credentials element of the $contextProps
variable. If I remove the test function call, then I get the same replication in $contextProps.

What is weird is that the error only shows in the one place and is sensitive to order. If I put the
call to the test function after the construction of the $contextProps value, then $contextProps gets
the repeated value and the test function returns a correct, unrepeated value.

I've been trying to create a simple test case to replicate this, but have not yet been successful.
It seems I need to run the big batch job, where the included module is 3600 lines long, and the
actual xquery is very long and complex as well.

But the code snippet that causes this bug is static code in the common:test() function, and should
always return the correct result.

I'm not sure what is causing this.  Could it be the new Binary Value caching/handling under weird
conditions that only happen when running under a scheduled task?  I've never managed to reproduce
this when calling xqueries directly (the ldap connection call is made all over the place in our web
app for authentication purposes).  Maybe util:binary-to-string() is at fault, but that seems
unlikely.  Maybe some of your path wrapper stuff?

This exact same code runs just fine as a scheduled task in an older version of trunk (from late
summer last year), so it's definitely something new in trunk that has manifested since then.

Any ideas on what might cause something this strange?  I has me very worried since this is pretty
basic stuff....and thus might cause other things like DB corruption, or incorrect values being
written or wrong logic paths being followed.

I'll keep trying to create a simple test case that illustrates this, but I think it may not be
possible to do so.

Help!

Thanks!

-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

[Exist-development] And yet another security/Ant issue....addendum

[Andrzej J. T. <an...@ch...>]

BTW, I create the myuser like this (before I change the admin password from empty):

<exist:adduser failonerror="false" uri="${exist.url}/xmlrpc/db" user="admin" password=""
name="myuser" secret="${password}" home="/db" primaryGroup="dba"/>

That should make myuser a DBA, right?

Logging into the admin web page shows that the user is not a DBA, since it can't do a database
shutdown.  But the user list (both in the admin web page and in the Java client) shows myuser as a
member of the dba group.

....A

> I use an ant task to run an xquery as follows:
> 
> <exist:xquery queryfile="${temp.dir}/importData.xql" outputproperty="importResults"
> uri="${exist.url}/xmlrpc/db" user="myuser" password="${password}"> </exist:xquery>
> 
> but it the fails with the following:
> 
> BUILD FAILED
> /home/andrzej/build.xml:1747: XMLDB exception caught while executing query: Permission denied,
> calling user 'myuser' must be a DBA to call this function. [at line 75, column 23, source: String]
> 
> The line in the xquery that is failing is:
> 
> 	let $list   := file:directory-list( $dir, "*.xml" )
> 
> which requires DBA permissions as expected.
> 
> The problem is that the ant task was initiated using user "myuser" which is part of the dba
> group...and thus should be a DBA, right?
> 
> This used to work fine....but is now broken with the new security stuff.
> 
> Any idea why myuser isn't being seen as having dba permissions when executing an xquery using the
> ant task?


-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

[Exist-development] And yet another security/Ant issue....

[Andrzej J. T. <an...@ch...>]

I use an ant task to run an xquery as follows:

<exist:xquery queryfile="${temp.dir}/importData.xql" outputproperty="importResults"
uri="${exist.url}/xmlrpc/db" user="myuser" password="${password}"> </exist:xquery>

but it the fails with the following:

BUILD FAILED
/home/andrzej/build.xml:1747: XMLDB exception caught while executing query: Permission denied,
calling user 'myuser' must be a DBA to call this function. [at line 75, column 23, source: String]

The line in the xquery that is failing is:

	let $list   := file:directory-list( $dir, "*.xml" )

which requires DBA permissions as expected.

The problem is that the ant task was initiated using user "myuser" which is part of the dba
group...and thus should be a DBA, right?

This used to work fine....but is now broken with the new security stuff.

Any idea why myuser isn't being seen as having dba permissions when executing an xquery using the
ant task?

Thanks!

-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

[Exist-development] Another Ant-related security problem....

[Andrzej J. T. <an...@ch...>]

Dmitry:

If you try to create a user a 2nd time, using the ant task, then you get the following exception:

[exist:adduser] XMLDB exception caught: Failed to invoke method addAccount in class
org.exist.xmlrpc.RpcConnection: Configuration can't be used by <account name="myuser"
id="9"></user>, because allready in use by <account name="myuser" id="6" home="/db"></user>

Which is fine.  But then the password on the userid gets screwed up and you can't use that user any
more!  Which is not so fine.  Incidentally, the second addition is being added with the same
password as the first one, so nothing should change.

I think that adding a user that already exists shouldn't cause this type of a problem.

-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] [Teleconf] Any takers?

[Dmitriy S. <sha...@gm...>]

yes, I'm

On Fri, Apr 29, 2011 at 6:23 PM, Adam Retter <ad...@ex...> wrote:

> Could be, if you are available then it would be good to discuss...
>
> On 29 April 2011 15:18, Dmitriy Shabanov <sha...@gm...> wrote:
> > triggers? Security settings?
>  >
>

-- 
Dmitriy Shabanov

Re: [Exist-development] [Teleconf] Any takers?

[Adam R. <ad...@ex...>]

Could be, if you are available then it would be good to discuss...

On 29 April 2011 15:18, Dmitriy Shabanov <sha...@gm...> wrote:
> triggers? Security settings?
>
> On Fri, Apr 29, 2011 at 4:29 PM, Adam Retter <ad...@ex...> wrote:
>>
>> Does anyone want to speak, discuss a topic or ask questions in a
>> Teleconference this evening at 5pm CEST?
>
>
> --
> Dmitriy Shabanov
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] [Teleconf] Any takers?

[Dmitriy S. <sha...@gm...>]

triggers? Security settings?

On Fri, Apr 29, 2011 at 4:29 PM, Adam Retter <ad...@ex...> wrote:

> Does anyone want to speak, discuss a topic or ask questions in a
> Teleconference this evening at 5pm CEST?
>


-- 
Dmitriy Shabanov

[Exist-development] [Teleconf] Any takers?

[Adam R. <ad...@ex...>]

Does anyone want to speak, discuss a topic or ask questions in a
Teleconference this evening at 5pm CEST?

Thanks Adam.

-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] Ant task issues in changing admin user password.

[Andrzej J. T. <an...@ch...>]

>     > While doing this operation you authenticated on server as?
> 
>     A user with DBA group membership!  But it's not the "admin" user.
> 
> 
> That should be fine.

But it's not....I don't seem to be able to change the admin user's empty password to a new value
using the create user Ant task using dba-enabled credentials.

> There is the 'password' Ant task. Please, try it and let me know if any problem.

There is? It's not documented anywhere on the Ant Task page that I could see.  I agree that this
would be the better way to proceed, since the task is available.

I'll have to read the code to see what the format of the task is....we should update the
documentation to include this and any other implemented but not documented Ant Tasks.

-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] Security problem in trunk with Ant tasks....

[Andrzej J. T. <an...@ch...>]

Dmitriy:

> The fix is committed http://exist.svn.sourceforge.net/exist/?rev=14329&view=rev
> <http://exist.svn.sourceforge.net/exist/?rev=14329&view=rev>

Muchos gracias...that fixed the issue!

....A

> On Thu, Apr 28, 2011 at 8:00 PM, Andrzej Jan Taramina <an...@ch...
> <mailto:an...@ch...>> wrote:
> 
>     Dmitriy:
> 
>     > I did commit a fix http://exist.svn.sourceforge.net/exist/?rev=14325&view=rev
>     <http://exist.svn.sourceforge.net/exist/?rev=14325&view=rev>
>     > <http://exist.svn.sourceforge.net/exist/?rev=14325&view=rev
>     <http://exist.svn.sourceforge.net/exist/?rev=14325&view=rev>>
> 
>     There is another NPE problem if you use the store ant task with the createcollection attribute set
>     to true:
> 
>     <exist:store uri="${exist.url}/xmlrpc/db/system/config/db/mycollection" createcollection="true"
>     user="${userid}" password="${password}">
>                            <fileset dir="${src.index}">
>                                    <include name="collection.xconf"/>
>                            </fileset>
>                    </exist:store>
> 
>     If the collection specified in the URI exists already then I get an NPE:
> 
>     /home/andrzej/build.xml:1483: XMLDB exception caught: Failed to invoke method describeCollection in
>     class org.exist.xmlrpc.RpcConnection: java.lang.NullPointerException
> 
>     and the stack trace cause is:
> 
>     28 Apr 2011 10:51:24,894 ["http-bio-/127.0.0.1-80"-exec-1] ERROR (XmlRpcErrorLogger.java [log]:36) -
>     Failed to invoke method describeCollection in class org.exist.xmlrpc.RpcConnection:
>     java.lang.NullPointerException
>     org.apache.xmlrpc.common.XmlRpcInvocationException: Failed to invoke method describeCollection in
>     class org.exist.xmlrpc.RpcConnection: java.lang.NullPointerException
>     .....
>     Caused by: java.lang.NullPointerException
>            at org.exist.xmlrpc.RpcConnection.describeCollection(RpcConnection.java:727)
>            at org.exist.xmlrpc.RpcConnection.describeCollection(RpcConnection.java:695)
>            ... 44 more
> 
>     Seems there is still an issue in there somewhere!
> 
>     > BTW, do 'userid' registered at db?
> 
>     Yes! ${userid} is substituted for the real DBA userid by ant during the build. Everything we do
>     tends to be parametrized to allow us to deploy to different production sites easily.
> 
> 
> 
> -- 
> Dmitriy Shabanov


-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] Security problem in trunk with Ant tasks....

[Dmitriy S. <sha...@gm...>]

The fix is committed
http://exist.svn.sourceforge.net/exist/?rev=14329&view=rev

On Thu, Apr 28, 2011 at 8:00 PM, Andrzej Jan Taramina
<an...@ch...>wrote:

> Dmitriy:
>
> > I did commit a fix
> http://exist.svn.sourceforge.net/exist/?rev=14325&view=rev
> > <http://exist.svn.sourceforge.net/exist/?rev=14325&view=rev>
>
> There is another NPE problem if you use the store ant task with the
> createcollection attribute set
> to true:
>
> <exist:store uri="${exist.url}/xmlrpc/db/system/config/db/mycollection"
> createcollection="true"
> user="${userid}" password="${password}">
>                        <fileset dir="${src.index}">
>                                <include name="collection.xconf"/>
>                        </fileset>
>                </exist:store>
>
> If the collection specified in the URI exists already then I get an NPE:
>
> /home/andrzej/build.xml:1483: XMLDB exception caught: Failed to invoke
> method describeCollection in
> class org.exist.xmlrpc.RpcConnection: java.lang.NullPointerException
>
> and the stack trace cause is:
>
> 28 Apr 2011 10:51:24,894 ["http-bio-/127.0.0.1-80"-exec-1] ERROR
> (XmlRpcErrorLogger.java [log]:36) -
> Failed to invoke method describeCollection in class
> org.exist.xmlrpc.RpcConnection:
> java.lang.NullPointerException
> org.apache.xmlrpc.common.XmlRpcInvocationException: Failed to invoke method
> describeCollection in
> class org.exist.xmlrpc.RpcConnection: java.lang.NullPointerException
> .....
> Caused by: java.lang.NullPointerException
>         at
> org.exist.xmlrpc.RpcConnection.describeCollection(RpcConnection.java:727)
>        at
> org.exist.xmlrpc.RpcConnection.describeCollection(RpcConnection.java:695)
>        ... 44 more
>
> Seems there is still an issue in there somewhere!
>
> > BTW, do 'userid' registered at db?
>
> Yes! ${userid} is substituted for the real DBA userid by ant during the
> build. Everything we do
> tends to be parametrized to allow us to deploy to different production
> sites easily.
>


-- 
Dmitriy Shabanov

Re: [Exist-development] Problem with Security stuff....

[Dmitriy S. <sha...@gm...>]

On Thu, Apr 28, 2011 at 8:03 PM, Andrzej Jan Taramina
<an...@ch...>wrote:

> Dmitriy:
>
> > While doing this operation you authenticated on server as?
>
> A user with DBA group membership!  But it's not the "admin" user.
>

That should be fine.


>
> > BTW, use add to change looks wrong to me :-)
>
> Maybe so, but that is the only way you could do something like a password
> change using the ant tasks
> and it worked perfectly well in the past.
>
> I suppose we could write a changeUser Ant task.....but it would be better
> to stay consistent with
> how it behaved in the past.
>

There is the 'password' Ant task. Please, try it and let me know if any
problem.

-- 
Dmitriy Shabanov

Re: [Exist-development] Getting lots of optimizer warnings in log....

[Andrzej J. T. <an...@ch...>]

Wolfgang:

> ok, I change this to write to the trace log only. However, the message
> indeed points to an optimization issue which I introduced myself
> recently by removing some PathExpr wrappers, so thanks for reporting
> the warning. I need to fix the underlying problem.

Super...thanks!

The "canary in the coalmine" is back and at it again!  LOL

-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] Problem with Security stuff....

[Andrzej J. T. <an...@ch...>]

Dmitriy:

> While doing this operation you authenticated on server as?

A user with DBA group membership!  But it's not the "admin" user.

> BTW, use add to change looks wrong to me :-)

Maybe so, but that is the only way you could do something like a password change using the ant tasks
and it worked perfectly well in the past.

I suppose we could write a changeUser Ant task.....but it would be better to stay consistent with
how it behaved in the past.

> On Thu, Apr 28, 2011 at 12:53 AM, Andrzej Jan Taramina <an...@ch...> wrote:
> 
>     It used to be that you could change the admin user's password using an ant task of the following
>     structure:
> 
>     <exist:adduser failonerror="false" uri="${exist.url}/xmlrpc/db" user="anotherAdminUser"
>     password="anotherAdminUserPassword" name="admin" secret="newAdminPassword" home="/db"
>     primaryGroup="dba"/>
> 
>     Now it errors with the following message:
> 
>     [exist:adduser] XMLDB exception caught: Failed to invoke method addAccount in class
>     org.exist.xmlrpc.RpcConnection: you are not allowed to change this user
> 
>     Why is it that we can no longer change the admin password in this way?  Leaving the admin password
>     unset is a bad idea, from a security perspective, and some of us want to be able to automate this
>     reset using Ant, as we've done in the past.


-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] Security problem in trunk with Ant tasks....

[Andrzej J. T. <an...@ch...>]

Dmitriy:

> I did commit a fix http://exist.svn.sourceforge.net/exist/?rev=14325&view=rev
> <http://exist.svn.sourceforge.net/exist/?rev=14325&view=rev>

There is another NPE problem if you use the store ant task with the createcollection attribute set
to true:

<exist:store uri="${exist.url}/xmlrpc/db/system/config/db/mycollection" createcollection="true"
user="${userid}" password="${password}">
			<fileset dir="${src.index}">
				<include name="collection.xconf"/>
			</fileset>
		</exist:store>

If the collection specified in the URI exists already then I get an NPE:

/home/andrzej/build.xml:1483: XMLDB exception caught: Failed to invoke method describeCollection in
class org.exist.xmlrpc.RpcConnection: java.lang.NullPointerException

and the stack trace cause is:

28 Apr 2011 10:51:24,894 ["http-bio-/127.0.0.1-80"-exec-1] ERROR (XmlRpcErrorLogger.java [log]:36) -
Failed to invoke method describeCollection in class org.exist.xmlrpc.RpcConnection:
java.lang.NullPointerException
org.apache.xmlrpc.common.XmlRpcInvocationException: Failed to invoke method describeCollection in
class org.exist.xmlrpc.RpcConnection: java.lang.NullPointerException
.....
Caused by: java.lang.NullPointerException
	at org.exist.xmlrpc.RpcConnection.describeCollection(RpcConnection.java:727)
	at org.exist.xmlrpc.RpcConnection.describeCollection(RpcConnection.java:695)
	... 44 more

Seems there is still an issue in there somewhere!

> BTW, do 'userid' registered at db?

Yes! ${userid} is substituted for the real DBA userid by ant during the build. Everything we do
tends to be parametrized to allow us to deploy to different production sites easily.

-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] Security problem in trunk with Ant tasks....

[Dmitriy S. <sha...@gm...>]

I did commit a fix
http://exist.svn.sourceforge.net/exist/?rev=14325&view=rev

BTW, do 'userid' registered at db?

On Thu, Apr 28, 2011 at 9:44 AM, Andrzej Jan Taramina
<an...@ch...>wrote:

> When I try to create a collection that already exists using the eXist Ant
> task:
>
>        <exist:create uri="${exist.url}/xmlrpc/db" collection="mycollection"
> user="${userid}"
>                      password="${password}"/>
>
> I get a NPE something like this:
>
> 28 Apr 2011 00:29:16,688 ["http-bio-/127.0.0.1-80"-exec-7] ERROR
> (XmlRpcErrorLogger.java [log]:36) -
> Failed to invoke method describeCollection in class
> org.exist.xmlrpc.RpcConnection:
> java.lang.NullPointerException
>
> or this:
>
> 28 Apr 2011 00:27:09,666 ["http-bio-/127.0.0.1-80"-exec-4] ERROR
> (XmlRpcErrorLogger.java [log]:36) -
> Failed to invoke method createCollection in class
> org.exist.xmlrpc.RpcConnection:
> java.lang.NullPointerException
>
> and further down in the stack trace, the culprit seems to be:
>
> Caused by: java.lang.NullPointerException
>        at
> org.exist.security.UnixStylePermission.write(UnixStylePermission.java:448)
>        at org.exist.collections.Collection.write(Collection.java:1514)
>        at
> org.exist.storage.NativeBroker.saveCollection(NativeBroker.java:1268)
>        at
> org.exist.xmlrpc.RpcConnection.createCollection(RpcConnection.java:175)
>
>
> which is this line in org.exist.security.UnixStylePermission.write():
>
>        ostream.writeInt(getOwner().getId());
>
> so either the owner or the id is null.
>
> In the past, if you tried to create a collection that was already there, it
> was a no-op....which was
> convenient for when reloading stuff into the database using Ant, so I don't
> think this should be
> happening!
>
> Help please!
>
> PS...told ya security wasn't yet totally stable....figures I find all the
> edge-cases, eh? ;-)
>
> --
> Andrzej Taramina
> Chaeron Corporation: Enterprise System Solutions
> http://www.chaeron.com
>
>
> ------------------------------------------------------------------------------
> WhatsUp Gold - Download Free Network Management Software
> The most intuitive, comprehensive, and cost-effective network
> management toolset available today.  Delivers lowest initial
> acquisition cost and overall TCO of any competing solution.
> http://p.sf.net/sfu/whatsupgold-sd
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>



-- 
Dmitriy Shabanov

Re: [Exist-development] Problem with Security stuff....

[Dmitriy S. <sha...@gm...>]

While doing this operation you authenticated on server as?

BTW, use add to change looks wrong to me :-)

On Thu, Apr 28, 2011 at 12:53 AM, Andrzej Jan Taramina
<an...@ch...>wrote:

> It used to be that you could change the admin user's password using an ant
> task of the following
> structure:
>
> <exist:adduser failonerror="false" uri="${exist.url}/xmlrpc/db"
> user="anotherAdminUser"
> password="anotherAdminUserPassword" name="admin" secret="newAdminPassword"
> home="/db"
> primaryGroup="dba"/>
>
> Now it errors with the following message:
>
> [exist:adduser] XMLDB exception caught: Failed to invoke method addAccount
> in class
> org.exist.xmlrpc.RpcConnection: you are not allowed to change this user
>
> Why is it that we can no longer change the admin password in this way?
>  Leaving the admin password
> unset is a bad idea, from a security perspective, and some of us want to be
> able to automate this
> reset using Ant, as we've done in the past.
>

-- 
Dmitriy Shabanov