Re: [Exist-development] Very weird and serious error in trunk!

[Andrzej J. T. <an...@ch...>]

Still happening with the latest fixes (BTree caching)....so it's not related to that.

....A

> More testing on this issue:
> 
> I changed the test function to not use the Binary stuff, so it looks like this:
> 
> declare function common:test()
> {
>     let $line := <property name="java.naming.security.credentials"
> 	                   value="{ xs:string( 'dGVzdA==' ) }"/>
> 
>     let $debug2 := util:log-system-out( concat( "Credentials XML String: ",
> 		       util:serialize(  $line, "method=xml media-type=text/xml indent=yes" ) ) )
> 
>     return $line
> };
> 
> And I still get the erroneous result, with 8 repeated values:
> 
> (Line: 494) Credentials XML String: <property name="java.naming.security.credentials"
>             value="dGVzdA== dGVzdA== dGVzdA== dGVzdA== dGVzdA== dGVzdA== dGVzdA== dGVzdA=="/>
> 
> 
> So that means your Binary stuff is off the hook, Adam!  The problem is somewhere else....maybe in
> the query parser or execution engine.
> 
> Changing the line that creates the xml snippet to:
> 
>    let $line := <property name="java.naming.security.credentials"
> 	                  value="{ let $tmp := 'dGVzdA==' return $tmp }"/>
> 
> shows the same issue....8 repeated strings, delimited with a space for the @value attribute.
> 
> But this variant does NOT repeat the values!
> 
> 	let $tmp := 'dGVzdA=='
> 
> 	let $line := <property name="java.naming.security.credentials" value="{ $tmp }"/>
> 
> and neither does this one:
> 
> 	let $tmp := 'dGVzdA=='
> 
>     	let $line := <property name="java.naming.security.credentials"
> 			       value="{ xs:string( 'dGVzdA==' ) }"/>
> 
> Which leads me to believe that the bug is partially position sensitive.  If the creation of the xml
> snippet is the first line of code in the function, then the values get replicated 8 times.  If there
> is any code prior to that line, then it does not replicate.
> 
> I then changed the test function to take a parameter (the string being replicated) and changed the
> calling function to pass in the string:
> 
> So this version:
> 
> declare function common:test( $tmp )
> {
>     let $line := <property name="java.naming.security.credentials" value="{ xs:string( $tmp ) }"/>
> 
>     .....
> };
> 
> replicates the attribute value 8 times (showing the bug), but this one:
> 
> declare function common:test( $tmp )
> {
>     let $line := <property name="java.naming.security.credentials" value="{ $tmp }"/>
> 
>     .....
> };
> 
> does not replicate (no bug!).
> 
> This leads me to believe that the bug is somewhere in how code fragments inside attribute values are
> evaluated.
> 
> Not being very familiar with the xquery parser nor evaluation engine, I can't easily dive into the
> code (which is very complex) to figure out what is going on. I'm hoping that all of these different
> test cases might twig something with you, Wolfgang, since creating a simple test case to replicate
> this is proving difficult.
> 
> .....Andrzej
> 
> 
>> On 30 April 2011 00:55, Andrzej Jan Taramina <an...@ch...> wrote:
>>> Wolfgang:
>>>
>>> I've hit a very weird error in trunk, that could potentially be very serious.
>>>
>>> I have some code in a module that looks like this:
>>>
>>> declare function common:test()
>>> {
>>>    let $line := <property name="java.naming.security.credentials"
>>>                           value="{ util:binary-to-string( xs:base64Binary( 'dGVzdAoK' ) ) }"/>
>>>
>>>    let $debug2 := util:log-system-out( concat( "Credentials XML: ",
>>>                        util:serialize(  $line, "method=xml media-type=text/xml indent=yes" ) ) )
>>>
>>>    return $line
>>> };
>>>
>>> declare function common:get-ldap-connection()  as xs:integer
>>> {
>>>    let $test := common:test()
>>>
>>>    let $contextProps :=
>>>    <properties>
>>>       <property name="java.naming.factory.initial" value="com.sun.jndi.ldap.LdapCtxFactory"/>
>>>       <property name="java.naming.provider.url"
>>>                 value="{ common:get-config()/config/ldap/url/text() }"/>
>>>       <property name="java.naming.security.authentication" value="simple"/>
>>>       <property name="java.naming.security.principal"
>>>                 value="{ common:get-config()/config/ldap/adminUser/text() }"/>
>>>       <property name="java.naming.security.credentials"
>>>                 value="{ util:binary-to-string( xs:base64Binary(
>>>                                common:get-config()/config/ldap/adminPswd/text() ) ) }"/>
>>>    </properties>
>>>
>>>    let $ctx := jndi:get-dir-context( $contextProps )
>>>
>>>    return $ctx
>>> };
>>>
>>> I then have a xquery that includes this module, and does something like this:
>>>
>>>        let $cnx := common:get-ldap-connection()
>>>
>>> All works fine if I call this from the sandbox and in the console log I will see the following line
>>> produced by the common:test() function:
>>>
>>> (Line: 494) Credentials XML: <property name="java.naming.security.credentials"
>>>                                       value="test&#xA;&#xA;"/>
>>>
>>>
>>> The problem I run into is that when I run this xquery from the scheduler (which is how it's intended
>>> to run...it's a batch reporting job), cause then I get the following output in the console log:
>>>
>>> (Line: 494) Credentials XML: <property name="java.naming.security.credentials" value="test&#xA;&#xA;
>>> test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA; test&#xA;&#xA;
>>> test&#xA;&#xA;"/>
>>>
>>> I get 8 concatenated decoded values in the value attribute!!!!
>>>
>>> This should never happen!!!!  Of course, in that situation, I fail to get an LDAP connection since
>>> the password (the credentials) is a concatenation of 8 copies of the actual password, and so fails,
>>> if I remove the call to the test function.  If I leave in the call to the test function, then the
>>> correct (non-replicated) password is inserted into the credentials element of the $contextProps
>>> variable. If I remove the test function call, then I get the same replication in $contextProps.
>>>
>>> What is weird is that the error only shows in the one place and is sensitive to order. If I put the
>>> call to the test function after the construction of the $contextProps value, then $contextProps gets
>>> the repeated value and the test function returns a correct, unrepeated value.
>>>
>>> I've been trying to create a simple test case to replicate this, but have not yet been successful.
>>> It seems I need to run the big batch job, where the included module is 3600 lines long, and the
>>> actual xquery is very long and complex as well.
>>>
>>> But the code snippet that causes this bug is static code in the common:test() function, and should
>>> always return the correct result.
>>>
>>> I'm not sure what is causing this.  Could it be the new Binary Value caching/handling under weird
>>> conditions that only happen when running under a scheduled task?  I've never managed to reproduce
>>> this when calling xqueries directly (the ldap connection call is made all over the place in our web
>>> app for authentication purposes).  Maybe util:binary-to-string() is at fault, but that seems
>>> unlikely.  Maybe some of your path wrapper stuff?
>>>
>>> This exact same code runs just fine as a scheduled task in an older version of trunk (from late
>>> summer last year), so it's definitely something new in trunk that has manifested since then.
>>>
>>> Any ideas on what might cause something this strange?  I has me very worried since this is pretty
>>> basic stuff....and thus might cause other things like DB corruption, or incorrect values being
>>> written or wrong logic paths being followed.
>>>
>>> I'll keep trying to create a simple test case that illustrates this, but I think it may not be
>>> possible to do so.
>>>
>>> Help!
>>>
>>> Thanks!
>>>
>>> --
>>> Andrzej Taramina
>>> Chaeron Corporation: Enterprise System Solutions
>>> http://www.chaeron.com
>>>
>>
>>
>>
> 
> 


-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com