python-ldap-dev

Referrals...

[Michael <mi...@st...>]

HI!

Back to a python-ldap build with OpenLDAP 1.2.11 libs:

It seems that referral exceptions are not raised if doing an
asynchronous search call. In the example below the entry
ldap://localhost:1389/c=DE is a referral entry holding the ref
attribute ldap://ldap.nameflow.net:1389. l is my LDAP connection
object.

>>> l.options=0
>>> l.search_s('c=DE',ldap.SCOPE_ONELEVEL,'(objectclass=*)')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ldap.PARTIAL_RESULTS: {'desc': 'Partial results and referral
received', 'info':
'Referral:\012ldap://ldap.nameflow.net:1389/c=DE', 'matched':
'c=DE'}

This seems to be ok.

>>> m=l.search('c=DE',ldap.SCOPE_ONELEVEL,'(objectclass=*)')
>>> l.result(m,0)
('RES_SEARCH_RESULT', [])

This seems not ok to me. Shouldn't there be an exception
ldap.PARTIAL_RESULTS raised? My OpenLDAP 2.0.6 slapd logs that a
referral occured immediately after invoking the l.search() method.

Ciao, Michael.

Re: Referrals...

[Konstantin C. <Kon...@da...>]

Hi All,

I seem to have found the bug (Michael, this time I'm quite sure ;-)

The problem is that python-ldap assumes ldap_result() returns, like
ldap_search() and many other C LDAP API functions, an LDAP error code,
which is wrong, according to the API. ldap_result() always returns the
type of the result retrieved, which is a constant like
LDAP_RES_SERARCH_ENTRY, LDAP_RES_SEARCH_RESULT, LDAP_RES_BIND_RESULT,
LDAP_RES_COMPARE_RESULT etc. To check if there was an LDAP error while
processing an asynchronous request, one needs to call int
ldap_result2error(LDAP *ld, LDAPMessage *res, int freeit) after
ldap_result(). python-ldap's static PyObject *l_ldap_result() does not
do it.
A possible solution would be the following change in
python-ldap-*/Modules/LDAPObject.c, function static PyObject
*l_ldap_result( LDAPObject* self, PyObject *args ):

     LDAP_BEGIN_ALLOW_THREADS( self );
     result = ldap_result( self->ldap, msgid, all, tvp, &msg );
     LDAP_END_ALLOW_THREADS( self );

for

     int result_type;
     ...
     LDAP_BEGIN_ALLOW_THREADS( self );
     result_type = ldap_result( self->ldap, msgid, all, tvp, &msg
     );
     result = ldap_result2error( self->ldap, msg, 0 );
     LDAP_END_ALLOW_THREADS( self );

... but the drawback is that in this case the result type variable is
getting hidden to a python caller.

Any ideas?

Michael Ströder wrote:

> HI!
>
> Back to a python-ldap build with OpenLDAP 1.2.11 libs:
>
> It seems that referral exceptions are not raised if doing an
> asynchronous search call. In the example below the entry
> ldap://localhost:1389/c=DE is a referral entry holding the ref
> attribute ldap://ldap.nameflow.net:1389. l is my LDAP connection
> object.
>
> >>> l.options=0
> >>> l.search_s('c=DE',ldap.SCOPE_ONELEVEL,'(objectclass=*)')
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> ldap.PARTIAL_RESULTS: {'desc': 'Partial results and referral
> received', 'info':
> 'Referral:\012ldap://ldap.nameflow.net:1389/c=DE', 'matched':
> 'c=DE'}
>
> This seems to be ok.
>
> >>> m=l.search('c=DE',ldap.SCOPE_ONELEVEL,'(objectclass=*)')
> >>> l.result(m,0)
> ('RES_SEARCH_RESULT', [])
>
> This seems not ok to me. Shouldn't there be an exception
> ldap.PARTIAL_RESULTS raised? My OpenLDAP 2.0.6 slapd logs that a
> referral occured immediately after invoking the l.search() method.
>
> Ciao, Michael.

--
          * *        Konstantin Chuguev - Application Engineer
       *      *              Francis House, 112 Hills Road
     *                       Cambridge CB2 1PQ, United Kingdom
 D  A  N  T  E       WWW:    http://www.dante.net

Re: Referrals...

[Michael <mi...@st...>]

Konstantin Chuguev wrote:
> 
> The problem is that python-ldap assumes ldap_result() returns, like
> ldap_search() and many other C LDAP API functions, an LDAP error code,
> which is wrong, according to the API.
> [..]
> To check if there was an LDAP error while
> processing an asynchronous request, one needs to call int
> ldap_result2error(LDAP *ld, LDAPMessage *res, int freeit) after
> ldap_result().

You're right.

>      result_type = ldap_result( self->ldap, msgid, all, tvp, &msg
>      );
>      result = ldap_result2error( self->ldap, msg, 0 );
> 
> ... but the drawback is that in this case the result type variable is
> getting hidden to a python caller.

No drawback since we have exceptions! :-) Look at the lines
following the ldap_result() call:

    if (result == -1)
        return LDAPerror( self->ldap, "ldap_result" );

etc.

I think this should raise an exception. IMHO David planned to do it
right but forgot to call ldap_result2error() function.

Ciao, Michael.

Re: Referrals...

[Michael <mi...@st...>]

Konstantin Chuguev wrote:
> 
>      LDAP_BEGIN_ALLOW_THREADS( self );
>      result_type = ldap_result( self->ldap, msgid, all, tvp, &msg
>      );
>      result = ldap_result2error( self->ldap, msg, 0 );
>      LDAP_END_ALLOW_THREADS( self );

Hmm, my slapd logs the occurence of the referral right after calling
the search() method of the LDAP object. I guess the function
ldap_result2error() should be called in l_ldap_search() either.

Ciao, Michael.

Re: Referrals...

[Konstantin C. <Kon...@da...>]

Michael Ströder wrote:

> Konstantin Chuguev wrote:
> >
> >      LDAP_BEGIN_ALLOW_THREADS( self );
> >      result_type = ldap_result( self->ldap, msgid, all, tvp, &msg
> >      );
> >      result = ldap_result2error( self->ldap, msg, 0 );
> >      LDAP_END_ALLOW_THREADS( self );
>
> Hmm, my slapd logs the occurence of the referral right after calling
> the search() method of the LDAP object. I guess the function
> ldap_result2error() should be called in l_ldap_search() either.
>

I think, this just means the LDAP server could answer immediately
(because of this low load and fast database). The result was probably
sent to the client and was caught by the LDAP library in some buffers.
You could issue other search requests and then start collecting results
of all the requests one by one with ldap_result() until LDAP library
buffers are empty. This is how I understand the asynchronous mode.

The l_ldap_result() python-ldap function is correct in the sense that it
just wraps the corresponding ldap_result() C function, no more no less.
Exception is raised only in case of LDAP or system error. But then msg
is converted to Python format (dictionary) and its C representation is
destroyed before exiting the function. It's not possible anymore to
provide data for ldap_result2error, even if there was a Python wrapper
for it. So the only solution I see is combining both C functions -
ldap_result() and ldap_result2error() in one wrapper.
No return data is lost because of Python exceptions.

I've attached a patch for l_ldap_result function.
Note that now the function raises exceptions both in case of an LDAP or
system error AND an unsuccessful LDAP operation result (including
LDAP_PARTIAL). It returns Py_None object if there was no result at all
(i.e. timeout has occured or no async requests were issued to the server
before the call).

--
          * *        Konstantin Chuguev - Application Engineer
       *      *              Francis House, 112 Hills Road
     *                       Cambridge CB2 1PQ, United Kingdom
 D  A  N  T  E       WWW:    http://www.dante.net