python-ldap-dev — discussion among developers

Re: more memory leakage

[David M. <da...@es...>]

looks like you've got it.  I've tested this change, doing thousands of searches and
my process doesn't grow in size by one byte (where as before, it grew to 20 MB
after about 7k searches)

also, that problem I mentioned earlier with ldap.delete_s not working on referrals
(though it returned success!) was an openldap bug, that is fixed in 1.2.10 or
possibly a bit earlier.



David Leonard wrote:

> how does this look?
>
> --- message.c   1999/09/19 01:53:21     1.7
> +++ message.c   2000/07/04 00:48:09
> @@ -45,6 +45,8 @@
>  #endif /* !CIDICT */
>
>          PyTuple_SetItem( entrytuple, 0, PyString_FromString(dn) );
> +        free(dn);
> +
>          PyTuple_SetItem( entrytuple, 1, attrdict );
>
>          PyList_SetItem( result, entry_index, entrytuple );
> @@ -64,7 +66,6 @@
>                  PyMapping_SetItemString( attrdict, attr, valuelist );
>              }
>
> -            Py_INCREF( valuelist );
>              if (bvals != NULL) {
>                 int i;
>                 for (i=0; bvals[i]; i++) {
> @@ -74,10 +75,10 @@
>                             bvals[i]->bv_val, bvals[i]->bv_len
>                         );
>                     PyList_Append( valuelist, valuestr );
> +                   Py_DECREF(valuestr);
>                 }
>                 ber_bvecfree(bvals);
>              }
> -            Py_DECREF( valuelist );
>          }
>       }
>       ldap_msgfree( m );
> On Mon, 3 Jul 2000, David Leonard typed thusly:
>
> >
> > these look great! i will have to verify them..
> > some C functions incref their args for you, and others don't. there are
> > docs on which are which.
> >
> > i think maybe i should convert things like PyList_Append to PySequence_SetItem
> > (with an appropriately allocated List). apparently they all have consistent
> > incref semantics and its much easier to visually check..
> >
> > having problems with cvs at the moment...
> >
> > 209.81.8.43: Connection timed out
> >
> > On Mon, 3 Jul 2000, David Margrave typed thusly:
> >
> > > line 80 on should be:
> > >
> > >             Py_DECREF( valuelist );
> > >         }
> > >         free(dn);
> > >     }
> > >     ldap_msgfree( m );
> > >     return result;
> > > }
> > >
> > > this helps a bit, but something is still leaking memory like
> > > crazy elsewhere.
> > >
> > > I'm not  able to do CVS just this moment, so could you add
> > > that in?
> >
> >
> > > Also in message.c, I added a Py_DECREF for valuestr:
> > >
> > >                 for (i=0; bvals[i]; i++) {
> > >                     PyObject *valuestr;
> > >
> > >                     valuestr = PyString_FromStringAndSize(
> > >                             bvals[i]->bv_val, bvals[i]->bv_len
> > >                         );
> > >                     PyList_Append( valuelist, valuestr );
> > >                     Py_DECREF(valuestr);
> > >                 }
> > >
> > > I'm not an expert on the python reference counting system, but adding this
> > > one has cleared up the memory leak (on openldap 1.2.10, compiled without
> > > caching and without threads.  Haven't tried with yet..)  I did try adding
> > > Py_DECREF in a couple other places and got core dumps.  For some reason,
> > > it seems to be valid to do it here.  Perhaps because the object in
> > > question is not a mapping or sequence?  Anyhow, memory consumption on my
> > > program that does thousands of searches no longer goes up monotonically!
> > >
> > > Dave
> > >
> >
> >
>
> --
> David Leonard                           Dav...@cs...
> Dept of Comp. Sci. and Elec. Engg   _   Room:78-640  Ph:+61 7 336 51187
> The University of Queensland       |+|  http://www.csee.uq.edu.au/~leonard/
> QLD 4072  AUSTRALIA               ~` '~ E2A24DC6446E5779D7AFC41AA04E6401

Re: more memory leakage

[David L. <dav...@cs...>]

how does this look?

--- message.c	1999/09/19 01:53:21	1.7
+++ message.c	2000/07/04 00:48:09
@@ -45,6 +45,8 @@
 #endif /* !CIDICT */
 
 	 PyTuple_SetItem( entrytuple, 0, PyString_FromString(dn) );
+	 free(dn);
+
 	 PyTuple_SetItem( entrytuple, 1, attrdict );
 
 	 PyList_SetItem( result, entry_index, entrytuple );
@@ -64,7 +66,6 @@
 		 PyMapping_SetItemString( attrdict, attr, valuelist );
 	     }
 
-	     Py_INCREF( valuelist );
 	     if (bvals != NULL) {
 	        int i;
 		for (i=0; bvals[i]; i++) {
@@ -74,10 +75,10 @@
 			    bvals[i]->bv_val, bvals[i]->bv_len 
 			);
 		    PyList_Append( valuelist, valuestr );
+		    Py_DECREF(valuestr);
 	    	}
 	     	ber_bvecfree(bvals);
 	     }
-	     Py_DECREF( valuelist );
 	 }
      }
      ldap_msgfree( m );
On Mon, 3 Jul 2000, David Leonard typed thusly:

> 
> these look great! i will have to verify them..
> some C functions incref their args for you, and others don't. there are
> docs on which are which.
> 
> i think maybe i should convert things like PyList_Append to PySequence_SetItem
> (with an appropriately allocated List). apparently they all have consistent
> incref semantics and its much easier to visually check..
> 
> having problems with cvs at the moment...
> 
> 209.81.8.43: Connection timed out
> 
> On Mon, 3 Jul 2000, David Margrave typed thusly:
> 
> > line 80 on should be:
> > 
> >             Py_DECREF( valuelist );
> >         }
> >         free(dn);
> >     }
> >     ldap_msgfree( m );
> >     return result;
> > }
> > 
> > this helps a bit, but something is still leaking memory like
> > crazy elsewhere.
> > 
> > I'm not  able to do CVS just this moment, so could you add
> > that in?
> 
> 
> > Also in message.c, I added a Py_DECREF for valuestr:
> > 
> >                 for (i=0; bvals[i]; i++) {
> >                     PyObject *valuestr;
> > 
> >                     valuestr = PyString_FromStringAndSize(
> >                             bvals[i]->bv_val, bvals[i]->bv_len
> >                         );
> >                     PyList_Append( valuelist, valuestr );
> >                     Py_DECREF(valuestr);
> >                 }
> > 
> > I'm not an expert on the python reference counting system, but adding this
> > one has cleared up the memory leak (on openldap 1.2.10, compiled without 
> > caching and without threads.  Haven't tried with yet..)  I did try adding
> > Py_DECREF in a couple other places and got core dumps.  For some reason,
> > it seems to be valid to do it here.  Perhaps because the object in
> > question is not a mapping or sequence?  Anyhow, memory consumption on my
> > program that does thousands of searches no longer goes up monotonically!
> > 
> > Dave
> > 
> 
> 

-- 
David Leonard                           Dav...@cs...
Dept of Comp. Sci. and Elec. Engg   _   Room:78-640  Ph:+61 7 336 51187
The University of Queensland       |+|  http://www.csee.uq.edu.au/~leonard/
QLD 4072  AUSTRALIA               ~` '~ E2A24DC6446E5779D7AFC41AA04E6401

Re: more memory leakage

[David L. <dav...@cs...>]

these look great! i will have to verify them..
some C functions incref their args for you, and others don't. there are
docs on which are which.

i think maybe i should convert things like PyList_Append to PySequence_SetItem
(with an appropriately allocated List). apparently they all have consistent
incref semantics and its much easier to visually check..

having problems with cvs at the moment...

209.81.8.43: Connection timed out

On Mon, 3 Jul 2000, David Margrave typed thusly:

> line 80 on should be:
> 
>             Py_DECREF( valuelist );
>         }
>         free(dn);
>     }
>     ldap_msgfree( m );
>     return result;
> }
> 
> this helps a bit, but something is still leaking memory like
> crazy elsewhere.
> 
> I'm not  able to do CVS just this moment, so could you add
> that in?


> Also in message.c, I added a Py_DECREF for valuestr:
> 
>                 for (i=0; bvals[i]; i++) {
>                     PyObject *valuestr;
> 
>                     valuestr = PyString_FromStringAndSize(
>                             bvals[i]->bv_val, bvals[i]->bv_len
>                         );
>                     PyList_Append( valuelist, valuestr );
>                     Py_DECREF(valuestr);
>                 }
> 
> I'm not an expert on the python reference counting system, but adding this
> one has cleared up the memory leak (on openldap 1.2.10, compiled without 
> caching and without threads.  Haven't tried with yet..)  I did try adding
> Py_DECREF in a couple other places and got core dumps.  For some reason,
> it seems to be valid to do it here.  Perhaps because the object in
> question is not a mapping or sequence?  Anyhow, memory consumption on my
> program that does thousands of searches no longer goes up monotonically!
> 
> Dave
> 

-- 
David Leonard                           Dav...@cs...
Dept of Comp. Sci. and Elec. Engg   _   Room:78-640  Ph:+61 7 336 51187
The University of Queensland       |+|  http://www.csee.uq.edu.au/~leonard/
QLD 4072  AUSTRALIA               ~` '~ E2A24DC6446E5779D7AFC41AA04E6401

Curses! - Mojo Jojo

Re: ldap-module objects pickeable?

[Michael <mi...@st...>]

David Leonard wrote:
> 
> On Sat, 1 Jul 2000, Michael Ströder typed thusly:
> 
> > Are LDAP connection objects pickeable?
> >
> no, but it couldnt be too hard to write a wrapper object that is

Hmm, but the wrapper object could not contain the LDAP cache.

Ciao, Michael.

Re: ldap-module objects pickeable?

[David L. <dav...@cs...>]

no, but it couldnt be too hard to write a wrapper object that is

On Sat, 1 Jul 2000, Michael Ströder typed thusly:

> HI!
> 
> Are LDAP connection objects pickeable?
> 
> Ciao, Michael.
> 
> _______________________________________________
> Python-LDAP-dev mailing list
> Pyt...@li...
> http://lists.sourceforge.net/mailman/listinfo/python-ldap-dev
> 

-- 
David Leonard                           Dav...@cs...
Dept of Comp. Sci. and Elec. Engg   _   Room:78-640  Ph:+61 7 336 51187
The University of Queensland       |+|  http://www.csee.uq.edu.au/~leonard/
QLD 4072  AUSTRALIA               ~` '~ E2A24DC6446E5779D7AFC41AA04E6401

Curses! - Mojo Jojo

ldap-module objects pickeable?

[Michael <mi...@st...>]

HI!

Are LDAP connection objects pickeable?

Ciao, Michael.

Re: python-ldap, Zope, and NDS

[Michael <mi...@st...>]

Timothy Wilson wrote:
> 
> On Sun, 25 Jun 2000, Michael [iso-8859-1] Ströder wrote:
> 
> > > Python 1.5.2 (#0, Apr  3 2000, 14:46:48)  [GCC 2.95.2 20000313 (Debian
> > > GNU/Linux)] on linux2
> > > Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
> > > >>> import ldap
> > > >>> l = ldap.open("206.131.108.2",389)
> > > >>> l.simple_bind_s("ou=do,o=isd_197","")
> >                        ^^^^^^^^^^^^^^^^^^^
> > Please read the doc of ldapmodule again. The DN is meant for binding
> > as this object (e.g. an entry describing a user) and is used for
> > authentication/authorization. If doing a anonymous bind you can
> > simply use an empty DN.
> 
> Aha! Changing to l.simple_bind_s("", "") makes everything work fine

Well, LDAP hosts should assume anonymous bind if password is empty.
So it's a little bit of a NDS issue.

> (At
> least as far as my simple example goes.) I will go through the source for
> ZLDAP and try to determine how it's being done there.

If they put a bind-dn in a simple_bind() call although you did not
tell the module doing so it's definitely a bug in ZLDAP.

Ciao, Michael.

Re: python-ldap, Zope, and NDS

[Timothy W. <wi...@vi...>]

On Sun, 25 Jun 2000, Michael [iso-8859-1] Ströder wrote:

> > Python 1.5.2 (#0, Apr  3 2000, 14:46:48)  [GCC 2.95.2 20000313 (Debian
> > GNU/Linux)] on linux2
> > Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
> > >>> import ldap
> > >>> l = ldap.open("206.131.108.2",389)
> > >>> l.simple_bind_s("ou=do,o=isd_197","")
>                        ^^^^^^^^^^^^^^^^^^^
> Please read the doc of ldapmodule again. The DN is meant for binding
> as this object (e.g. an entry describing a user) and is used for
> authentication/authorization. If doing a anonymous bind you can
> simply use an empty DN.

Aha! Changing to l.simple_bind_s("", "") makes everything work fine. (At
least as far as my simple example goes.) I will go through the source for
ZLDAP and try to determine how it's being done there.

-Tim

--
Tim Wilson      | Visit Sibley online:         | Check out:
Henry Sibley HS | http://www.isd197.k12.mn.us/ | http://www.zope.org/
W. St. Paul, MN |                              | http://slashdot.org/
wi...@vi... |   <dtml-var pithy_quote>     | http://linux.com/

Re: python-ldap, Zope, and NDS

[Michael <mi...@st...>]

Timothy Wilson wrote:
> 
> Python 1.5.2 (#0, Apr  3 2000, 14:46:48)  [GCC 2.95.2 20000313 (Debian
> GNU/Linux)] on linux2
> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
> >>> import ldap
> >>> l = ldap.open("206.131.108.2",389)
> >>> l.simple_bind_s("ou=do,o=isd_197","")
                       ^^^^^^^^^^^^^^^^^^^
Please read the doc of ldapmodule again. The DN is meant for binding
as this object (e.g. an entry describing a user) and is used for
authentication/authorization. If doing a anonymous bind you can
simply use an empty DN.

> >>> test = l.search_s("ou=do,o=isd_197", ldap.SCOPE_SUBTREE,
> "objectclass=*")
> >>> test2 = l.search_s("ou=do,o=isd_197", ldap.SCOPE_ONELEVEL,
> "objectclass=*")
> Traceback (innermost last):
>   File "<stdin>", line 1, in ?
> ldap.INVALID_DN_SYNTAX: {'desc': 'Invalid DN syntax'}
> >>>

Hmm. And this very special search works with web2ldap?

> I'm more convinced than ever that this is an NDS issue.

But if it works with the more complex web2ldap it should work with
your simple source too since both uses ldapmodule. There's not extra
voodoo in web2ldap. E.g. web2ldap works with ldap://nldap.com which
is a (rather ancient) NDS server.

> those bindery objects in the NDS tree is really just a kludge to prevent
> breaking legacy apps that depend on the pre-NDS technology.

> I'm not sure what you mean by "canonifying the DN." Care to expand on that?

In your simple case it means removing whitespaces around the comma.
In more complex situations it also means handling escape sequences
for special characters like + , ; etc.

Ciao, Michael.

Re: python-ldap, Zope, and NDS

[Timothy W. <wi...@vi...>]

On Sat, 24 Jun 2000, Michael [iso-8859-1] Ströder wrote:

> Does one-level browsing with web2ldap work?

Yes it did. I searched an ou that has some of those pesky bindery objects,
and web2ldap seemed fine.

> > It turns out that the bug I found (wherever its source) only manifests
> > itself when I use the 'browse' feature of Jeffrey Shell's ZLDAPConnection
> > product for Zope.
> 
> I don't know this module. If you reimplement one-level browsing in a
> primitive test program and it works you have to dig into
> ZLDAPConnection. Do you have any debugging messages in your server
> logs? It would be helpful to observe the DNs, scope and search
> filter it receives.

The code I posted in my original post showed that using ldap.SCOPE_ONELEVEL
fails when searching 'ou's that have the troublesome bindery objects.

Here it is again:

Python 1.5.2 (#0, Apr  3 2000, 14:46:48)  [GCC 2.95.2 20000313 (Debian
GNU/Linux)] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import ldap
>>> l = ldap.open("206.131.108.2",389)
>>> l.simple_bind_s("ou=do,o=isd_197","")
>>> test = l.search_s("ou=do,o=isd_197", ldap.SCOPE_SUBTREE,
"objectclass=*")
>>> test2 = l.search_s("ou=do,o=isd_197", ldap.SCOPE_ONELEVEL,
"objectclass=*")
Traceback (innermost last):
  File "<stdin>", line 1, in ?
ldap.INVALID_DN_SYNTAX: {'desc': 'Invalid DN syntax'}
>>> 

> I'm not really sure that this is a bug in the OpenLDAP lib. I
> experienced exactly the same error when doing LDAP-queries with
> postfix-MTA (build with OpenLDAP libs) against a Lotus Domino
> server. But browsing and searching this server with web2ldap
> (python-ldap build with OpenLDAP lib) works just fine except some
> strange things Domino is doing (e.g. case-sensitive mail
> attributes). IMHO you have to take care about canonifying the DN.
> There are e.g. some strange things with LDAPv2- and LDAPv3-DN
> conversion.

I'm more convinced than ever that this is an NDS issue. The presence of
those bindery objects in the NDS tree is really just a kludge to prevent
breaking legacy apps that depend on the pre-NDS technology.

I'm not sure what you mean by "canonifying the DN." Care to expand on that?

Thanks for the help.

-Tim

--
Tim Wilson      | Visit Sibley online:         | Check out:
Henry Sibley HS | http://www.isd197.k12.mn.us/ | http://www.zope.org/
W. St. Paul, MN |                              | http://slashdot.org/
wi...@vi... |   <dtml-var pithy_quote>     | http://linux.com/

Re: python-ldap, Zope, and NDS

[Michael <mi...@st...>]

Timothy Wilson wrote:
> 
> On Sat, 24 Jun 2000, David Leonard wrote:
> 
> > How did it go? Did Michael's answer help?
> 
> I took Michael's advice, but it didn't change anything.

Does one-level browsing with web2ldap work?

> It turns out that the bug I found (wherever its source) only manifests
> itself when I use the 'browse' feature of Jeffrey Shell's ZLDAPConnection
> product for Zope.

I don't know this module. If you reimplement one-level browsing in a
primitive test program and it works you have to dig into
ZLDAPConnection. Do you have any debugging messages in your server
logs? It would be helpful to observe the DNs, scope and search
filter it receives.

> > If it didn't then the problem may lie somewhere in the LDAP C library,
> > since python is simply reporting the errors that it finds.  Are you using
> > OpenLDAP, or solaris or netscape's ldap or...?
> 
> I suspect that the problem is a result of a bug in the LDAP C library as 
> you
> suggest or a case of < 100% compliance to the ldap standard on the part of
> Novell's NDS.

I'm not really sure that this is a bug in the OpenLDAP lib. I
experienced exactly the same error when doing LDAP-queries with
postfix-MTA (build with OpenLDAP libs) against a Lotus Domino
server. But browsing and searching this server with web2ldap
(python-ldap build with OpenLDAP lib) works just fine except some
strange things Domino is doing (e.g. case-sensitive mail
attributes). IMHO you have to take care about canonifying the DN.
There are e.g. some strange things with LDAPv2- and LDAPv3-DN
conversion.

Ciao, Michael.

P.S.: Please, let's stay on the mailing list with this discussion.

Re: complex "search_s" filter

[David L. <dav...@cs...>]

hi

On Fri, 23 Jun 2000, Jeffrey C. Ollie typed thusly:
> (&(objectclass=person)(|(st=MN)(st=IA))(!(givenName=Tim)))
> See RFC 1558 for a complete description.

note that RFC 1960 obsoletes 1558.

-- 
David Leonard                           Dav...@cs...
Dept of Comp. Sci. and Elec. Engg   _   Room:78-640  Ph:+61 7 336 51187
The University of Queensland       |+|  http://www.csee.uq.edu.au/~leonard/
QLD 4072  AUSTRALIA               ~` '~ E2A24DC6446E5779D7AFC41AA04E6401

Curses! - Mojo Jojo

Re: complex "search_s" filter

[Timothy W. <wi...@vi...>]

On Fri, 23 Jun 2000, Gene Yu wrote:

> I suggest you take a look at RFC 2254 -  The String Representation of LDAP
> Search Filters:
> 
> 	http://www.ietf.org/rfc/rfc2254.txt
> 
> It has many examples.

Thanks to everyone who offered suggestions. I'll have a look at the RFCs.

-Tim

--
Tim Wilson      | Visit Sibley online:         | Check out:
Henry Sibley HS | http://www.isd197.k12.mn.us/ | http://www.zope.org/
W. St. Paul, MN |                              | http://slashdot.org/
wi...@vi... |   <dtml-var pithy_quote>     | http://linux.com/

Re: complex "search_s" filter

[Gene Yu <ge...@si...>]

On Fri, 23 Jun 2000, Timothy Wilson wrote:
>
> [...]
> 
> 1. How would I search objectclass=person and return all records for people
> with sn=Wilson, but not givenName=Tim?

Timothy,

You're using prefix notation, so you have the operator followed by however
many arguments.  Your example #1 would look like this:

	(&(sn=Wilson)(!(givenName=Tim)))

> 2. Search objectclass=person and return all records for people with st=MN OR
> st=IA, but not givenName=Tim? (A rather useless search I realize.)

You can just add terms forever:

	(&(a=1)(b=2)(c=3)(!(d=4)) ...)

I suggest you take a look at RFC 2254 -  The String Representation of LDAP
Search Filters:

	http://www.ietf.org/rfc/rfc2254.txt

It has many examples.

Gene

Re: complex "search_s" filter

[Jeffrey C. O. <je...@ol...>]

On Fri, Jun 23, 2000 at 02:41:00PM -0500, Timothy Wilson wrote:
>
> 1. How would I search objectclass=person and return all records for people
> with sn=Wilson, but not givenName=Tim?

(&(objectclass=person)(sn=Wilson)(!(givenName=Tim)))

> 2. Search objectclass=person and return all records for people with st=MN OR
> st=IA, but not givenName=Tim? (A rather useless search I realize.)

(&(objectclass=person)(|(st=MN)(st=IA))(!(givenName=Tim)))

See RFC 1558 for a complete description.

Jeff

Re: complex "search_s" filter

[Timothy W. <wi...@vi...>]

On Fri, 23 Jun 2000, Gene Yu wrote:

> The filter you're looking for would look like this:
> 
> 	(&(objectclass=person)(sn=Wilson))

Thanks. That worked perfectly.

> > It looks like once I understand the "and," the "or" and "not" should be
> > obvious.

OK, I was wrong. I'm stumped as to how to combine boolean operators. How
about two more examples? :-)

1. How would I search objectclass=person and return all records for people
with sn=Wilson, but not givenName=Tim?

2. Search objectclass=person and return all records for people with st=MN OR
st=IA, but not givenName=Tim? (A rather useless search I realize.)

Thanks for the help.

-Tim

--
Tim Wilson      | Visit Sibley online:         | Check out:
Henry Sibley HS | http://www.isd197.k12.mn.us/ | http://www.zope.org/
W. St. Paul, MN |                              | http://slashdot.org/
wi...@vi... |   <dtml-var pithy_quote>     | http://linux.com/

Re: complex "search_s" filter

[Gene Yu <ge...@si...>]

Timothy,

The filter you're looking for would look like this:

	(&(objectclass=person)(sn=Wilson))

You'll have to sort the results in Python.

Gene


On Fri, 23 Jun 2000, Timothy Wilson wrote:

> Hi everyone,
> 
> I'm making good progress implementing Jeffrey Shell's ZLDAP product. I've
> got it printing an employee directory using data from our NDS server. I've
> read David Leonard's "Python LDAP Module Reference," but there are a couple
> things I don't understand.
> 
> 1. What's the syntax for doing more complex filtering of search results in a
> "search_s" operation. I saw the reference to BNF in David's docs, and I had
> to look up what BNF meant. Unfortunately, I haven't been able to figure out
> how to filter my searches for two different attribute variables
> simultaneously. Would someone be kind enough to provide a couple concrete
> examples. Here's the type of thing I'm looking for:
> 
> Filter for --> objectclass=person AND sn=Wilson
> 
> It looks like once I understand the "and," the "or" and "not" should be
> obvious.
> 
> 2. Is there any way to sort the results of the search other than doing a
> separate python operation of the search result. (I haven't tried it, but I
> assume you could have python sort the results.)
> 
> Thanks in advance for any help. I think I'm going to be using python-ldap
> quite a bit.
> 
> -Tim
> 
> --
> Tim Wilson      | Visit Sibley online:         | Check out:
> Henry Sibley HS | http://www.isd197.k12.mn.us/ | http://www.zope.org/
> W. St. Paul, MN |                              | http://slashdot.org/
> wi...@vi... |   <dtml-var pithy_quote>     | http://linux.com/
> 
> 
> _______________________________________________
> Python-LDAP-dev mailing list
> Pyt...@li...
> http://lists.sourceforge.net/mailman/listinfo/python-ldap-dev
> 

complex "search_s" filter

[Timothy W. <wi...@vi...>]

Hi everyone,

I'm making good progress implementing Jeffrey Shell's ZLDAP product. I've
got it printing an employee directory using data from our NDS server. I've
read David Leonard's "Python LDAP Module Reference," but there are a couple
things I don't understand.

1. What's the syntax for doing more complex filtering of search results in a
"search_s" operation. I saw the reference to BNF in David's docs, and I had
to look up what BNF meant. Unfortunately, I haven't been able to figure out
how to filter my searches for two different attribute variables
simultaneously. Would someone be kind enough to provide a couple concrete
examples. Here's the type of thing I'm looking for:

Filter for --> objectclass=person AND sn=Wilson

It looks like once I understand the "and," the "or" and "not" should be
obvious.

2. Is there any way to sort the results of the search other than doing a
separate python operation of the search result. (I haven't tried it, but I
assume you could have python sort the results.)

Thanks in advance for any help. I think I'm going to be using python-ldap
quite a bit.

-Tim

--
Tim Wilson      | Visit Sibley online:         | Check out:
Henry Sibley HS | http://www.isd197.k12.mn.us/ | http://www.zope.org/
W. St. Paul, MN |                              | http://slashdot.org/
wi...@vi... |   <dtml-var pithy_quote>     | http://linux.com/

Re: python-ldap, Zope, and NDS

[Michael <mi...@st...>]

Timothy Wilson wrote:
> 
> our NDS (v7.45) directory.
> [..]
> 'Browse' tab, I got an INVALID_DN_SYNTAX error.
> [..]
> problem lies in searching with SCOPE_ONELEVEL.
> [..]
> "ou=do, o=ISD_197"
         ^
Those spaces sometimes confuse the libs. I usually normalize the DNs
(deleting those spaces etc.).

> >>> import ldap
> >>> l = ldap.open("10.3.6.1", 389)
> >>> l.simple_bind_s("o=ISD_197", "")
> >>> test1 = l.search_s("ou=do, o=ISD_197", ldap.SCOPE_SUBTREE,
> "objectclass=*")
> >>> test2 = l.search_s("ou=do, o=ISD_197", ldap.SCOPE_ONELEVEL,
> "objectclass=*")
> Traceback (innermost last):
>   File "<stdin>", line 1, in ?
> ldap.INVALID_DN_SYNTAX: {'desc': 'Invalid DN syntax'}

IMHO this exception message is generated by the server.

You might wanna check if my package http://web2ldap.de is able to
browse your server.

> I changed the reference in ZLDAP.py to SCOPE_SUBTREE and ZLDAP works fine.

Strange.

> Except, and this is a big 'except,' when I try to browse our entire tree. I
> think that using SCOPE_SUBTREE is causing the search to retrieve the same
> data over and over again.

When browsing you should limit the attributes to receive for saving
bandwidth and server load. See web2ldap.

Ciao, Michael.

python-ldap, Zope, and NDS

[Timothy W. <wi...@vi...>]

Hi everyone,

I've just posted a bug report at the python-ldap Sourceforge site, but I
thought I would give some more information here.

I'm building a Zope-powered Web site for my school and I'd like to leverage
all the data that we've currently got in our NDS (v7.45) directory. I
discovered a problem with python-ldap that manifested itself when I was
implementing the ZLDAP product.

I installed ZLDAP (Zope 2.1.6) and created a connection to our NDS tree. No
problem there. However, when I tried to browse the tree from the ZLDAP
'Browse' tab, I got an INVALID_DN_SYNTAX error. After hacking around with
some little Python programs I created to query NDS, I discovered that the
problem lies in searching with SCOPE_ONELEVEL. Here's an example using our
NDS server. The part of the tree represented by "ou=do, o=ISD_197" contains
some bindery objects.

Bindery objects are often found in NDS trees where the underlying Netware
software has been upgraded from previous, non-NDS versions. These bindery
objects seem to be the cause of the problem. Browsing subtrees that don't
contain any bindery objects works perfectly.

wilsont@gershwin:~$ python
Python 1.5.2 (#0, Apr  3 2000, 14:46:48)  [GCC 2.95.2 20000313 (Debian
GNU/Linux)] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import ldap
>>> l = ldap.open("10.3.6.1", 389)
>>> l.simple_bind_s("o=ISD_197", "")
>>> test1 = l.search_s("ou=do, o=ISD_197", ldap.SCOPE_SUBTREE,
"objectclass=*")
>>> test2 = l.search_s("ou=do, o=ISD_197", ldap.SCOPE_ONELEVEL,
"objectclass=*")
Traceback (innermost last):
  File "<stdin>", line 1, in ?
ldap.INVALID_DN_SYNTAX: {'desc': 'Invalid DN syntax'}

Because the ZLDAP product uses the SCOPE_ONELEVEL option to get the ldap
records, browsing a tree that contains bindery objects using ZLDAP fails.

I changed the reference in ZLDAP.py to SCOPE_SUBTREE and ZLDAP works fine.
Except, and this is a big 'except,' when I try to browse our entire tree. I
think that using SCOPE_SUBTREE is causing the search to retrieve the same
data over and over again. When I tried it, all the available RAM (128 MB) 
and swap (128 MB) on my Linux server was eventually used up, the python
process was killed off, and our NDS/ldap server sat spinning its wheels
until we restarted the ldap process. Browsing a subtree of our NDS directory
seems to work fine with the modification in ZLDAP.py and returns what
appears to be the correct information. Obviously, there's a lot less data in
the subtrees.

Is this helpful? I can provide more information if anyone needs it. I look
forward to hearing any comments that any of you might have.

-Tim

--
Tim Wilson      | Visit Sibley online:         | Check out:
Henry Sibley HS | http://www.isd197.k12.mn.us/ | http://www.zope.org/
W. St. Paul, MN |                              | http://slashdot.org/
wi...@vi... |   <dtml-var pithy_quote>     | http://linux.com/

Patch for NDS client libraries (fwd)

[David L. <dav...@cs...>]

---------- Forwarded message ----------
Date: Tue, 30 May 2000 16:23:18 -0400 (EDT)
From: Cristian Gafton <ga...@re...>
To: le...@us...
Subject: Patch for NDS client libraries


Hello,

This patch is necessary for getting the python-ldap module compiled and
linked against Novell's C SDK libraries.

Thanks,

Cristian
--
----------------------------------------------------------------------
Cristian Gafton     --     ga...@re...      --     Red Hat, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  "How could this be a problem in a country where we have Intel and 
   Microsoft?"  --Al Gore on Y2K

(no subject)

[Gene Yu <ge...@si...>]

I'm new to the list but I have some thoughts to add to the discussion in
the archives from February.  I'm using the LDAP module to build Web
interfaces for directories that are very large and efficient pagination is
a serious concern.

Thanks to Michael Stroeder for pointing out that I can to use asynchronous
searching to pull one record at a time, canceling the search when I reach
the last record on the page.  I've used a similar method with relational
databases in Perl.  But I don't like pulling so much data that I'm just
going to throw away.  Also, without some way of pulling the records in
sorted order, I need the whole set anyway if I want to present them in a
logical way.

The Netscape SDK has a Server-Side Sorting control and a Virtual List View
control that I'd love to be able to use in python.  They published a
couple of Internet Drafts:

http://www.ietf.org/internet-drafts/draft-ietf-ldapext-ldapv3-vlv-04.txt
http://www.ietf.org/internet-drafts/draft-ietf-ldapext-sorting-02.txt

Another related topic I discussed with Michael is callbacks.  I think I
see now how I can get some of the effects of callbacks using asynchronous
calls, but it would be a useful feature in the API as well.  HTML
formatting of results, for instance, can sometimes be much simplified.

Thanks also to everyone who contributed to the LDAP module.  It's a
crucial protocol for any language to support.

Gene

-- 
Gene Yu <ge...@si...>

Re: swig

[David L. <le...@cs...>]

On Tue, 28 Mar 2000, David Margrave wrote:

> I recall reading in the README file for your ldapmodule, that if you had
> it all to do over again you might use swig.   Are you sure about that?
> I've tried wrapping a few things with swig and found that I have to use
> so many typemaps, that I begin to question whether it wouldn't just be
> easier to write the whole thing in C.

perhaps. writing typemaps is like writing C anyway as I recall, so these
would be a similar task.

I would think that the abstractions provided by swig would be a win. maybe
its harder to debug? maybe swig isn't always reliable? dunno. i'm not a
swig expert.

in fact i havent done much for the python-ldap project for quite some time
its been very quiet. so much for open source enthusiasm! :)

maybe after i've shifted house, finished these papers and other work
and other projects that i'm involved in i will find time.... :(

d
--
David Leonard                           Dav...@cs...
Dept of Comp. Sci. and Elec. Engg   _   Room:78-624  Ph:+61 7 336 52447
The University of Queensland       |+|  http://www.csee.uq.edu.au/~leonard/
QLD 4072  AUSTRALIA               ~` '~ E2A24DC6446E5779D7AFC41AA04E6401

This is not SPAM. [...] If you don't wish to receive these messages, you 
should not post at these type of sites.
	- Extract from a spam message ([63.31.76.17] 2000-03-13 07:54:31 PST)

Re: Future of ldapmodule

[<fo...@mi...>]

Scavenging the mail folder uncovered Sascha Gresk's letter:
> Federico Di Gregorio wrote:
> > are you duplicationg my work? pls, give a look on it...
> > 
> 
> Sounds funny - what youre doing ? - I send you the source in a separate
> letter now ...

uh, ok. i made (another) mistake. your code is a low-level API.
i thought you were building some high-level python classes too,
exactly what i am doing. (the swig stuff looks really nice.) 

ciao,
federico
 
-- 
Federico Di Gregorio
MIXAD LIVE System Programmer                           fo...@mi...
Debian GNU/Linux Developer & Italian Press Contact        fo...@de...
             Quis custodiet ipsos custodes? -- Juvenal, Satires, VI, 347

Re: Future of ldapmodule

[Sascha G. <sa...@fr...>]

Federico Di Gregorio wrote:
> 
> Scavenging the mail folder uncovered Sascha Gresk's letter:
> >
> > > I would suggest that ldapmodule stays as close to the C API as
> > > possible with having exception for error handling just as it is
> > > right now. Higher level abstractions (all kind of schema handling)
> > > can be build into a separate Python class library (e.g. fog's
> > > modules).
> > >
> >
> > Implementing ...
> > Now the discussion is : typemaps or classes ?
> 



> are you duplicationg my work? pls, give a look on it...
> 

Sounds funny - what youre doing ? - I send you the source in a separate
letter now ...


[...]

Sascha