python-ldap-dev — discussion among developers

Re: problem using python-ldap under fcgi

[Matej V. <ve...@de...>]

Michael Ströder <mi...@st...> writes:

> But let's look at the blog entry which convinced you to try patching 
> python-ldap (see 
> http://www.notes.xythian.net/2007/10/24/python-cdb-032-52ubuntu2-with-python-25-causes-double-free-corruption-crash-on-dealloc/):
>
> "Some other searching suggests that python-cdb’s use of PyMem_DEL is no 
> longer recommended."
>
> That's pretty unprecise, not even a single URL.
>
> But if somebody can add more detailed information to this it could be 
> helpful to dive into this. IMO an admirable goal of python-ldap is not 
> to fall back behind what's considered current best practice when writing 
> extension modules for CPython.

I think the blog writer was bitten by <http://bugs.debian.org/468993>.
In Python 2.5, PyMem_DEL and PyObject_Del are no longer interchangeable;
memory allocated by PyMem_NEW needs to be deallocated by PyMem_DEL, and
likewise for PyObject_New and PyObject_Del.

A similar (automated) report was filed for python-ldap, and I've
verified that it doesn't apply -- <http://bugs.debian.org/468995>.

Cheers,

Matej

Re: problem using python-ldap under fcgi

[Michael S. <mi...@st...>]

Matt Bartolome wrote:
> My apologies on the wild goose chase but after using valgrind on my
> fcgi process it is python cx_Oracle (would have never guessed that!)
> which triggers the segmentation fault when ldap.initialize() is
> called. Why it does this is beyond me but a simple alteration of my
> code makes the problem go away completely. I was creating a global
> oracle db cursor which I'm now creating inside the functions that use
> it.
> 
> I'm not sure about the glib c error and patch now. Using the original
> release without modification works so I will leave it at that.

Glad you figured out what the issue was. It's good if you don't run a 
patched version of python-ldap. In general and thanks to the 
contributors who provided patches in the past python-ldap seems fairly 
stable.

But let's look at the blog entry which convinced you to try patching 
python-ldap (see 
http://www.notes.xythian.net/2007/10/24/python-cdb-032-52ubuntu2-with-python-25-causes-double-free-corruption-crash-on-dealloc/):

"Some other searching suggests that python-cdb’s use of PyMem_DEL is no 
longer recommended."

That's pretty unprecise, not even a single URL.

But if somebody can add more detailed information to this it could be 
helpful to dive into this. IMO an admirable goal of python-ldap is not 
to fall back behind what's considered current best practice when writing 
extension modules for CPython.

Ciao, Michael.

Re: problem using python-ldap under fcgi

[Matt B. <mat...@gm...>]

My apologies on the wild goose chase but after using valgrind on my
fcgi process it is python cx_Oracle (would have never guessed that!)
which triggers the segmentation fault when ldap.initialize() is
called. Why it does this is beyond me but a simple alteration of my
code makes the problem go away completely. I was creating a global
oracle db cursor which I'm now creating inside the functions that use
it.

I'm not sure about the glib c error and patch now. Using the original
release without modification works so I will leave it at that.

Thanks,
Matt

On Wed, Aug 27, 2008 at 12:26 AM, Michael Ströder <mi...@st...> wrote:
> Matt Bartolome wrote:
>>
>> Hey Michael. Thank you for your response. I modified LDAPObject.c and
>> ldapcontrol.c to use the solution described here:
>>
>> https://bugs.launchpad.net/ubuntu/+source/python-cdb/+bug/157251
>>
>> Looks like it is something specific to glibc in ubuntu and the
>> "recommended" use of PyObject_Del instead of PyMem_DEL. I've attached
>> a patch showing the modifications for anyone else ripping their hair
>> out. The glibc python free() error is now gone.
>
> If this is really a more general issue I'd like to see this fixed. So I
> tried your patch. But it seg faults on my openSUSE 11.0 system. I noticed
> some warnings during build (see below).
>
> Ciao, Michael.
>
> --------------------------- snip ---------------------------
> [..]
> Modules/LDAPObject.c: In function 'Tuple_to_LDAPMod':
> Modules/LDAPObject.c:126: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/LDAPObject.c:134: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/LDAPObject.c:143: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/LDAPObject.c:146: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/LDAPObject.c:154: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/LDAPObject.c:158: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/LDAPObject.c: In function 'List_to_LDAPMods':
> Modules/LDAPObject.c:226: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/LDAPObject.c: In function 'attrs_from_List':
> Modules/LDAPObject.c:273: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall
> -Wstrict-prototypes -fPIC -DHAVE_LIBLDAP_R -DHAVE_SASL -DHAVE_TLS
> -DLDAPMODULE_VERSION=2.3.6 -IModules -I/opt/openldap-HEAD/include
> -I/opt/sasl/include/sasl -I/usr/include/sasl -I/usr/include/python2.6 -c
> Modules/ldapcontrol.c -o build/temp.linux-i686-2.6/Modules/ldapcontrol.o -g
> Modules/ldapcontrol.c: In function 'Tuple_to_LDAPControl':
> Modules/ldapcontrol.c:83: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/ldapcontrol.c:92: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/ldapcontrol.c: In function 'List_to_LDAPControls':
> Modules/ldapcontrol.c:139: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> [..]
>

Re: problem using python-ldap under fcgi

[Michael S. <mi...@st...>]

Matt Bartolome wrote:
> Hey Michael. Thank you for your response. I modified LDAPObject.c and
> ldapcontrol.c to use the solution described here:
> 
> https://bugs.launchpad.net/ubuntu/+source/python-cdb/+bug/157251
> 
> Looks like it is something specific to glibc in ubuntu and the
> "recommended" use of PyObject_Del instead of PyMem_DEL. I've attached
> a patch showing the modifications for anyone else ripping their hair
> out. The glibc python free() error is now gone.

If this is really a more general issue I'd like to see this fixed. So I 
tried your patch. But it seg faults on my openSUSE 11.0 system. I 
noticed some warnings during build (see below).

Ciao, Michael.

--------------------------- snip ---------------------------
[..]
Modules/LDAPObject.c: In function ‘Tuple_to_LDAPMod’:
Modules/LDAPObject.c:126: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/LDAPObject.c:134: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/LDAPObject.c:143: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/LDAPObject.c:146: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/LDAPObject.c:154: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/LDAPObject.c:158: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/LDAPObject.c: In function ‘List_to_LDAPMods’:
Modules/LDAPObject.c:226: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/LDAPObject.c: In function ‘attrs_from_List’:
Modules/LDAPObject.c:273: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall 
-Wstrict-prototypes -fPIC -DHAVE_LIBLDAP_R -DHAVE_SASL -DHAVE_TLS 
-DLDAPMODULE_VERSION=2.3.6 -IModules -I/opt/openldap-HEAD/include 
-I/opt/sasl/include/sasl -I/usr/include/sasl -I/usr/include/python2.6 -c 
Modules/ldapcontrol.c -o build/temp.linux-i686-2.6/Modules/ldapcontrol.o -g
Modules/ldapcontrol.c: In function ‘Tuple_to_LDAPControl’:
Modules/ldapcontrol.c:83: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/ldapcontrol.c:92: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/ldapcontrol.c: In function ‘List_to_LDAPControls’:
Modules/ldapcontrol.c:139: warning: passing argument 1 of 
‘_PyObject_New’ makes pointer from integer without a cast
[..]

Re: problem using python-ldap under fcgi

[Matt B. <mat...@gm...>]

Hey Michael. Thank you for your response. I modified LDAPObject.c and
ldapcontrol.c to use the solution described here:

https://bugs.launchpad.net/ubuntu/+source/python-cdb/+bug/157251

Looks like it is something specific to glibc in ubuntu and the
"recommended" use of PyObject_Del instead of PyMem_DEL. I've attached
a patch showing the modifications for anyone else ripping their hair
out. The glibc python free() error is now gone.

Now I can focus on my actual problem which I think has to do with my
python-ldap calls being blocked under fcgi. I tried compiling openldap
--without-threads thinking that it might magically work but no dice.

regards,
-Matt

On Tue, Aug 26, 2008 at 12:39 AM, Michael Ströder <mi...@st...> wrote:
> Matt Bartolome wrote:
>> I've got a little problem and I'm not sure how to track down the error
>> I'm getting. My web server is spitting back a 504 gateway timeout
>> which isn't helpful at all so I'm hoping someone here can point me in
>> the right direction.
>>
>> When I attempt to:
>>
>>             l = ldap.initialize(settings.AD_LDAP_URL)
>>             l.simple_bind_s(binddn,password)
>>             l.unbind_s()
>>
>> I get the gateway timeout but this works perfectly fine in the python
>> interpreter. I'm running django as an fcgi preforked socket and
>> pointing my nginx fcgi server to it.
>
> As my web2ldap - based on python-ldap - runs as a FastCGI server this
> seems to work. But web2ldap is multi-threaded and does not fork when
> running as FastCGI process because of keeping the LDAP connections
> persistent.
>
>> All web requests work fine until
>> I hit the l.simple_bind_s() function so I'm having a heck of a time
>> finding out what the actual error is. The only other output I've been
>> able to produce is:
>>
>> *** glibc detected *** python: free(): invalid pointer: 0x08e9b1ec ***
>>
>> this is when I don't background the fcgi process.
>
> No clue especially since I don't know the Django stuff.
>
> Ciao, Michael.
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Python-LDAP-dev mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-ldap-dev
>

Re: problem using python-ldap under fcgi

[Michael S. <mi...@st...>]

Matt Bartolome wrote:
> I've got a little problem and I'm not sure how to track down the error
> I'm getting. My web server is spitting back a 504 gateway timeout
> which isn't helpful at all so I'm hoping someone here can point me in
> the right direction.
> 
> When I attempt to:
> 
>             l = ldap.initialize(settings.AD_LDAP_URL)
>             l.simple_bind_s(binddn,password)
>             l.unbind_s()
> 
> I get the gateway timeout but this works perfectly fine in the python
> interpreter. I'm running django as an fcgi preforked socket and
> pointing my nginx fcgi server to it.

As my web2ldap - based on python-ldap - runs as a FastCGI server this 
seems to work. But web2ldap is multi-threaded and does not fork when 
running as FastCGI process because of keeping the LDAP connections 
persistent.

> All web requests work fine until
> I hit the l.simple_bind_s() function so I'm having a heck of a time
> finding out what the actual error is. The only other output I've been
> able to produce is:
> 
> *** glibc detected *** python: free(): invalid pointer: 0x08e9b1ec ***
> 
> this is when I don't background the fcgi process.

No clue especially since I don't know the Django stuff.

Ciao, Michael.

problem using python-ldap under fcgi

[Matt B. <mat...@gm...>]

Hi,
I've got a little problem and I'm not sure how to track down the error
I'm getting. My web server is spitting back a 504 gateway timeout
which isn't helpful at all so I'm hoping someone here can point me in
the right direction.

When I attempt to:

            l = ldap.initialize(settings.AD_LDAP_URL)
            l.simple_bind_s(binddn,password)
            l.unbind_s()

I get the gateway timeout but this works perfectly fine in the python
interpreter. I'm running django as an fcgi preforked socket and
pointing my nginx fcgi server to it. All web requests work fine until
I hit the l.simple_bind_s() function so I'm having a heck of a time
finding out what the actual error is. The only other output I've been
able to produce is:

*** glibc detected *** python: free(): invalid pointer: 0x08e9b1ec ***

this is when I don't background the fcgi process. I'm fully aware that
this may have nothing to do with python-ldap but just hoping someone
here may have a clue for me.


python 2.5.2
python-ldap 2.3.5
ubuntu 7.10

my fcgi socket is spawned like so (if it helps):
python /var/django/gis/manage.py runfcgi --settings=intranet_settings
method=prefork \
  pidfile=/tmp/fcgi.pid \
  socket=/tmp/fcgi.sock

Thanks,
Matt

Re: SunStudio compile

[Matej V. <ve...@de...>]

Michael Ströder <mi...@st...> writes:

> I will commit this change.

Thanks.

> Please also examine
>
> http://python-ldap.cvs.sourceforge.net/python-ldap/python-ldap/Modules/LDAPObject.c?r1=1.79&r2=1.80
>
> http://python-ldap.cvs.sourceforge.net/python-ldap/python-ldap/Modules/options.c?r1=1.19&r2=1.20

Both look good.

Cheers,

Matej

Re: SunStudio compile

[Michael S. <mi...@st...>]

Matej Vela wrote:
> Yup.  If you haven't already, try replacing lines 614 and 615
> 
>     sasl_interact_t *interact = in;
>     PyObject *SASLObject = defaults;
> 
> with
> 
>     sasl_interact_t *interact = (sasl_interact_t *) in;
>     PyObject *SASLObject = (PyObject *) defaults;

I will commit this change.

Please also examine

http://python-ldap.cvs.sourceforge.net/python-ldap/python-ldap/Modules/LDAPObject.c?r1=1.79&r2=1.80

http://python-ldap.cvs.sourceforge.net/python-ldap/python-ldap/Modules/options.c?r1=1.19&r2=1.20

Ciao, Michael.

Re: SunStudio compile

[Matej V. <ve...@de...>]

Anil <an...@en...> writes:

> Thanks, that worked! I am able to compile it now, but if I enable sasl, 
> it also fails:
>
> "Modules/LDAPObject.c", line 568: Warning: String literal converted to 
> char* in formal argument format in call to 
> _PyObject_CallMethod_SizeT(_object*, char*, char*, ...).
> "Modules/LDAPObject.c", line 614: Error: Cannot use void* to initialize 
> sasl_interact*.
> "Modules/LDAPObject.c", line 615: Error: Cannot use void* to initialize 
> _object*.
> "Modules/LDAPObject.c", line 689: Warning (Anachronism): Formal argument 
> proc of type extern "C" int(*)(ldap*,unsigned,void*,void*) in call to 
> ldap_sasl_interactive_bind_s(ldap*, const char*, const char*, 
> ldapcontrol**, ldapcontrol**, unsigned, extern "C" 
> int(*)(ldap*,unsigned,void*,void*), void*) is being passed 
> int(*)(ldap*,unsigned,void*,void*).
>
> Sounds like the same thing.

Yup.  If you haven't already, try replacing lines 614 and 615

    sasl_interact_t *interact = in;
    PyObject *SASLObject = defaults;

with

    sasl_interact_t *interact = (sasl_interact_t *) in;
    PyObject *SASLObject = (PyObject *) defaults;

(Sorry for the late reply.)

Cheers,

Matej

Re: SunStudio compile

[Matej V. <ve...@de...>]

Michael Ströder <mi...@st...> writes:

> Matej Vela wrote:
>> The type checking seems to be a bit overzealous.  Try changing line 108
>> of Modules/options.c from
>> 
>>         LDAPControl_List_DEL(ptr);
>> 
>> to
>> 
>>         LDAPControl_List_DEL((LDAPControl**) ptr);
>
> Matej, do you think we should change this in general?

Yes, can't hurt (though most compilers are fine with both).

Cheers,

Matej

Re: SunStudio compile

[Michael S. <mi...@st...>]

Anil wrote:
> Thanks, that worked! I am able to compile it now, but if I enable sasl, 
> it also fails:
> 
> "Modules/LDAPObject.c", line 568: Warning: String literal converted to 
> char* in formal argument format in call to 
> _PyObject_CallMethod_SizeT(_object*, char*, char*, ...).
> "Modules/LDAPObject.c", line 614: Error: Cannot use void* to initialize 
> sasl_interact*.
> "Modules/LDAPObject.c", line 615: Error: Cannot use void* to initialize 
> _object*.
> "Modules/LDAPObject.c", line 689: Warning (Anachronism): Formal argument 
> proc of type extern "C" int(*)(ldap*,unsigned,void*,void*) in call to 
> ldap_sasl_interactive_bind_s(ldap*, const char*, const char*, 
> ldapcontrol**, ldapcontrol**, unsigned, extern "C" 
> int(*)(ldap*,unsigned,void*,void*), void*) is being passed 
> int(*)(ldap*,unsigned,void*,void*).
> 
> Sounds like the same thing.

And could you fix it with the same approach Matej suggested?

Ciao, Michael.

Re: SunStudio compile

[Anil <an...@en...>]

Thanks, that worked! I am able to compile it now, but if I enable sasl, 
it also fails:

"Modules/LDAPObject.c", line 568: Warning: String literal converted to 
char* in formal argument format in call to 
_PyObject_CallMethod_SizeT(_object*, char*, char*, ...).
"Modules/LDAPObject.c", line 614: Error: Cannot use void* to initialize 
sasl_interact*.
"Modules/LDAPObject.c", line 615: Error: Cannot use void* to initialize 
_object*.
"Modules/LDAPObject.c", line 689: Warning (Anachronism): Formal argument 
proc of type extern "C" int(*)(ldap*,unsigned,void*,void*) in call to 
ldap_sasl_interactive_bind_s(ldap*, const char*, const char*, 
ldapcontrol**, ldapcontrol**, unsigned, extern "C" 
int(*)(ldap*,unsigned,void*,void*), void*) is being passed 
int(*)(ldap*,unsigned,void*,void*).


Sounds like the same thing.


Matej Vela wrote:
> Anil <an...@en...> writes:
>
>   
>> I am trying to compile python ldap with SunStudio, I get this error. Any 
>> idea?
>>     
> [...]
>   
>> "Modules/options.c", line 108: Error: Formal argument 1 of type 
>> ldapcontrol** in call to LDAPControl_List_DEL(ldapcontrol**) is being 
>> passed void*.
>>     
>
> The type checking seems to be a bit overzealous.  Try changing line 108
> of Modules/options.c from
>
>         LDAPControl_List_DEL(ptr);
>
> to
>
>         LDAPControl_List_DEL((LDAPControl**) ptr);
>
> Cheers,
>
> Matej
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Python-LDAP-dev mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-ldap-dev
>   

Re: SunStudio compile

[Michael S. <mi...@st...>]

Matej Vela wrote:
> Anil <an...@en...> writes:
> 
>> I am trying to compile python ldap with SunStudio, I get this error. Any 
>> idea?
> [...]
>> "Modules/options.c", line 108: Error: Formal argument 1 of type 
>> ldapcontrol** in call to LDAPControl_List_DEL(ldapcontrol**) is being 
>> passed void*.
> 
> The type checking seems to be a bit overzealous.  Try changing line 108
> of Modules/options.c from
> 
>         LDAPControl_List_DEL(ptr);
> 
> to
> 
>         LDAPControl_List_DEL((LDAPControl**) ptr);

Matej, do you think we should change this in general?

Ciao, Michael.

Re: SunStudio compile

[Matej V. <ve...@de...>]

Anil <an...@en...> writes:

> I am trying to compile python ldap with SunStudio, I get this error. Any 
> idea?
[...]
> "Modules/options.c", line 108: Error: Formal argument 1 of type 
> ldapcontrol** in call to LDAPControl_List_DEL(ldapcontrol**) is being 
> passed void*.

The type checking seems to be a bit overzealous.  Try changing line 108
of Modules/options.c from

        LDAPControl_List_DEL(ptr);

to

        LDAPControl_List_DEL((LDAPControl**) ptr);

Cheers,

Matej

SunStudio compile

[Anil <an...@en...>]

I am trying to compile python ldap with SunStudio, I get this error. Any 
idea?

...
/opt/SUNWspro/bin/CC -I/opt/coolstack/include -DNDEBUG -fast -xipo 
-xtarget=generic -xcode=pic32 -DHAVE_LIBLDAP_R -DHAVE_TLS 
-DLDAPMODULE_VERSION=2.3.5 -IModules -I/opt/coolstack/include 
-I/opt/coolstack/include/sasl -I/opt/coolstack/include/python2.5 -c 
Modules/message.c -o build/temp.solaris-2.10-sun4u-2.5/Modules/message.o
"Modules/message.c", line 39: Warning: String literal converted to char* 
in formal argument msg in call to LDAPerror(ldap*, char*).
"Modules/message.c", line 134: Warning: String literal converted to 
char* in formal argument msg in call to LDAPerror(ldap*, char*).
2 Warning(s) detected.
/opt/SUNWspro/bin/CC -I/opt/coolstack/include -DNDEBUG -fast -xipo 
-xtarget=generic -xcode=pic32 -DHAVE_LIBLDAP_R -DHAVE_TLS 
-DLDAPMODULE_VERSION=2.3.5 -IModules -I/opt/coolstack/include 
-I/opt/coolstack/include/sasl -I/opt/coolstack/include/python2.5 -c 
Modules/version.c -o build/temp.solaris-2.10-sun4u-2.5/Modules/version.o
/opt/SUNWspro/bin/CC -I/opt/coolstack/include -DNDEBUG -fast -xipo 
-xtarget=generic -xcode=pic32 -DHAVE_LIBLDAP_R -DHAVE_TLS 
-DLDAPMODULE_VERSION=2.3.5 -IModules -I/opt/coolstack/include 
-I/opt/coolstack/include/sasl -I/opt/coolstack/include/python2.5 -c 
Modules/options.c -o build/temp.solaris-2.10-sun4u-2.5/Modules/options.o
"Modules/options.c", line 108: Error: Formal argument 1 of type 
ldapcontrol** in call to LDAPControl_List_DEL(ldapcontrol**) is being 
passed void*.
1 Error(s) detected.
error: command '/opt/SUNWspro/bin/CC' failed with exit status 1

Re: Problems importing from LDIF file generated with python-ldap

[Lars E. K. <lar...@gm...>]

Thanks a lot, this worked perfectly! You really saved my day (or week,
for that matter). I will rewrite the script when I have the time and
maybe a better understanding of LDAP and Python-ldap.

Best regards,

Lars Erik

On Wed, Aug 13, 2008 at 3:28 PM, Michael Ströder <mi...@st...> wrote:
> Lars Erik Kolden wrote:
>>
>> ldapmodify: modify operation type is missing at line 26, entry
>> "cn=audio,ou=Group,dc=ourdc,dc=no"
>> [..]
>> When I look in the LDAP docs, this looks reasonable, as it states that
>> you ned an "add: memberUid" statement with the changetype: modify. But
>> how come it worked before? And when I try to incorporate this into the
>> LDIF generator script, which uses python-ldap, it just won't work.
>
> I don't know why it worked in the past. The LDIF generator script is wrong
> since it uses module ldif for generating entry records (provided by a dict)
> instead of providing a modification list (list type) which would make
> LDIFWriter.unparse() to generate a change record.
>
> See __doc__ string in ldif.py:
>
> class LDIFWriter:
> [..]
>  def unparse(self,dn,record):
>    """
>    dn
>          string-representation of distinguished name
>    record
>          Either a dictionary holding the LDAP entry {attrtype:record}
>          or a list with a modify list like for LDAPObject.modify().
>    """
>
>> relevant code looked like this:
>>
>> #########
>>      # add the new user to a set of default groups:
>>      # audio, cdrom, floppy, plugdev, video
>>
>>      entry={ 'changetype' : ['modify'],
>>              'memberUid': [username],
>>      }
>>      dn='cn=audio,ou=Group,dc=ourdc,dc=no'
>>      ldif_writer=ldif.LDIFWriter(newusers)
>>      ldif_writer.unparse(dn,entry)
>
> Should be:
>
> modlist=[(ldap.MOD_ADD,'memberUid',[username])]
> ldif_writer.unparse(dn,modlist)
>
> BTW: Anyway I'd recommend to directly use a LDAP connection for this task,
> not generate LDIF and then using command-line tools. This would give you
> much better control in case of LDAP errors.
>
> Ciao, Michael.
>

Re: Problems importing from LDIF file generated with python-ldap

[Michael S. <mi...@st...>]

Lars Erik Kolden wrote:
> ldapmodify: modify operation type is missing at line 26, entry
> "cn=audio,ou=Group,dc=ourdc,dc=no"
> [..]
> When I look in the LDAP docs, this looks reasonable, as it states that
> you ned an "add: memberUid" statement with the changetype: modify. But
> how come it worked before? And when I try to incorporate this into the
> LDIF generator script, which uses python-ldap, it just won't work.

I don't know why it worked in the past. The LDIF generator script is 
wrong since it uses module ldif for generating entry records (provided 
by a dict) instead of providing a modification list (list type) which 
would make LDIFWriter.unparse() to generate a change record.

See __doc__ string in ldif.py:

class LDIFWriter:
[..]
   def unparse(self,dn,record):
     """
     dn
           string-representation of distinguished name
     record
           Either a dictionary holding the LDAP entry {attrtype:record}
           or a list with a modify list like for LDAPObject.modify().
     """

> relevant code looked like this:
> 
> #########
>       # add the new user to a set of default groups:
>       # audio, cdrom, floppy, plugdev, video
> 
>       entry={ 'changetype' : ['modify'],
> 	      'memberUid': [username],
>       }
>       dn='cn=audio,ou=Group,dc=ourdc,dc=no'
>       ldif_writer=ldif.LDIFWriter(newusers)
>       ldif_writer.unparse(dn,entry)

Should be:

modlist=[(ldap.MOD_ADD,'memberUid',[username])]
ldif_writer.unparse(dn,modlist)

BTW: Anyway I'd recommend to directly use a LDAP connection for this 
task, not generate LDIF and then using command-line tools. This would 
give you much better control in case of LDAP errors.

Ciao, Michael.

Problems importing from LDIF file generated with python-ldap

[Lars E. K. <lar...@gm...>]

Hi.

I am sorry if this is a stupid question. I have pretty basic knowledge
of both LDAP and python, and am having trouble with some scripts for
creating LDAP records, written by my predecessor.

This script worked fine from an Ubuntu 7.10 client environment before
the summer, but now, after actually upgrading to Ubuntu 8.04, the
script yields an error message when adding the user to groups, using
the modify changetype operator. The offending statement looks like
this:

#########
dn: cn=audio,ou=Group,dc=ourdc,dc=no
changetype: modify
memberUid: newuser
#########

The errormessage from ldapmodify:

#########
larsekol@skarphedin:~$ /usr/bin/ldapmodify -ZZ -h ldap.server -D
"cn=Manager,dc=ourdc,dc=no" -w passwrrd -x -a -f ./newaccounts.ldif

adding new entry "uid=newuser,ou=people,dc=ourdc,dc=no"

adding new entry "cn=newuser,ou=Group,dc=ourdc,dc=no"

ldapmodify: modify operation type is missing at line 26, entry
"cn=audio,ou=Group,dc=ourdc,dc=no"

larsekol@skarphedin:~$
#########

When I look in the LDAP docs, this looks reasonable, as it states that
you ned an "add: memberUid" statement with the changetype: modify. But
how come it worked before? And when I try to incorporate this into the
LDIF generator script, which uses python-ldap, it just won't work. The
relevant code looked like this:

#########
      # add the new user to a set of default groups:
      # audio, cdrom, floppy, plugdev, video

      entry={ 'changetype' : ['modify'],
	      'memberUid': [username],
      }
      dn='cn=audio,ou=Group,dc=ourdc,dc=no'
      ldif_writer=ldif.LDIFWriter(newusers)
      ldif_writer.unparse(dn,entry)
#########

I thought I could add another element to the dict, "'add':
['memberUid']", but that's probably a naiive assumption, and wishful
thinking. When I add it between the changetype and memberuid elements,
it will appear in the top of the LDIF statement, which won't work:

My "improvement":
#########
      # add the new user to a set of default groups:
      # audio, cdrom, floppy, plugdev, video

      entry={ 'changetype' : ['modify'],
              'add': ['memberUid'],
	      'memberUid': [username],
      }
      dn='cn=audio,ou=Group,dc=ourdc,dc=no'
      ldif_writer=ldif.LDIFWriter(newusers)
      ldif_writer.unparse(dn,entry)
#########

Result:
########
dn: cn=audio,ou=Group,dc=ourdc,dc=no
add: memberUid
changetype: modify
memberUid: newuser
########

Here the add and changetype statements should be the other way
(changetype first, add afterwards), if I understand the docs right. If
I do that manually in the LDIF file, ldapmodify will add the user to
the audio group with no complaints. But I haven't been able to do that
thru the python-ldap libraries.

Could someone please point me to where I've totally misunderstood
here, or how I could make this right with python-ldap (adding the user
to audio group)?

Thanks from a noob.

Lars Erik
Lars Erik

Re: Error while building

[Michael S. <mi...@st...>]

Sig wrote:
> Matej Vela a écrit :
>> Sig <si...@ak...> writes:
>>> I can't build python-ldap-2.3.5. What should I do ?
>> Looks like you're missing libldap2-dev.  Please run
>>
>>     # apt-get install build-essential libldap2-dev libsasl2-dev python-dev
> [..]
> Too bad all these dev libs are required for installing python-ldap from 
> the cheeseshop. Or maybe these requirements should be clearly indicated 
> on the python-ldap package page ?

The build prerequisites are mentioned in the python-ldap docs:
http://python-ldap.sourceforge.net/doc/html/installing.html#prerequisites

Note that apt-get is specific for Debian-based Linux distributions. The 
python-ldap docs have to be generic. I'd accept distribution-specific 
sections for the installation docs though.

Ciao, Michael.

Re: Error while building

[Ryan L. <ry...@st...>]

On Thu, Aug 07, 2008 at 12:03:29AM +0200, Sig wrote:
> Matej Vela a écrit :
> > Sig <si...@ak...> writes:
> >> I can't build python-ldap-2.3.5. What should I do ?
> > Looks like you're missing libldap2-dev.  Please run
> > 
> >     # apt-get install build-essential libldap2-dev libsasl2-dev python-dev
> 
> It helped, thanks ! I also had to add libssl-dev (before that I was 
> getting another message saying that lssl could not be found).
> 
> Now it works OK.
> 
> Too bad all these dev libs are required for installing python-ldap from 
> the cheeseshop. Or maybe these requirements should be clearly indicated 
> on the python-ldap package page ?
> 
> Anyway, thanks for your help and reactivity !

In the future you can try

    # apt-get build-dep python-ldap

apt-get.8: "build-dep causes apt-get to install/remove packages in an
attempt to satisfy the build dependencies for a source package."

Ryan

Re: Error while building

[Sig <si...@ak...>]

Matej Vela a écrit :
> Sig <si...@ak...> writes:
>> I can't build python-ldap-2.3.5. What should I do ?
> Looks like you're missing libldap2-dev.  Please run
> 
>     # apt-get install build-essential libldap2-dev libsasl2-dev python-dev

It helped, thanks ! I also had to add libssl-dev (before that I was 
getting another message saying that lssl could not be found).

Now it works OK.

Too bad all these dev libs are required for installing python-ldap from 
the cheeseshop. Or maybe these requirements should be clearly indicated 
on the python-ldap package page ?

Anyway, thanks for your help and reactivity !

-- Sig

Re: Error while building

[Matej V. <ve...@de...>]

Sig <si...@ak...> writes:

> I can't build python-ldap-2.3.5. What should I do ?
>
> My steps:
> - downloaded the 2.3.5 tarball from sourceforge
> - tar xvzf ...
> - python setup.py build
> - The error and output messages are attached to this email.
>
> My setup:
> - Python 2.5.2
> - libldap-2.4-2 (ubuntu package version = 2.4.9-0ubuntu0.8.04.1, same
> for slapd)
> - Ubuntu hardy
>
> Tell me if you need any other piece of info.

Looks like you're missing libldap2-dev.  Please run

    # apt-get install build-essential libldap2-dev libsasl2-dev python-dev

[...]
> By the way #2 : there is no web archive of this list ?

See <http://sf.net/mailarchive/forum.php?forum_name=python-ldap-dev>.

Cheers,

Matej

Error while building

[Sig <si...@ak...>]

I can't build python-ldap-2.3.5. What should I do ?

My steps:
- downloaded the 2.3.5 tarball from sourceforge
- tar xvzf ...
- python setup.py build
- The error and output messages are attached to this email.

My setup:
- Python 2.5.2
- libldap-2.4-2 (ubuntu package version = 2.4.9-0ubuntu0.8.04.1, same 
for slapd)
- Ubuntu hardy

Tell me if you need any other piece of info.

By the way, I first met this problem when trying to refresh a Plone 
buildout (PloneUserFolder has a python-ldap dependency). Which lead me 
to this post on a blog :

http://boz.tumblr.com/post/43534569/error-couldnt-install-python-ldap-2-3-5

But I am not happy enough with the workaround suggested there.

By the way #2 : there is no web archive of this list ?

Thanks.

-- Sig

Re: REG:python-ldap able to handle unicode characters

[Michael S. <mi...@st...>]

deepti jawalkar wrote:
> well these are my 2 cases :
> 
> *with python-ldap *: so in this case it works even though the object i 
> am passing has unicode characters in it's distinguished name
> eg: CN=Sen-po 胡森� 
> (senpo),OU=Users,OU=TPE,OU=Offices,DC=corp,DC=google,DC=com

Well, *I* can't read this distinguished name. I don't have the necessary 
fonts and installed and I don't understand them anyway. ;-)

> i can print the distinguished name without encoding it in utf-8 format 
> and also remove or add this user to a group.

Off course you can pass around UTF-8-encoded Unicode strings. But you 
have to invoke .decode() and .encode() in your application code (e.g. 
like my web2ldap does). python-ldap does *never* invoke these methods 
internally.

> Can you let me know how exactly is this happening in python ldap and how 
> is it able to add and remove accounts with unicode characters.

You can always just treat the DNs opaque. ;-)

Ciao, Michael.