python-ldap-dev

problem using python-ldap under fcgi

[Matt B. <mat...@gm...>]

Hi,
I've got a little problem and I'm not sure how to track down the error
I'm getting. My web server is spitting back a 504 gateway timeout
which isn't helpful at all so I'm hoping someone here can point me in
the right direction.

When I attempt to:

            l = ldap.initialize(settings.AD_LDAP_URL)
            l.simple_bind_s(binddn,password)
            l.unbind_s()

I get the gateway timeout but this works perfectly fine in the python
interpreter. I'm running django as an fcgi preforked socket and
pointing my nginx fcgi server to it. All web requests work fine until
I hit the l.simple_bind_s() function so I'm having a heck of a time
finding out what the actual error is. The only other output I've been
able to produce is:

*** glibc detected *** python: free(): invalid pointer: 0x08e9b1ec ***

this is when I don't background the fcgi process. I'm fully aware that
this may have nothing to do with python-ldap but just hoping someone
here may have a clue for me.


python 2.5.2
python-ldap 2.3.5
ubuntu 7.10

my fcgi socket is spawned like so (if it helps):
python /var/django/gis/manage.py runfcgi --settings=intranet_settings
method=prefork \
  pidfile=/tmp/fcgi.pid \
  socket=/tmp/fcgi.sock

Thanks,
Matt

Re: problem using python-ldap under fcgi

[Michael S. <mi...@st...>]

Matt Bartolome wrote:
> I've got a little problem and I'm not sure how to track down the error
> I'm getting. My web server is spitting back a 504 gateway timeout
> which isn't helpful at all so I'm hoping someone here can point me in
> the right direction.
> 
> When I attempt to:
> 
>             l = ldap.initialize(settings.AD_LDAP_URL)
>             l.simple_bind_s(binddn,password)
>             l.unbind_s()
> 
> I get the gateway timeout but this works perfectly fine in the python
> interpreter. I'm running django as an fcgi preforked socket and
> pointing my nginx fcgi server to it.

As my web2ldap - based on python-ldap - runs as a FastCGI server this 
seems to work. But web2ldap is multi-threaded and does not fork when 
running as FastCGI process because of keeping the LDAP connections 
persistent.

> All web requests work fine until
> I hit the l.simple_bind_s() function so I'm having a heck of a time
> finding out what the actual error is. The only other output I've been
> able to produce is:
> 
> *** glibc detected *** python: free(): invalid pointer: 0x08e9b1ec ***
> 
> this is when I don't background the fcgi process.

No clue especially since I don't know the Django stuff.

Ciao, Michael.

Re: problem using python-ldap under fcgi

[Matt B. <mat...@gm...>]

Hey Michael. Thank you for your response. I modified LDAPObject.c and
ldapcontrol.c to use the solution described here:

https://bugs.launchpad.net/ubuntu/+source/python-cdb/+bug/157251

Looks like it is something specific to glibc in ubuntu and the
"recommended" use of PyObject_Del instead of PyMem_DEL. I've attached
a patch showing the modifications for anyone else ripping their hair
out. The glibc python free() error is now gone.

Now I can focus on my actual problem which I think has to do with my
python-ldap calls being blocked under fcgi. I tried compiling openldap
--without-threads thinking that it might magically work but no dice.

regards,
-Matt

On Tue, Aug 26, 2008 at 12:39 AM, Michael Ströder <mi...@st...> wrote:
> Matt Bartolome wrote:
>> I've got a little problem and I'm not sure how to track down the error
>> I'm getting. My web server is spitting back a 504 gateway timeout
>> which isn't helpful at all so I'm hoping someone here can point me in
>> the right direction.
>>
>> When I attempt to:
>>
>>             l = ldap.initialize(settings.AD_LDAP_URL)
>>             l.simple_bind_s(binddn,password)
>>             l.unbind_s()
>>
>> I get the gateway timeout but this works perfectly fine in the python
>> interpreter. I'm running django as an fcgi preforked socket and
>> pointing my nginx fcgi server to it.
>
> As my web2ldap - based on python-ldap - runs as a FastCGI server this
> seems to work. But web2ldap is multi-threaded and does not fork when
> running as FastCGI process because of keeping the LDAP connections
> persistent.
>
>> All web requests work fine until
>> I hit the l.simple_bind_s() function so I'm having a heck of a time
>> finding out what the actual error is. The only other output I've been
>> able to produce is:
>>
>> *** glibc detected *** python: free(): invalid pointer: 0x08e9b1ec ***
>>
>> this is when I don't background the fcgi process.
>
> No clue especially since I don't know the Django stuff.
>
> Ciao, Michael.
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Python-LDAP-dev mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-ldap-dev
>

Re: problem using python-ldap under fcgi

[Michael S. <mi...@st...>]

Matt Bartolome wrote:
> Hey Michael. Thank you for your response. I modified LDAPObject.c and
> ldapcontrol.c to use the solution described here:
> 
> https://bugs.launchpad.net/ubuntu/+source/python-cdb/+bug/157251
> 
> Looks like it is something specific to glibc in ubuntu and the
> "recommended" use of PyObject_Del instead of PyMem_DEL. I've attached
> a patch showing the modifications for anyone else ripping their hair
> out. The glibc python free() error is now gone.

If this is really a more general issue I'd like to see this fixed. So I 
tried your patch. But it seg faults on my openSUSE 11.0 system. I 
noticed some warnings during build (see below).

Ciao, Michael.

--------------------------- snip ---------------------------
[..]
Modules/LDAPObject.c: In function ‘Tuple_to_LDAPMod’:
Modules/LDAPObject.c:126: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/LDAPObject.c:134: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/LDAPObject.c:143: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/LDAPObject.c:146: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/LDAPObject.c:154: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/LDAPObject.c:158: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/LDAPObject.c: In function ‘List_to_LDAPMods’:
Modules/LDAPObject.c:226: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/LDAPObject.c: In function ‘attrs_from_List’:
Modules/LDAPObject.c:273: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall 
-Wstrict-prototypes -fPIC -DHAVE_LIBLDAP_R -DHAVE_SASL -DHAVE_TLS 
-DLDAPMODULE_VERSION=2.3.6 -IModules -I/opt/openldap-HEAD/include 
-I/opt/sasl/include/sasl -I/usr/include/sasl -I/usr/include/python2.6 -c 
Modules/ldapcontrol.c -o build/temp.linux-i686-2.6/Modules/ldapcontrol.o -g
Modules/ldapcontrol.c: In function ‘Tuple_to_LDAPControl’:
Modules/ldapcontrol.c:83: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/ldapcontrol.c:92: warning: passing argument 1 of ‘_PyObject_New’ 
makes pointer from integer without a cast
Modules/ldapcontrol.c: In function ‘List_to_LDAPControls’:
Modules/ldapcontrol.c:139: warning: passing argument 1 of 
‘_PyObject_New’ makes pointer from integer without a cast
[..]

Re: problem using python-ldap under fcgi

[Matt B. <mat...@gm...>]

My apologies on the wild goose chase but after using valgrind on my
fcgi process it is python cx_Oracle (would have never guessed that!)
which triggers the segmentation fault when ldap.initialize() is
called. Why it does this is beyond me but a simple alteration of my
code makes the problem go away completely. I was creating a global
oracle db cursor which I'm now creating inside the functions that use
it.

I'm not sure about the glib c error and patch now. Using the original
release without modification works so I will leave it at that.

Thanks,
Matt

On Wed, Aug 27, 2008 at 12:26 AM, Michael Ströder <mi...@st...> wrote:
> Matt Bartolome wrote:
>>
>> Hey Michael. Thank you for your response. I modified LDAPObject.c and
>> ldapcontrol.c to use the solution described here:
>>
>> https://bugs.launchpad.net/ubuntu/+source/python-cdb/+bug/157251
>>
>> Looks like it is something specific to glibc in ubuntu and the
>> "recommended" use of PyObject_Del instead of PyMem_DEL. I've attached
>> a patch showing the modifications for anyone else ripping their hair
>> out. The glibc python free() error is now gone.
>
> If this is really a more general issue I'd like to see this fixed. So I
> tried your patch. But it seg faults on my openSUSE 11.0 system. I noticed
> some warnings during build (see below).
>
> Ciao, Michael.
>
> --------------------------- snip ---------------------------
> [..]
> Modules/LDAPObject.c: In function 'Tuple_to_LDAPMod':
> Modules/LDAPObject.c:126: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/LDAPObject.c:134: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/LDAPObject.c:143: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/LDAPObject.c:146: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/LDAPObject.c:154: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/LDAPObject.c:158: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/LDAPObject.c: In function 'List_to_LDAPMods':
> Modules/LDAPObject.c:226: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/LDAPObject.c: In function 'attrs_from_List':
> Modules/LDAPObject.c:273: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall
> -Wstrict-prototypes -fPIC -DHAVE_LIBLDAP_R -DHAVE_SASL -DHAVE_TLS
> -DLDAPMODULE_VERSION=2.3.6 -IModules -I/opt/openldap-HEAD/include
> -I/opt/sasl/include/sasl -I/usr/include/sasl -I/usr/include/python2.6 -c
> Modules/ldapcontrol.c -o build/temp.linux-i686-2.6/Modules/ldapcontrol.o -g
> Modules/ldapcontrol.c: In function 'Tuple_to_LDAPControl':
> Modules/ldapcontrol.c:83: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/ldapcontrol.c:92: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> Modules/ldapcontrol.c: In function 'List_to_LDAPControls':
> Modules/ldapcontrol.c:139: warning: passing argument 1 of '_PyObject_New'
> makes pointer from integer without a cast
> [..]
>

Re: problem using python-ldap under fcgi

[Michael S. <mi...@st...>]

Matt Bartolome wrote:
> My apologies on the wild goose chase but after using valgrind on my
> fcgi process it is python cx_Oracle (would have never guessed that!)
> which triggers the segmentation fault when ldap.initialize() is
> called. Why it does this is beyond me but a simple alteration of my
> code makes the problem go away completely. I was creating a global
> oracle db cursor which I'm now creating inside the functions that use
> it.
> 
> I'm not sure about the glib c error and patch now. Using the original
> release without modification works so I will leave it at that.

Glad you figured out what the issue was. It's good if you don't run a 
patched version of python-ldap. In general and thanks to the 
contributors who provided patches in the past python-ldap seems fairly 
stable.

But let's look at the blog entry which convinced you to try patching 
python-ldap (see 
http://www.notes.xythian.net/2007/10/24/python-cdb-032-52ubuntu2-with-python-25-causes-double-free-corruption-crash-on-dealloc/):

"Some other searching suggests that python-cdb’s use of PyMem_DEL is no 
longer recommended."

That's pretty unprecise, not even a single URL.

But if somebody can add more detailed information to this it could be 
helpful to dive into this. IMO an admirable goal of python-ldap is not 
to fall back behind what's considered current best practice when writing 
extension modules for CPython.

Ciao, Michael.

Re: problem using python-ldap under fcgi

[Matej V. <ve...@de...>]

Michael Ströder <mi...@st...> writes:

> But let's look at the blog entry which convinced you to try patching 
> python-ldap (see 
> http://www.notes.xythian.net/2007/10/24/python-cdb-032-52ubuntu2-with-python-25-causes-double-free-corruption-crash-on-dealloc/):
>
> "Some other searching suggests that python-cdb’s use of PyMem_DEL is no 
> longer recommended."
>
> That's pretty unprecise, not even a single URL.
>
> But if somebody can add more detailed information to this it could be 
> helpful to dive into this. IMO an admirable goal of python-ldap is not 
> to fall back behind what's considered current best practice when writing 
> extension modules for CPython.

I think the blog writer was bitten by <http://bugs.debian.org/468993>.
In Python 2.5, PyMem_DEL and PyObject_Del are no longer interchangeable;
memory allocated by PyMem_NEW needs to be deallocated by PyMem_DEL, and
likewise for PyObject_New and PyObject_Del.

A similar (automated) report was filed for python-ldap, and I've
verified that it doesn't apply -- <http://bugs.debian.org/468995>.

Cheers,

Matej

Re: problem using python-ldap under fcgi

[Matt B. <mat...@gm...>]

On Wed, Aug 27, 2008 at 1:30 PM, Michael Ströder <mi...@st...> wrote:
> Matt Bartolome wrote:
>> My apologies on the wild goose chase but after using valgrind on my
>> fcgi process it is python cx_Oracle (would have never guessed that!)
>> which triggers the segmentation fault when ldap.initialize() is
>> called. Why it does this is beyond me but a simple alteration of my
>> code makes the problem go away completely. I was creating a global
>> oracle db cursor which I'm now creating inside the functions that use
>> it.
>>
>> I'm not sure about the glib c error and patch now. Using the original
>> release without modification works so I will leave it at that.
>
> Glad you figured out what the issue was. It's good if you don't run a
> patched version of python-ldap. In general and thanks to the
> contributors who provided patches in the past python-ldap seems fairly
> stable.

I'll take a stab at this. I'll give you fair warning though that I
don't know much about C. It looks like the modifications would be
fairly straight forward though given I can find the recommended usage
and documentation. I left off at the type cast build warnings so I
just need to figure that in plus make sure it is backwards compatible
(I saw an example showing how to do that).

>
> But let's look at the blog entry which convinced you to try patching
> python-ldap (see
> http://www.notes.xythian.net/2007/10/24/python-cdb-032-52ubuntu2-with-python-25-causes-double-free-corruption-crash-on-dealloc/):
>
> "Some other searching suggests that python-cdb's use of PyMem_DEL is no
> longer recommended."
>
> That's pretty unprecise, not even a single URL.
>
> But if somebody can add more detailed information to this it could be
> helpful to dive into this. IMO an admirable goal of python-ldap is not
> to fall back behind what's considered current best practice when writing
> extension modules for CPython.
>
> Ciao, Michael.
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Python-LDAP-dev mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-ldap-dev
>

Re: problem using python-ldap under fcgi

[Michael S. <mi...@st...>]

Matt Bartolome wrote:
> I'll take a stab at this. I'll give you fair warning though that I
> don't know much about C. It looks like the modifications would be
> fairly straight forward though given I can find the recommended usage
> and documentation. I left off at the type cast build warnings so I
> just need to figure that in plus make sure it is backwards compatible
> (I saw an example showing how to do that).

Given Matej's answer and the fact that your problem is fixed it seems to 
me nothing has to be done.

Ciao, Michael.