pyopenssl-list

[pyOpenSSL] Does PKey.generate_key lock the GIL?

[Tobias O. <tob...@ta...>]

I am using PyOpenSSL from within Twisted and want to generate new keys without blocking the Twisted networking.

To do so, I use the deferToThread() Twisted feature, which runs functions on a thread from a background thread pool.

However,

PKey.generate_key

still seems to block everything.

Does above function lock the GIL?

Re: [pyOpenSSL] Does PKey.generate_key lock the GIL?

[Phil M. <p.m...@im...>]

On 25/11/11 13:12, Tobias Oberstein wrote:
> I am using PyOpenSSL from within Twisted and want to generate new keys without blocking the Twisted networking.
>
> To do so, I use the deferToThread() Twisted feature, which runs functions on a thread from a background thread pool.
>
> However,
>
> PKey.generate_key
>
> still seems to block everything.
>
> Does above function lock the GIL?

Other way round. The GIL is held unless you explicitly release it, which 
the current source code for that function does not seem to:

http://bazaar.launchpad.net/~exarkun/pyopenssl/trunk/view/head:/OpenSSL/crypto/pkey.c#L39

So AFAICT yes, it will block forever.

Perhaps you could shell out to "openssl rsa" in a subprocess. Not ideal, 
but it won't require source code changes.

Re: [pyOpenSSL] Does PKey.generate_key lock the GIL?

[Tobias O. <tob...@ta...>]

> > However,
> >
> > PKey.generate_key
> >
> > still seems to block everything.
> >
> > Does above function lock the GIL?
> 
> Other way round. The GIL is held unless you explicitly release it, which the
> current source code for that function does not seem to:
> 
> http://bazaar.launchpad.net/~exarkun/pyopenssl/trunk/view/head:/OpenS
> SL/crypto/pkey.c#L39
> 
> So AFAICT yes, it will block forever.
> 
> Perhaps you could shell out to "openssl rsa" in a subprocess. Not ideal, but it
> won't require source code changes.

Thanks for clarifying.

Unfortunately, I am also missing other stuff (like dump pub key from cert to verify that cert imported actually is for a given priv key).

So I checked out M2Crypto.

It seems to release the GIL during key generation ..

The API is ... creative. IOW: it sucks. But I guess thats because it's a SWIG generated OpenSSL wrapper.

Anyway .. will move to M2Crypto.

Re: [pyOpenSSL] Does PKey.generate_key lock the GIL?

[Glyph L. <gl...@tw...>]

On Nov 25, 2011, at 12:13 PM, Tobias Oberstein <tob...@ta...> wrote:

>>> However,
>>> 
>>> PKey.generate_key
>>> 
>>> still seems to block everything.
>>> 
>>> Does above function lock the GIL?
>> 
>> Other way round. The GIL is held unless you explicitly release it, which the
>> current source code for that function does not seem to:
>> 
>> http://bazaar.launchpad.net/~exarkun/pyopenssl/trunk/view/head:/OpenS
>> SL/crypto/pkey.c#L39
>> 
>> So AFAICT yes, it will block forever.
>> 
>> Perhaps you could shell out to "openssl rsa" in a subprocess. Not ideal, but it
>> won't require source code changes.
> 
> Thanks for clarifying.
> 
> Unfortunately, I am also missing other stuff (like dump pub key from cert to verify that cert imported actually is for a given priv key).
> 
> So I checked out M2Crypto.
> 
> It seems to release the GIL during key generation ..
> 
> The API is ... creative. IOW: it sucks. But I guess thats because it's a SWIG generated OpenSSL wrapper.
> 
> Anyway .. will move to M2Crypto.

Rather than fixing, or even reporting, this one bug in pyopenssl?

Re: [pyOpenSSL] Does PKey.generate_key lock the GIL?

[Tobias O. <tob...@ta...>]

> >>> However,
> >>>
> >>> PKey.generate_key
> >>>
> >>> still seems to block everything.
> >>>
> >>> Does above function lock the GIL?
> >>
> >> Other way round. The GIL is held unless you explicitly release it,
> >> which the current source code for that function does not seem to:
> >>
> >>
> http://bazaar.launchpad.net/~exarkun/pyopenssl/trunk/view/head:/OpenS
> >> SL/crypto/pkey.c#L39
> >>
> >> So AFAICT yes, it will block forever.
> >>
> >> Perhaps you could shell out to "openssl rsa" in a subprocess. Not
> >> ideal, but it won't require source code changes.
> >
> > Thanks for clarifying.
> >
> > Unfortunately, I am also missing other stuff (like dump pub key from cert
> to verify that cert imported actually is for a given priv key).
> >
> > So I checked out M2Crypto.
> >
> > It seems to release the GIL during key generation ..
> >
> > The API is ... creative. IOW: it sucks. But I guess thats because it's a SWIG
> generated OpenSSL wrapper.
> >
> > Anyway .. will move to M2Crypto.
> 
> Rather than fixing, or even reporting, this one bug in pyopenssl?

It's 2 "bugs": GIL + no dump_publickey (or something similar whichlets me do above).

I'll factor our code and maybe come back to this .. bit of time pressure right now.

Re: [pyOpenSSL] Does PKey.generate_key lock the GIL?

[Phil M. <p.m...@im...>]

On 11/25/2011 09:34 PM, Glyph Lefkowitz wrote:

>> Anyway .. will move to M2Crypto.
>
> Rather than fixing, or even reporting, this one bug in pyopenssl?

I have opened

https://bugs.launchpad.net/pyopenssl/+bug/896526

...so it doesn't get forgotten.