Menu
▾
▴
1 Attachments
#1208 curl is unable to load non-default openssl engines
curl is unable to load non-default openssl engines, because openssl isn't initialized properly - curl ignores the openssl configutration file. for example curl cannot load ccgost engine, provided since 1.0.0:
# /usr/local/openssl/bin/openssl engine
(rsax) RSAX engine support
(rdrand) Intel RDRAND engine
(dynamic) Dynamic engine loading support
(gost) Reference implementation of GOST engine
# ldd /usr/local/curl/bin/curl
linux-vdso.so.1 => (0x00007fff0c9ff000)
libcurl.so.4 => /usr/local/curl/lib/libcurl.so.4 (0x00007fb5f1f17000)
libz.so.1 => /usr/lib/libz.so.1 (0x00007fb5f1d00000)
librt.so.1 => /lib/librt.so.1 (0x00007fb5f1af7000)
libc.so.6 => /lib/libc.so.6 (0x00007fb5f1795000)
libssl.so.1.0.0 => /usr/local/openssl/lib/libssl.so.1.0.0 (0x00007fb5f152c000)
libcrypto.so.1.0.0 => /usr/local/openssl/lib/libcrypto.so.1.0.0 (0x00007fb5f1151000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00007fb5f0f35000)
/lib64/ld-linux-x86-64.so.2 (0x00007fb5f2178000)
libdl.so.2 => /lib/libdl.so.2 (0x00007fb5f0d31000)
# /usr/local/curl/bin//curl --engine list
Build-time engines:
rsax
rdrand
dynamic
"It is strongly recommended that all new applications call OPENSSL_config() or the more sophisticated functions such as CONF_modules_load() during initialization (that is before starting any threads). By doing this an application does not need to keep track of all configuration options and some new functionality can be supported automatically." - curl totally ignores this openssl note.
The patch provided fixes the issue.
Thanks. The use of that function needs to be conditional as we still support OpenSSL 0.9.6 and this function was added in 0.9.7 according to the OpenSSL docs. So a little adjustment of configure.ac and an updated patch would be great!
Finally, let me say that "curl totally ignores this openssl note" is a bit of a misnomer. The OpenSSL API is absurdly confusing, inconsistent, moving-target and a never-ending nuisance for us. We truly do our best in trying to follow the expected and recommended procedure. We don't "totally ignore" any docs, but we sometimes totally fail to find the docs we should've read...
Yup, I'm sorry for being too hard. I agree the use of this function must be conditional - not only in the way of handling old versions with #ifdef: I just got the same issue with openssl PHP extension, which I tried to fix the same way; it appeared that calling OPENSSL_config() two times in two different modules (though curl extension doesn't do it directly) leads to openssl configuration error. So, this seems to be not that trivial. I've also found old discussions in the mailing list - this workaround is known long ago, seems it has it own reasons for not being commited.
So can CONF_modules_load_file() with a NULL filename be used instead? Possibly with the CONF_MFLAGS_IGNORE_ERRORS option?
This definitely needs some decent testing; nobody wants to make things worse :); I'm on vacation for now, I'll try to come here next week with a piece of tested code. My intention is to get a patch that will not only add the config handling capability, but also won't break anything in PHP, with curl module enabled and regardless of the module loading order. Right now it works only if modules are loaded in particular order, because the same patch for php openssl module is also known and widely spread across the php bugtracker.
Any news?
This report will be closed shortly unless we get more feedback/help. It will then instead become a KNOWN_BUG.
Closed for now. This is now documented as KNOWN_BUG #83. Hopefully someone in possession of external openssl engines can help us work on this in the future.
"We don't "totally ignore" any docs, but we sometimes totally fail to find the docs we should've read." OpenSSL has some of the worst documentation on the web. Don't worry about it.
Viega, Messier, and Chandra's "Network Security with OpenSSL" is the book I use most often (I try not to bother with the docs). http://shop.oreilly.com/product/9780596002701.do.
An example of initialization can be found at http://wiki.openssl.org/index.php/Libcrypto_API. It was written by a fellow named Matt who is good about checking his work. Initializing should be as simple as calling OPENSSL_config(). OPENSSL_config(), in turn, calls OPENSSL_load_builtin_modules() and ENGINE_load_builtin_engines(). See, for example, http://www.opensource.apple.com/source/OpenSSL098/OpenSSL098-35.1/src/crypto/conf/conf_sap.c.
An example of loading the Intel RDRAND engine is given at http://wiki.openssl.org/index.php/Random_Numbers#Hardware. Its a simple program, and does not call any additional configuration functions. That is, it avoids OPENSSL_config() which avoids ENGINE_load_builtin_engines().
Thanks Jeffrey, but your explanation of how to do it ("just call OPENSSL_config") contradicts what drook already seems to have tried and reported to fail in an earlier comment: https://sourceforge.net/p/curl/bugs/1208/#9c55
Here's a Talk page by Patrick Pelletier: http://wiki.opensslfoundation.com/index.php/Talk:Libcrypto_API. It goes into detail on the odd behaviors from all those initialization functions.
Its like a Rube Goldberg machine. Hence the reason I try to avoid calling any of them and attempt do everything myself.
Hi,
The proposed patch break libcurl in the case /usr/lib/ssl/openssl.cnf in unreadable (similar to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=584911)
OpenSSL throw a few lines of errors then the whole thing exit().
Removing the OPENSSL_config call make the problem go away.
It's probably an openssl bug, but maybe adding a check that this file can be open before calling OPENSSL_config would be great.
The problem OPENSSL_config causes is tracked in bug #1401: https://sourceforge.net/p/curl/bugs/1401/
There is a possible fix at https://sourceforge.net/p/curl/bugs/1401/?limit=10&page=1#842a