Menu
▾
▴
#1475 multithreaded httpslibcurl crash
closed-invalid
5
2015-02-23
2015-01-26
No
Hi .
We have a problem with a tool (collectd) working with libcurl that is crashing only whem doing multiple HTTPS queries over apache+ssl. (it gathers statistics data from apache mod_status and sends to a database backend)
The tool is working good with HTTP and single HTTPS queries.
Collectd was linked with curl-7.35.0
System: RHEL 5.7 (Linux 2.6.18-274.el5 #1 SMP Fri Jul 8 17:36:59 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux)
OpenSSL: openssl-0.9.8e-27.el5_10.3
I have also an opened Issue on collectd site
https://github.com/collectd/collectd/issues/858
The code seems correctly made for a multithread environment.
https://github.com/collectd/collectd/blob/master/src/apache.c
but after a while running ( sometimes hours sometimes days) we get a SEGFAULT and the tool crash. This only happens when more than one HTTPS apache instance configured
This is the backtrace generated
Core was generated by `/opt/collectd/sbin/collectd -C /opt/collectd/etc/collectd.conf'.
Program terminated with signal 11, Segmentation fault.
#0 0x0000003d5787c39b in ?? ()
(gdb) backtrace
#0 0x0000003d5787c39b in ?? ()
#1 0x00000036efcdeb5e in SHA1_Update () from /lib64/libcrypto.so.6
#2 0x00000036efcdbd0f in ?? () from /lib64/libcrypto.so.6
#3 0x00000036efcdb5a6 in ?? () from /lib64/libcrypto.so.6
#4 0x00000036f001d351 in ssl3_client_hello () from /lib64/libssl.so.6
#5 0x00000036f001e7c9 in ssl3_connect () from /lib64/libssl.so.6
#6 0x00002aed7c3d180d in ossl_connect_common () from /opt/collectd/lib/libcurl.so.4
#7 0x00002aed7c3d273d in Curl_ssl_connect_nonblocking () from /opt/collectd/lib/libcurl.so.4
#8 0x00002aed7c398cde in https_connecting () from /opt/collectd/lib/libcurl.so.4
#9 0x00002aed7c3a5a3e in Curl_protocol_connect () from /opt/collectd/lib/libcurl.so.4
#10 0x00002aed7c3b888f in multi_runsingle () from /opt/collectd/lib/libcurl.so.4
#11 0x00002aed7c3b9895 in curl_multi_perform () from /opt/collectd/lib/libcurl.so.4
#12 0x00002aed7c3b33f4 in curl_easy_perform () from /opt/collectd/lib/libcurl.so.4
#13 0x00002aed7c189521 in apache_read_host (user_data=0xb96a768) at apache.c:615
#14 0x000000000041017f in plugin_read_thread (args=0x0) at plugin.c:462
#15 0x0000003d5840673d in ?? ()
#16 0x0000000000000000 in ?? ()
Could you help us fix this bug ?
Discussion
The webservers that I'm queiying are configured with this cyphers
<ifmodule mod_ssl.c="">
SSLEngine on
SSLProtocol ALL -SSLv2
SSLCipherSuite -ALL:!ADH:RC4+SHA:RC4+MD5:RC4+RSA:3DES:!EDH:!KRB5:+TLSv1:+SSLv3:!SSLv2:!RC2:!DES:!EXP
SSLHonorCipherOrder on
SSLCertificateFile cerfile.cer
SSLCertificateKeyFile keyfile.key
SSLCertificateChainFile chain.cer
SSLSessionCacheTimeout 300
</ifmodule>
So does the application set and use the proper OpenSSL mutex callbacks?
I can check this question if you can give me an example code with the mutex callbacks you are refering
http://curl.haxx.se/libcurl/c/opensslthreadlock.html
I will ask to the collectd developer team, but could be the origin of the crash.
Hi Daniel after review the collectd code it seems not to do this initialization.
we are discussing about how and when to do this thread_setup() because of the plugable behavior of the collectd .
Only some plugins needs to do HTTPS-multithread queries. So the most easy way is to do it in each plugin initialization.
Do you know if we can execute more than once the thread_setup() with different id/locking callback functions ?
Should we do thread_setup() before or after the curl_global_init (CURL_GLOBAL_SSL) ?
Last edit: Toni Moreno 2015-02-02
That example only sets the lock mutexes, is doesn't do any libcurl transfers so it isn't complete.
I'm closing this report now as the lack of mutex callbacks is a reason for crashing multi-threaded SSL programs.
If you need further help or have questions, that's what we have the curl-library mailing list for!
Hi Daniel , I've patched with your solution after 1 week it seems to be working fine.. Thank you very much !!