segmentation violation during cleanup
Brought to you by:
gr84b8,
johnksterling
Menu
▾
▴
I recently made a FreeBSD port of mod_cplusplus so I can use it to run sipX. I have encountered a bug that I think is in mod_cplusplus, though it could also come from sipX. When I run httpd to test an httpd.conf figuration file with httpd -t -f httpd.conf (and also when I run httpd normally), it core dumps on me when httpd exits.
When I put httpd into a debugger with the core file, I see:
(gdb) bt
#0 0x28582398 in ?? ()
#1 0x28215f19 in run_cleanups () from /usr/local/lib/libapr-1.so.2
#2 0x28216a32 in apr_pool_destroy () from /usr/local/lib/libapr-1.so.2
#3 0x08061148 in destroy_and_exit_process ()
#4 0x08061daa in main ()
(gdb) l *0x28582398
No source file for address 0x28582398.
If I add a breakpoint to destroy_and_exit_process(), I can get a source line number for that memory address:
Breakpoint 5, 0x0806113c in destroy_and_exit_process ()
(gdb) l *0x28582398
0x28582398 is in delete_handler (apache_handler.cpp:99).
94 {
95 CALL_REQ_FUNCTION(logger);
96 }
97
98 static apr_status_t delete_handler(void *data)
99 {
100 ApacheHandler *handler = (ApacheHandler *)data;
101 delete handler;
102 return APR_SUCCESS;
103 }
(gdb)
This is telling me that mod_cplusplus is unloaded before delete_handler is called as a pool cleanup function from run_cleanups().
I can also verify this from the output of "info sharedlibrary" when I check from my breakpoint in destroy_and_exit_process() versus after the SIGSEGV.
Before SIGSEGV:
(gdb) info sharedlibrary
From To Syms Read Shared Object Library
0x280d0414 0x280df43c Yes /lib/libm.so.4
0x280e9188 0x280f5b9c Yes /usr/local/lib/libaprutil-1.so.2
0x280fa090 0x2810e6e8 Yes /usr/local/lib/libexpat.so.6
0x281174a0 0x28128d20 Yes /usr/local/lib/libiconv.so.3
0x2820b7a8 0x28220770 Yes /usr/local/lib/libapr-1.so.2
0x28223e90 0x28227b00 Yes /lib/libcrypt.so.3
0x28240080 0x282596c4 Yes /lib/libpthread.so.2
0x2827ea80 0x2831e42c Yes /lib/libc.so.6
0x28098b04 0x280b2c70 Yes /libexec/ld-elf.so.1
0x283467f8 0x28346c94 Yes /usr/local/libexec/apache22/mod_authn_file.so
0x2834882c 0x28348c04 Yes /usr/local/libexec/apache22/mod_authn_dbm.so
0x2834a77c 0x2834aafc Yes /usr/local/libexec/apache22/mod_authn_anon.so
0x2834c730 0x2834c950 Yes /usr/local/libexec/apache22/mod_authn_default.so
0x2834e924 0x2834eddc Yes /usr/local/libexec/apache22/mod_authn_alias.so
0x28351978 0x283520cc Yes /usr/local/libexec/apache22/mod_authz_host.so
0x28354b98 0x28355324 Yes /usr/local/libexec/apache22/mod_authz_groupfile.so
0x28357790 0x28357ac0 Yes /usr/local/libexec/apache22/mod_authz_user.so
0x28359a10 0x2835a034 Yes /usr/local/libexec/apache22/mod_authz_dbm.so
0x2835c87c 0x2835ce44 Yes /usr/local/libexec/apache22/mod_authz_owner.so
0x2835f6b4 0x2835f8e0 Yes /usr/local/libexec/apache22/mod_authz_default.so
0x283619ec 0x28361fdc Yes /usr/local/libexec/apache22/mod_auth_basic.so
0x28365328 0x28368268 Yes /usr/local/libexec/apache22/mod_auth_digest.so
0x2836bd3c 0x2836c57c Yes /usr/local/libexec/apache22/mod_file_cache.so
0x2836f9f8 0x28372fd0 Yes /usr/local/libexec/apache22/mod_cache.so
0x283760ec 0x28377d80 Yes /usr/local/libexec/apache22/mod_disk_cache.so
0x2837b68c 0x28380c54 Yes /usr/local/libexec/apache22/mod_include.so
0x28383c9c 0x2838502c Yes /usr/local/libexec/apache22/mod_filter.so
0x28387b78 0x283893b8 Yes /usr/local/libexec/apache22/mod_charset_lite.so
0x2838bfbc 0x2838dc50 Yes /usr/local/libexec/apache22/mod_deflate.so
0x283916d4 0x2839b03c Yes /lib/libz.so.3
0x283a2498 0x283a473c Yes /usr/local/libexec/apache22/mod_log_config.so
0x283a6928 0x283a6d90 Yes /usr/local/libexec/apache22/mod_logio.so
0x283a8840 0x283a8c0c Yes /usr/local/libexec/apache22/mod_env.so
0x283ab10c 0x283ada2c Yes /usr/local/libexec/apache22/mod_mime_magic.so
0x283b0a50 0x283b10c0 Yes /usr/local/libexec/apache22/mod_cern_meta.so
0x283b3b90 0x283b4588 Yes /usr/local/libexec/apache22/mod_expires.so
0x283b6ee8 0x283b7f78 Yes /usr/local/libexec/apache22/mod_headers.so
0x283bab8c 0x283bb5e4 Yes /usr/local/libexec/apache22/mod_usertrack.so
0x283bd89c 0x283be1b8 Yes /usr/local/libexec/apache22/mod_unique_id.so
0x283c0adc 0x283c15cc Yes /usr/local/libexec/apache22/mod_setenvif.so
0x283c37b8 0x283c3d7c Yes /usr/local/libexec/apache22/mod_version.so
0x283ce34c 0x283e073c Yes /usr/local/libexec/apache22/mod_ssl.so
0x283f4e28 0x28411790 Yes /usr/lib/libssl.so.4
0x2844d0f0 0x284dd2cc Yes /lib/libcrypto.so.4
0x2850fe84 0x2851136c Yes /usr/local/libexec/apache22/mod_mime.so
0x28515c64 0x2852061c Yes /usr/local/libexec/apache22/mod_dav.so
0x28526d10 0x28528a90 Yes /usr/local/libexec/apache22/mod_status.so
0x2852c8a0 0x28530698 Yes /usr/local/libexec/apache22/mod_autoindex.so
0x28533848 0x28533c7c Yes /usr/local/libexec/apache22/mod_asis.so
0x2853611c 0x28537578 Yes /usr/local/libexec/apache22/mod_info.so
0x2853ba64 0x2853dbd0 Yes /usr/local/libexec/apache22/mod_cgi.so
0x28541e5c 0x28546f9c Yes /usr/local/libexec/apache22/mod_dav_fs.so
0x2854a930 0x2854b4a8 Yes /usr/local/libexec/apache22/mod_vhost_alias.so
0x2854e6a8 0x28552b30 Yes /usr/local/libexec/apache22/mod_negotiation.so
0x28555884 0x28555e64 Yes /usr/local/libexec/apache22/mod_dir.so
0x28560af4 0x28562378 Yes /usr/local/libexec/apache22/mod_imagemap.so
0x285648f4 0x28564e20 Yes /usr/local/libexec/apache22/mod_actions.so
0x28566c10 0x285679c8 Yes /usr/local/libexec/apache22/mod_speling.so
0x2856994c 0x2856a034 Yes /usr/local/libexec/apache22/mod_userdir.so
0x2856cb94 0x2856d764 Yes /usr/local/libexec/apache22/mod_alias.so
0x28571600 0x28578e6c Yes /usr/local/libexec/apache22/mod_rewrite.so
0x2857c840 0x2857cb18 Yes /usr/local/libexec/apache22/mod_suexec.so
0x28581c38 0x28589be4 Yes /usr/local/libexec/apache22/mod_cplusplus.so
0x285cdf1c 0x28632bf0 Yes /usr/lib/libstdc++.so.5
0x2865a58c 0x2865ca08 Yes /usr/local/libexec/apache22/libmod_sipxauth.so
0x286769d8 0x286e0568 Yes /usr/local/sipx/lib/libsipXcommserver.so.1
0x287055d4 0x2874b420 Yes /usr/local/lib/libodbc.so.1
0x2878f834 0x28826870 Yes /usr/local/sipx/lib/libsipXtack.so.2
0x288829b8 0x288c1ddc Yes /usr/local/sipx/lib/libsipXport.so.2
0x288de404 0x288eb634 Yes /usr/local/lib/libpcre.so.0
After SIGSEGV:
(gdb) c
Continuing.
Program received signal SIGSEGV, Segmentation fault.
0x28582398 in ?? ()
(gdb) info sharedlibrary
From To Syms Read Shared Object Library
0x280d0414 0x280df43c Yes /lib/libm.so.4
0x280e9188 0x280f5b9c Yes /usr/local/lib/libaprutil-1.so.2
0x280fa090 0x2810e6e8 Yes /usr/local/lib/libexpat.so.6
0x281174a0 0x28128d20 Yes /usr/local/lib/libiconv.so.3
0x2820b7a8 0x28220770 Yes /usr/local/lib/libapr-1.so.2
0x28223e90 0x28227b00 Yes /lib/libcrypt.so.3
0x28240080 0x282596c4 Yes /lib/libpthread.so.2
0x2827ea80 0x2831e42c Yes /lib/libc.so.6
0x28098b04 0x280b2c70 Yes /libexec/ld-elf.so.1
0x283f4e28 0x28411790 Yes /usr/lib/libssl.so.4
0x2844d0f0 0x284dd2cc Yes /lib/libcrypto.so.4
0x285cdf1c 0x28632bf0 Yes /usr/lib/libstdc++.so.5
0x2865a58c 0x2865ca08 Yes /usr/local/libexec/apache22/libmod_sipxauth.so
0x286769d8 0x286e0568 Yes /usr/local/sipx/lib/libsipXcommserver.so.1
0x287055d4 0x2874b420 Yes /usr/local/lib/libodbc.so.1
0x2878f834 0x28826870 Yes /usr/local/sipx/lib/libsipXtack.so.2
0x288829b8 0x288c1ddc Yes /usr/local/sipx/lib/libsipXport.so.2
0x288de404 0x288eb634 Yes /usr/local/lib/libpcre.so.0
(gdb)
Unfortunately, I do not know enough about the workings of apr or apache modules to track this down any further.
mike
Hi Mike -
Thanks for the post. Couple of questions:
1) Is this a patched version of mod_cplusplus? or stock from cvs?
2) What MPM are you running?
3) What's your uname -a?
There is a related issue someone reported about that cleanup. The intention is to have it called when the child is closed, and it is not happening then. I'll be looking into it this week.
I'm building from version 1.5.2. I do add one trivial patch during the FreeBSD build process. configure.in uses "==" to compare text strings in a couple places. That works with bash, but not sh. I changed those constructs from:
! if test "$SHLIB_EXT" == "" ; then
to
! if test "x$SHLIB_EXT" = "x" ; then
I believe the MPM is prefork.
As for uname -a:
FreeBSD man.boogie.com 6.2-PRERELEASE FreeBSD 6.2-PRERELEASE #0: Fri Oct 13 10:41:27 MDT 2006 root@man.boogie.com:/usr/obj/usr/src/sys/BOOGIE i386
mike
Hi MIke -
I have made a fix to cvs that I think will solve your issue (and others). Do you have access to the CVS version to test? If not I can re-package a new release for this.
Thanks -
John
John,
Sorry I'm late getting back to you. I've been busy with some other
work-related stuff. I will do my best to test your changes later today.
mike
No problem - thanks for following up. Let me know if you run any more problems.
I just updated to the new code, and it does fix the core dump problem.
Thanks!
I do have one minor patch for you though. If you add this, then the
configure script will work on OSes that do not use bash for the Bourne
shell. The construct [ $foo == $bar ] is a bashism. For /bin/sh without
bash extensions, you want [ "x$foo" = "x$bar" ]. Vanilla
sh does not have the double equal operator. The x prefix is there to
avoid syntax errors if one of the variables is empty.
Here is the patch I'm using in the FreeBSD port (tabs have probably been
converted to spaces during copy/paste):
--- configure.in.orig Sun Aug 7 14:36:49 2005
+++ configure.in Tue Feb 6 13:34:52 2007
@@ -9,7 +9,7 @@
dnl hack to get shlib extension
AC_MSG_CHECKING(what the shared library extension is)
eval SHLIB_EXT=\"$shrext\"
-if test "$SHLIB_EXT" == "" ; then
+if test "x$SHLIB_EXT" = "x" ; then
AC_MSG_RESULT(could not determine shared library extension. Defaulting
to .so)
SHLIB_EXT=".so"
fi
@@ -58,12 +58,12 @@
fi
],)
-if test "$INCLUDES" == "" ; then
+if test "x$INCLUDES" = "x" ; then
AC_MSG_ERROR(could not generate includes with apxs set to "$APXS" - you
must specify a valid path to apxs using the --with-apxs flag)
fi
AC_CHECK_LIB(stdc++, __gxx_personality_v0, LIB_STDCPP="-lstdc++")
- if test "$LIB_STDCPP" == "" ; then
+ if test "x$LIB_STDCPP" = "x" ; then
CPPFLAGS="$CPPFLAGS -DNO_STDCPP"
fi
CPLUSPLUS_BUILDDIR=`pwd`
Thanks for the patch - I'll make sure to resolve the issue. Its tough to get the right combination for every environment in those build scripts...
If you decide to re-roll the 1.5.3 release with the configure file tweaks, could you
let me know so I can keep the FreeBSD port in sync?
Thanks,
mike