[Nagios-devel] Bus error while processing comment

[Stefan B. <st...@la...>]

I'm on:
FreeBSD krokodil.zs64.net 8.0-CURRENT FreeBSD 8.0-CURRENT #12: Fri Jun  
12 06:29:20 UTC 2009    ro...@lo...:/usr/obj/usr/src/sys/ 
EISENBOOT  amd64
with FreeBSD sources from June 11.

Nagios starts up, runs a few (almost all?) checks, then crashes with a  
bus error.  The debug log does not contain anything useful.  I've  
tried net-mgmt/nagios (3.0.6) and net-mgmt/nagios-devel (3.1.2) with  
and without embedded perl, to no avail.

After looking through the archives, I found this thread: <http://article.gmane.org/gmane.network.nagios.devel/3311 
 >

I've been having this issue for a couple of months now (nagios isn't  
critical for me, so it had a rather low priority), and I tried various  
things to fix or at least work around it, including wiping /var/spool/ 
nagios.  I now remember that nagios kept running every time I wiped  
the files, and started crashing shortly after I acknowledged a problem.

I've compiled nagios with CFLAGS=-g, and I get this in gdb:

# gdb base/nagios
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and  
you are
welcome to change it and/or distribute copies of it under certain  
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for  
details.
This GDB was configured as "amd64-marcel-freebsd"...
(gdb) r /usr/local/etc/nagios/nagios.cfg
Starting program: /var/ports/work/net-mgmt/nagios-devel/nagios-3.1.2/ 
base/nagios /usr/local/etc/nagios/nagios.cfg
[New LWP 100227]
[New Thread 800c021c0 (LWP 100227)]

Nagios 3.1.2
Copyright (c) 2009 Nagios Core Development Team and Community  
Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 06-23-2009
License: GPL

Website: http://www.nagios.org
Nagios 3.1.2 starting... (PID=55280)
Local time is Tue Jul 28 09:21:56 UTC 2009
Warning: Host 'tivo' has no services associated with it!
[New Thread 800c511c0 (LWP 100449)]

Program received signal SIGBUS, Bus error.
[Switching to Thread 800c021c0 (LWP 100227)]
0x000000000044869a in get_next_comment_by_host (host_name=0x800c6f960  
"slingbox", start=0x800c488e0) at ../common/comments.c:632
632		for(;temp_comment && compare_hashdata(temp_comment- 
 >host_name,NULL,host_name,NULL)<0;temp_comment=temp_comment->nexthash);
(gdb) bt
#0  0x000000000044869a in get_next_comment_by_host  
(host_name=0x800c6f960 "slingbox", start=0x800c488e0) at ../common/ 
comments.c:632
#1  0x0000000000447ad1 in delete_host_acknowledgement_comments  
(hst=0x800c16800) at ../common/comments.c:301
#2  0x00000000004362e6 in handle_host_state (hst=0x800c16800) at  
sehandlers.c:731
#3  0x0000000000412955 in process_host_check_result_3x  
(hst=0x800c16800, new_state=0,
   old_plugin_output=0x800c486a0 "CRITICAL - slingbox.lassitu.de: Host  
unreachable @ 44.128.127.15. rta nan, lost 100%",
   check_options=0, reschedule_check=1, use_cached_result=1,  
check_timestamp_horizon=15) at checks.c:3744
#4  0x00000000004117ac in handle_async_host_check_result_3x  
(temp_host=0x800c16800, queued_check_result=0x800c6b0c0) at checks.c: 
3380
#5  0x000000000040a7c7 in reap_check_results () at checks.c:206
#6  0x000000000042574d in handle_timed_event (event=0x800c86fe0) at  
events.c:1307
#7  0x0000000000424cc8 in event_execution_loop () at events.c:1002
#8  0x000000000040a3c2 in main (argc=2, argv=0x7fffffffea58) at  
nagios.c:833
(gdb) p temp_comment
$1 = (comment *) 0x5a5a5a5a5a5a5a5a

5a is a guard value filled in by FreeBSD's malloc when certain config  
flags are set.  Quoting from the man page:
     J       Each byte of new memory allocated by malloc(), realloc() or
             reallocf() will be initialized to 0xa5.  All memory  
returned by
             free(), realloc() or reallocf() will be initialized to  
0x5a.
             This is intended for debugging and will impact  
performance nega-
             tively.

It seems plausible that some structure was free'd before the loop in  
get_next_comment processes that structure.


Stefan

-- 
Stefan Bethke <st...@la...>   Fon +49 151 14070811