#2035 Agent crashes on AIX with hrStorage
It does not crash every time. I am running AIX 5.3 (AIX aixdev24 3 5 00C163BF4C00). This is Net-snmp 5.4.2.1 though I have tried the other current versions as well.
I compiled using the following options:
--prefix=/opt/net-snmp --enable-as-needed --disable-shared --without-kmem-usage --with-persistent-directory=/var/net-snmp --with-default-snmp-version=2 --with-sys-contact=unixadmin@leggmason.com --with-sys-location=LMTS --with-logfile=/var/log/snmpd.log CC=cc_r
The command line options I use:
/opt/net-snmp/sbin/snmpd -c /opt/net-snmp/share/snmp/snmpd_base.conf,/opt/net-snmp/share/snmp/snmpd_custom.conf,/opt/net-snmp/share/snmp/snmpd_cacti.conf -f -Lf /var/log/snmpd.log
If I also add -I -hr_storage to the command line the agent will not crash, but then I cannot retreive filesystem data.
Here is the last 25 lines of output in debug mode at the crash:
trace: _callback_unlock(): callback.c, 164:
9:callback:lock: unlocked (APP,null)
trace: netsnmp_add_varbind_to_cache(): snmp_agent.c, 1870:
snmp_agent: add_vb_to_cache(110331430, 1, HOST-RESOURCES-MIB::hrStorageType.31, 1101d2b50)
trace: netsnmp_add_varbind_to_cache(): snmp_agent.c, 1941:
snmp_agent: tp->start HOST-RESOURCES-MIB::hrStorageType, tp->end HOST-RESOURCES-MIB::hrStorageDescr,
trace: netsnmp_add_varbind_to_cache(): snmp_agent.c, 1961:
verbose:asp: asp 110331430 reqinfo 11032d170 assigned to request
trace: netsnmp_add_varbind_to_cache(): snmp_agent.c, 1968:
verbose:asp: asp 110331430 reqinfo 11032d170 assigned to request
trace: netsnmp_call_handlers(): agent_handler.c, 510:
handler📲 main handler bulk_to_next
trace: netsnmp_call_handler(): agent_handler.c, 430:
handler📲 calling handler bulk_to_next for mode GET
trace: netsnmp_call_handler(): agent_handler.c, 438:
handler:returned: handler bulk_to_next returned 0
trace: netsnmp_call_handler(): agent_handler.c, 430:
handler📲 calling handler old_api for mode GET
trace: header_hrstoreEntry(): host/hr_storage.c, 373:
host/hr_storage: var_hrstoreEntry: request HOST-RESOURCES-MIB::hrStorageType.31 exact=1
trace: header_hrstoreEntry(): host/hr_storage.c, 381:
host/hr_storage: var_hrstoreEntry: compare HOST-RESOURCES-MIB::hrStorageType => 1
(index 31 ....HOST-RESOURCES-MIB::hrStorageType.31
trace: header_hrstoreEntry(): host/hr_storage.c, 476:
host/hr_storage: var_hrstoreEntry: process HOST-RESOURCES-MIB::hrStorageType.31 (ffffffff)
Here is the gdb trace of the crash:
root@aixdev24 # gdb /opt/net-snmp/sbin/snmpd
GNU gdb 6.0
Copyright 2003 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "powerpc-ibm-aix5.1.0.0"...
(gdb) run -c /opt/net-snmp/share/snmp/snmpd_base.conf,/opt/net-snmp/share/snmp/snmpd_custom.conf,/opt/net-snmp/share/snmp/snmpd_cacti.conf -f -Lf /var/log/sn
mpd.log
Starting program: /opt/net-snmp/sbin/snmpd -c /opt/net-snmp/share/snmp/snmpd_base.conf,/opt/net-snmp/share/snmp/snmpd_custom.conf,/opt/net-snmp/share/snmp/sn
mpd_cacti.conf -f -Lf /var/log/snmpd.log
Program received signal SIGSEGV, Segmentation fault.
0x0000d410 in ?? ()
(gdb) where
#0 0x0000d410 in ?? ()
(gdb) list
411 snmpd.c: A file or directory in the path name does not exist..
in snmpd.c
(gdb)
I also experienced this problem. The problem seems to be in agent/mibgroup/host/hr_storage.c:560.
At this point the system function hasmntopt is called with the struct HRFS_entry. HRFS_entry is defined in mntent.h like this:
extern char *hasmntopt(const struct mntent *, const char *);
HRFS_entry is for AIX defined in hr_filesys.c like this:
struct HRFS_entry {
char *HRFS_name;
char *HRFS_mount;
int HRFS_type;
int HRFS_flags;
} *HRFS_entry;
The definition in mntent.h from AIX is:
struct mntent {
char *mnt_fsname; /* Name of block special */
char *mnt_dir; /* Mount point */
char *mnt_type; /* vfs type */
char *mnt_opts; /* mount options */
int mnt_freq; /* dump frequency */
int mnt_passno; /* fsck pass number */
};
So this explains to me why the SEGV occurs.
The workaround for me was to disable the function hasmntopt() in Net-SNMP by changing the "configure" file in line 39513 by just removing "hasmntopt" from the test. So it will be undefined and not used. The only impact is that I don't get the information if I got a ro or rw filesystem or in case of a loop filesystem, that it is a "Removable Disk".
The final fix could be changing the struct definition or using the AIX native struct like on all other operating systems.
I've applied a stop-gap fix that simply skips the use of hasmntopt() in hr_storage.c for AIX.
Improved patches welcome!
Thanks for the bug report!
We've fixed the problem in the 5.2.x, 5.3.x
and 5.4.x code branches and the main development
tree, so it should be fixed in future releases
of the Net-SNMP package.
SVN Rev. 17562.