Menu ▾ ▴

#16 Crash while blocking blacklist entries using ipfw

fixed
Kevin Zheng
None
High
2015-06-04
2014-09-30
westvovik
No

i have this message in log
kernel: pid 64493 (sshguard), uid 0: exited on signal 10 (core dumped)

ps axgwww | grep sshguard
25116 ?? Is 0:00.06 /usr/local/sbin/sshguard -a 3 -b 10:/var/db/sshguard/blacklist -w /var/db/sshguard/whitelist

Here's a backtrace of one of sshguard's coredumps:

root@online:/var/log # gdb /usr/local/sbin/sshguard /sshguard.core
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...(no debugging symbols found)...
Core was generated by `sshguard'.
Program terminated with signal 10, Bus error.
Reading symbols from /lib/libthr.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib/libthr.so.3
Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.7
Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done.
Loaded symbols for /libexec/ld-elf.so.1

0 0x0000000800b7c28d in getenv () from /lib/libc.so.7

[New Thread 801007800 (LWP 107255/sshguard)]
[New Thread 801007400 (LWP 110216/sshguard)]
(gdb) bt

0 0x0000000800b7c28d in getenv () from /lib/libc.so.7

1 0x0000000800b60491 in tzsetwall () from /lib/libc.so.7

2 0x0000000800b60792 in localtime_r () from /lib/libc.so.7

3 0x0000000800b608f0 in ctime_r () from /lib/libc.so.7

4 0x0000000800b5bfb8 in vsyslog () from /lib/libc.so.7

5 0x0000000800b5beb8 in syslog () from /lib/libc.so.7

uname -a
FreeBSD online.xxx.ru 9.1-RELEASE-p4 FreeBSD 9.1-RELEASE-p4 #12: Thu Jun 20 18:00:14 MSK 2013 root@online.xxx.ru:/usr/obj/usr/src/sys/MYKERNEL amd64

I hope this is enough info, I can make a debug build if needed.

At first sshguard normally worked, but then started giving out in a log (core dumped)

after

/ etc/rc.d/syslogd restart

Didn't help, I didn't start working

That helped that I at first deleted /var/db/sshguard/blacklist the file

then

/ etc/rc.d/syslogd stop
/ etc/rc.d/syslogd start

1 Attachments
patch-ipfw.diff

Discussion

  • rustamabd

    rustamabd - 2015-05-24

    Same error happening in 10.1-RELEASE amd64:

    >uname -a
FreeBSD xxx 10.1-RELEASE-p9 FreeBSD 10.1-RELEASE-p9 #0: Tue Apr  7 01:09:46 UTC 2015     root@xxx:/usr/obj/usr/src/sys/GENERIC  amd64

    Backtrace:

    #0  0x0000000800c08b03 in getenv () from /lib/libc.so.7
[New Thread 801406400 (LWP 100351/sshguard)]
(gdb) bt
#0  0x0000000800c08b03 in getenv () from /lib/libc.so.7
#1  0x0000000800b568f6 in execvp () from /lib/libc.so.7
#2  0x000000000040a299 in ?? ()
#3  0x000000000040a356 in ?? ()
#4  0x000000000040274e in ?? ()
#5  0x000000000040236f in ?? ()
#6  0x00000008006a8000 in ?? ()
#7  0x0000000000000000 in ?? ()
     

    Last edit: rustamabd 2015-05-24

  • Kevin Zheng

    Kevin Zheng - 2015-05-24

    Just to check -- is this from the 1.6.0 release? Latest version from ports?

     

  • rustamabd

    rustamabd - 2015-05-24

    This is sshguard-ipfw-1.6.0_1 from packages.

     

  • Kevin Zheng

    Kevin Zheng - 2015-05-24

    Let me see if I'm understanding the problem right: you're running SSHGuard from syslogd, and when you restart syslogd SSHGuard crashes, but only when you're using a blacklist? And after it crashes, it also crashes more when using the same blacklist?

    If so, could you attach a 'broken' blacklist file? A debug build would be helpful, but I think I might be able to understand what's going on from the blacklist file itself.

     

  • rustamabd

    rustamabd - 2015-05-24

    I'm running sshguard as a daemon like this:

    /usr/local/sbin/sshguard -b 40:/var/db/sshguard/blacklist.db -l /var/log/auth.log -l /var/log/exim/mainlog -a 40 -p 420 -s 1200 -w /usr/local/etc/sshguard.whitelist -i /var/run/sshguard.pid

    Previously I ran the basic version (no firewall attachment), now I switched to the version that is linked with IPFW support and it refuses to start, dumping core.

    Update: after I deleted the blacklist file it started up fine.

     
    blacklist.db

  • Kevin Zheng

    Kevin Zheng - 2015-05-24

    Sounds like it might be something interesting going on with the ipfw backend. I used your 'blacklist.db' on pf and everything worked fine. Could you run a debug build and paste your backtrace?

     

  • rustamabd

    rustamabd - 2015-05-26

    Hope this helps...

    #0  0x0000000800bfcb23 in getenv () from /lib/libc.so.7
#1  0x0000000800b54776 in execvp () from /lib/libc.so.7
#2  0x000000000040e7cb in ipfwmod_runcommand (command=0x6911e0 "/sbin/ipfw",
    args=0x691240 "add 55040 drop ip from 208.109.250.139,110.77.140.129,218.87.111.116,50.63.137.18,43.255.188.148,113.195.145.70,182.74.73.134,80.88.167.50,184.168.119.196,94.88.123.26,220.225.7.21,208.109.86.203,182."...) at ipfw.c:279
#3  0x000000000040e951 in fw_block_list (addresses=0x801430080, addrkind=4, service_codes=0x80143a140) at ipfw.c:141
#4  0x00000000004037f2 in process_blacklisted_addresses () at sshguard.c:567
#5  0x0000000000402785 in main (argc=17, argv=0x7fffffffdc30) at sshguard.c:198
     

  • Kevin Zheng

    Kevin Zheng - 2015-05-26

    Thanks! We definitely know what's going on now. An errata notice went on the mailing list with additional details and possible workarounds. Hopefully we'll have this fixed shortly.

     

  • Kevin Zheng

    Kevin Zheng - 2015-05-27
    • Description has changed:

    Diff:

    --- old
+++ new
@@ -33,62 +33,6 @@
 #3  0x0000000800b608f0 in ctime_r () from /lib/libc.so.7
 #4  0x0000000800b5bfb8 in vsyslog () from /lib/libc.so.7
 #5  0x0000000800b5beb8 in syslog () from /lib/libc.so.7
-#6  0x0000000000403c2f in ?? ()
-#7  0x00000000004024d6 in ?? ()
-#8  0x000000000040205e in ?? ()
-#9  0x0000000800661000 in ?? ()
-#10 0x0000000000000000 in ?? ()
-#11 0x0000000000000000 in ?? ()
-#12 0x0000000000000007 in ?? ()
-#13 0x00007fffffffdd58 in ?? ()
-#14 0x00007fffffffdd71 in ?? ()
-#15 0x00007fffffffdd74 in ?? ()
-#16 0x00007fffffffdd76 in ?? ()
-#17 0x00007fffffffdd79 in ?? ()
-#18 0x00007fffffffdd97 in ?? ()
-#19 0x00007fffffffdd9a in ?? ()
-#20 0x0000000000000000 in ?? ()
-#21 0x00007fffffffddb5 in ?? ()
-#22 0x00007fffffffddce in ?? ()
-#23 0x00007fffffffddd9 in ?? ()
-#24 0x00007fffffffdde6 in ?? ()
-#25 0x00007fffffffddf1 in ?? ()
-#26 0x00007fffffffde00 in ?? ()
-#27 0x00007fffffffde10 in ?? ()
-#28 0x00007fffffffde24 in ?? ()
-#29 0x00007fffffffde7b in ?? ()
-#30 0x00007fffffffde85 in ?? ()
-#31 0x00007fffffffdea1 in ?? ()
-#32 0x00007fffffffdeb8 in ?? ()
-#33 0x00007fffffffdec4 in ?? ()
-#34 0x00007fffffffdeca in ?? ()
-#35 0x00007fffffffded6 in ?? ()
-#36 0x00007fffffffdee1 in ?? ()
-#37 0x00007fffffffdeeb in ?? ()
-#38 0x00007fffffffdef6 in ?? ()
-#39 0x00007fffffffdf05 in ?? ()
-#40 0x00007fffffffdf16 in ?? ()
-#41 0x00007fffffffdf2a in ?? ()
-#42 0x00007fffffffdf38 in ?? ()
-#43 0x00007fffffffdf46 in ?? ()
-#44 0x00007fffffffdf52 in ?? ()
-#45 0x00007fffffffdf5f in ?? ()
-#46 0x0000000000000000 in ?? ()
-#47 0x0000000000000003 in ?? ()
-#48 0x0000000000400040 in ?? ()
-#49 0x0000000000000004 in ?? ()
-#50 0x0000000000000038 in ?? ()
-#51 0x0000000000000005 in ?? ()
-#52 0x0000000000000008 in ?? ()
-#53 0x0000000000000006 in ?? ()
-#54 0x0000000000001000 in ?? ()
-#55 0x0000000000000008 in ?? ()
-#56 0x0000000000000000 in ?? ()
-#57 0x0000000000000009 in ?? ()
-#58 0x0000000000401fd0 in ?? ()
-#59 0x0000000000000007 in ?? ()
-#60 0x0000000800646000 in ?? ()
-#61 0x000000000000000f in ?? ()

 uname -a
 FreeBSD online.xxx.ru 9.1-RELEASE-p4 FreeBSD 9.1-RELEASE-p4 #12: Thu Jun 20 18:00:14 MSK 2013     root@online.xxx.ru:/usr/obj/usr/src/sys/MYKERNEL  amd64
    • status: open --> feedback
    • assigned_to: Kevin Zheng
    • Attachments has changed:

    Diff:

    --- old
+++ new
@@ -0,0 +1 @@
+patch-ipfw.diff (8.8 kB; text/x-patch)
    • Priority: 5 --> High
     

  • Kevin Zheng

    Kevin Zheng - 2015-05-27

    A patch is attached (and on the mailing list). Could you test it and see if it works?

     

  • Kevin Zheng

    Kevin Zheng - 2015-06-04
    • summary: exited on signal 10 (core dumped) --> Crash while blocking blacklist entries using ipfw
    • status: feedback --> fixed
     

  • Kevin Zheng

    Kevin Zheng - 2015-06-04

    Fixed in 3e8591f with the new ipfw backend.

     

Log in to post a comment.