Menu ▾ ▴

9-STABLE, mpd5, pppoe, constant kernel panics

Open Discussion
fjcst
2013-08-24
2015-05-07

  • fjcst

    fjcst - 2013-08-24

    I'm having some issues running a KVM-virtualized FreeBSD 9-STABLE (somewhat four weeks old, so it has the latest netgraph code). I'm also running a 7.4-RELEASE (also virtualized) system on the same machine which has a way better stability record, but I think it would be way more interesting to find out what is keeping my 9-STABLE from being rock solid instead of relying on FreeBSD 7.4 forever.

    Right now it serves a 4000 clients network as a PPPoE server. There's also another six servers (non-virtualized), all running 7.4-RELEASE, serving the same network, so I can experiment with my 9-STABLE installation.

    I will copy the stack trace from one of the four core.txt files I currently have. If requested I can copy more info, recompile my kernel with debugging flags to test etc. Just ask.

    Unread portion of the kernel message buffer:
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 32079 (mpd5)
trap number             = 12
panic: page fault
cpuid = 0
KDB: stack backtrace:
#0 0xffffffff804b0c86 at kdb_backtrace+0x66
#1 0xffffffff80476bee at panic+0x1ce
#2 0xffffffff806fbc30 at trap_fatal+0x290
#3 0xffffffff806fbf91 at trap_pfault+0x211
#4 0xffffffff806fc544 at trap+0x344
#5 0xffffffff806e5873 at calltrap+0x8
#6 0xffffffff80553e47 at ng_add_hook+0xb7
#7 0xffffffff8055598f at ng_apply_item+0x3bf
#8 0xffffffff80554934 at ng_snd_item+0x3b4
#9 0xffffffff8056400f at ngc_send+0x1df
#10 0xffffffff804e71c6 at sosend_generic+0x3f6
#11 0xffffffff804eaaa3 at kern_sendit+0x1a3
#12 0xffffffff804ead5c at sendit+0xdc
#13 0xffffffff804eae4d at sys_sendto+0x4d
#14 0xffffffff806fb3da at amd64_syscall+0x5ea
#15 0xffffffff806e5b57 at Xfast_syscall+0xf7
Uptime: 3h42m8s
Dumping 118 out of 1011 MB:..14%..27%..41%..54%..68%..81%..95%

#0  doadump (textdump=<value optimized out>) at pcpu.h:234
234     pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) #0  doadump (textdump=<value optimized out>) at pcpu.h:234
#1  0xffffffff804766c6 in kern_reboot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:449
#2  0xffffffff80476bc7 in panic (fmt=0x1 <Address 0x1 out of bounds>)
    at /usr/src/sys/kern/kern_shutdown.c:637
#3  0xffffffff806fbc30 in trap_fatal (frame=0xc, eva=<value optimized out>)
    at /usr/src/sys/amd64/amd64/trap.c:879
#4  0xffffffff806fbf91 in trap_pfault (frame=0xffffff80003c7610, usermode=0)
    at /usr/src/sys/amd64/amd64/trap.c:795
#5  0xffffffff806fc544 in trap (frame=0xffffff80003c7610)
    at /usr/src/sys/amd64/amd64/trap.c:463
#6  0xffffffff806e5873 in calltrap ()
    at /usr/src/sys/amd64/amd64/exception.S:232
#7  0xffffffff80556f86 in ng_bpf_newhook (node=<value optimized out>, 
    hook=0xfffffe0001b9c380, name=0xfffffe0011e71258 "0-0-m")
    at /usr/src/sys/netgraph/ng_bpf.c:249
#8  0xffffffff80553e47 in ng_add_hook (node=0xfffffe0026021100, 
    name=0xfffffe0011e71258 "0-0-m", hookp=0xffffff80003c7808)
    at /usr/src/sys/netgraph/ng_base.c:1092
#9  0xffffffff8055598f in ng_apply_item (node=0xfffffe0026021100, 
    item=0xfffffe0011d90980, rw=1) at /usr/src/sys/netgraph/ng_base.c:1544
#10 0xffffffff80554934 in ng_snd_item (item=<value optimized out>, flags=0)
    at /usr/src/sys/netgraph/ng_base.c:2314
#11 0xffffffff8056400f in ngc_send (so=<value optimized out>, 
    flags=<value optimized out>, m=0xfffffe0011c36000, 
    addr=<value optimized out>, control=<value optimized out>, 
    td=<value optimized out>) at /usr/src/sys/netgraph/ng_socket.c:317
#12 0xffffffff804e71c6 in sosend_generic (so=0xfffffe0001982000, 
    addr=0xfffffe002602fa20, uio=0xffffff80003c7a00, top=0xfffffe0011c36000, 
    control=0x0, flags=<value optimized out>, td=0xfffffe00018ba000)
    at /usr/src/sys/kern/uipc_socket.c:1367
#13 0xffffffff804eaaa3 in kern_sendit (td=0xfffffe00018ba000, s=5, 
    mp=0xffffff80003c7ad0, flags=0, control=0x0, segflg=<value optimized out>)
    at /usr/src/sys/kern/uipc_syscalls.c:811
#14 0xffffffff804ead5c in sendit (td=0xfffffe00018ba000, s=5, 
    mp=0xffffff80003c7ad0, flags=0) at /usr/src/sys/kern/uipc_syscalls.c:739
#15 0xffffffff804eae4d in sys_sendto (td=<value optimized out>, 
    uap=<value optimized out>) at /usr/src/sys/kern/uipc_syscalls.c:863
#16 0xffffffff806fb3da in amd64_syscall (td=0xfffffe00018ba000, traced=0)
    at subr_syscall.c:135
#17 0xffffffff806e5b57 in Xfast_syscall ()
    at /usr/src/sys/amd64/amd64/exception.S:391
#18 0x000000080225adcc in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb)
     

    • roysbike

      roysbike - 2013-08-29

      Show

      cat /boot/loader.conf
      cat /etc/sysctl.conf

       

  • fjcst

    fjcst - 2013-08-31

    I didn't change anything, everything is default.

     

  • Dmitriy Zamuraev

    Dmitriy Zamuraev - 2013-09-02

    try to use another hardware.

    I have two identical servers, on one physical server freebsd crashes one time per week, on another server freebsd works perfectly more than year.
    other guest OS (like linux-based, windows-based) works perfectly on every physical server.

     

  • Dmitriy Zamuraev

    Dmitriy Zamuraev - 2013-09-02

    p.s.

    I have in production use FreeBSD 7.2 PPPoE servers (now in qemu-kvm environment with VT-d technology using external PCI-E physical network cards like EXPI9402), no craches anywhere!

     

  • tolich

    tolich - 2015-05-07

    Здравствуйте. Поднимаю тему. Две машины FreeBSD 10.1-STABLE mpd 5.7 и 5.8 регулярно зависают 

    panic: general protection fault

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
795 (mpd5)
trap number = 9
panic: general protection fault
cpuid = 1
KDB: stack backtrace:
#0 0xffffffff8096c510 at kdb_backtrace+0x60
#1 0xffffffff80930855 at panic+0x155
#2 0xffffffff80d2f10f at trap_fatal+0x38f
#3 0xffffffff80d2ed6c at trap+0x75c
#4 0xffffffff80d14f12 at calltrap+0x8
#5 0xffffffff81a4c088 at ng_rmhook_self+0x58
#6 0xffffffff81a4b847 at ng_destroy_hook+0x97
#7 0xffffffff81a4db6f at ng_apply_item+0xcbf
#8 0xffffffff81a4cd1a at ng_snd_item+0x38a
#9 0xffffffff81a479f1 at ngc_send+0x221
#10 0xffffffff809a72d5 at sosend_generic+0x475
#11 0xffffffff809ad635 at kern_sendit+0x245
#12 0xffffffff809ad959 at sendit+0x129
#13 0xffffffff809ad81d at sys_sendto+0x4d
#14 0xffffffff80d2fa27 at amd64_syscall+0x357
#15 0xffffffff80d151fb at Xfast_syscall+0xfb
Uptime: 10d9h51m0s
Dumping 534 out of 2983 MB:..3%..12%..21%..33%..42%..51%..63%..72%..81%..93%

Reading symbols from /boot/kernel/pflog.ko.symbols...done.
Loaded symbols for /boot/kernel/pflog.ko.symbols
Reading symbols from /boot/kernel/pf.ko.symbols...done.
Loaded symbols for /boot/kernel/pf.ko.symbols
Reading symbols from /boot/kernel/ng_socket.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_socket.ko.symbols
Reading symbols from /boot/kernel/netgraph.ko.symbols...done.
Loaded symbols for /boot/kernel/netgraph.ko.symbols
Reading symbols from /boot/kernel/ng_mppc.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_mppc.ko.symbols
Reading symbols from /boot/kernel/rc4.ko.symbols...done.
Loaded symbols for /boot/kernel/rc4.ko.symbols
Reading symbols from /boot/kernel/ng_ether.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_ether.ko.symbols
Reading symbols from /boot/kernel/ng_pppoe.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_pppoe.ko.symbols
Reading symbols from /boot/kernel/ng_tee.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_tee.ko.symbols
Reading symbols from /boot/kernel/ng_iface.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_iface.ko.symbols
Reading symbols from /boot/kernel/ng_ppp.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_ppp.ko.symbols
Reading symbols from /boot/kernel/ng_tcpmss.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_tcpmss.ko.symbols
Reading symbols from /boot/kernel/ng_bpf.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_bpf.ko.symbols
Reading symbols from /boot/kernel/ng_car.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_car.ko.symbols
Reading symbols from /boot/kernel/ng_vjc.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_vjc.ko.symbols
#0  doadump (textdump=<value optimized out>) at pcpu.h:219
219 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) #0  doadump (textdump=<value optimized out>) at pcpu.h:219
#1  0xffffffff809304d2 in kern_reboot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:452
#2  0xffffffff80930894 in panic (fmt=<value optimized out>)
    at /usr/src/sys/kern/kern_shutdown.c:759
#3  0xffffffff80d2f10f in trap_fatal (frame=<value optimized out>, 
    eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:859
#4  0xffffffff80d2ed6c in trap (frame=<value optimized out>)
    at /usr/src/sys/amd64/amd64/trap.c:203
#5  0xffffffff80d14f12 in calltrap ()
    at /usr/src/sys/amd64/amd64/exception.S:236
#6  0xffffffff80d14124 in atomic_add_barr_int (p=0xe9dfb4e00034493, v=0)
    at atomic.h:310
#7  0xffffffff81a4c088 in ng_rmhook_self (hook=0xfffff800281d5700)
    at refcount.h:53
#8  0xffffffff81a4b847 in ng_destroy_hook (hook=0xfffff8003ac61280)
    at /usr/src/sys/modules/netgraph/netgraph/../../../netgraph/ng_base.c:1188
#9  0xffffffff81a4db6f in ng_apply_item (node=0xfffff8005ba5de00, 
    item=0xfffff8005bb49e00, rw=1)
    at /usr/src/sys/modules/netgraph/netgraph/../../../netgraph/ng_base.c:2582
#10 0xffffffff81a4cd1a in ng_snd_item (item=0xfffff8005bb49e00, 
    flags=<value optimized out>)
    at /usr/src/sys/modules/netgraph/netgraph/../../../netgraph/ng_base.c:2314
#11 0xffffffff81a479f1 in ngc_send (so=<value optimized out>, 
    flags=<value optimized out>, m=0xfffff80059582a00, 
    addr=<value optimized out>, control=<value optimized out>, td=0x0)
    at /usr/src/sys/modules/netgraph/socket/../../../netgraph/ng_socket.c:338
#12 0xffffffff809a72d5 in sosend_generic (so=0xfffff800105af2b8, 
    addr=0xfffff800103a66a0, uio=0xfffffe00d96f39d0, 
    top=<value optimized out>, control=<value optimized out>, 
    flags=<value optimized out>, td=0x3000000010)
    at /usr/src/sys/kern/uipc_socket.c:1283
#13 0xffffffff809ad635 in kern_sendit (td=0xfffff8001042d000, s=24, 
    mp=0xfffffe00d96f3aa0, flags=0, control=0x0, segflg=<value optimized out>)
    at /usr/src/sys/kern/uipc_syscalls.c:944
#14 0xffffffff809ad959 in sendit (td=0xfffff8001042d000, 
    s=<value optimized out>, mp=0xfffffe00d96f3aa0, flags=0)
    at /usr/src/sys/kern/uipc_syscalls.c:871
#15 0xffffffff809ad81d in sys_sendto (td=0xe9dfb4e00034493, 
    uap=<value optimized out>) at /usr/src/sys/kern/uipc_syscalls.c:995
#16 0xffffffff80d2fa27 in amd64_syscall (td=0xfffff8001042d000, traced=0)
    at subr_syscall.c:134
#17 0xffffffff80d151fb in Xfast_syscall ()
    at /usr/src/sys/amd64/amd64/exception.S:396
#18 0x000000080230a08a in ?? ()
Previous frame inner to this frame (corrupt stack?)
Current language:  auto; currently minimal
(kgdb)
     

Log in to post a comment.

MongoDB Logo MongoDB