Menu ▾ ▴

#271 jfsCommit using 100% CPU

bug
open
David Kleikamp
utilities (15)
5
2014-10-15
2009-03-10
rwk
No

When creating a very large file (using amanda backup) jfsCommit CPU usage reaches a continuous 100% until the amanda backup is terminated. The file growth slows to a crawl. This is consistently reproduceable and has happened each of tree times I have tried it.

I am running:

. Linux 2.6.28-gentoo-r2 #1 SMP Thu Mar 5 07:41:56 MST 2009 i686 Intel(R) Pentium(R) III CPU
. jfsutils-1.1.13
. on a brand new JFS filesystem build.
. filesystem is 13TB
. file produced is 800GB at the tile jfsCommit reaches 100%.

I am not sure at what file size the problem begins, but jfsCommit starts out a 0% - 2% at the beginning. Running "jfs_fsck -v -n" on the mounted filesystem with jfsCommit at 100% follows. This is only the first 100 lines, as it repeats thousands of times until I kill the fsck:

jfs_fsck version 1.1.13, 17-Jul-2008
processing started: 3/10/2009 1.6.50
Filesystem is currently mounted.
WARNING: Checking a mounted filesystem does not produce dependable results.
The current device is: /dev/evms/space
Open(...READONLY...) returned rc = 0
Primary superblock is valid.
The type of file system for the device is JFS.
Block size in bytes: 4096
Filesystem size in blocks: 3303833599
**Phase 1 - Check Blocks, Files/Directories, and Directory Entries
File system object FF1876225 has corrupt data (10).
**Phase 2 - Count links
**Phase 3 - Duplicate Block Rescan and Directory Connectedness
**Phase 4 - Report Problems
File system object FF1876225 is linked as: /dumps/hold/20090309062823/utility.krautclan.com._store_media.0.tmp
cannot repair the data format error(s) in this file.
cannot repair FF1876225.
**Phase 5 - Check Connectivity
**Phase 6 - Perform Approved Corrections
**Phase 7 - Verify File/Directory Allocation Maps
Discrepancies detected in the Free IAG List. (F,3)
Discrepancies detected in the Free Inode Extent List for Allocation Group F464. (2)
Discrepancies detected in the Free Inode List for Allocation Group F464. (3)
Inode Allocation Group F464 is inconsistent.
The Inode Allocation Map has an incorrect number of backed inodes value (F).
The Inode Allocation Map has an incorrect number of free inodes value (F).
The Inode Allocation Map control information has an incorrect number of free inodes value for AG F10.
The Inode Allocation Map control information has an incorrect number of free inodes value for AG F19.
The Inode Allocation Map control information has an incorrect number of free inodes value for AG F20.
The Inode Allocation Map control information has an incorrect number of free inodes value for AG F21.
The Inode Allocation Map control information has an incorrect number of free inodes value for AG F23.
The Inode Allocation Map control information has an incorrect number of free inodes value for AG F24.
The Inode Allocation Map control information has an incorrect number of backed inodes value for AG F25.
The Inode Allocation Map control information has an incorrect number of free inodes value for AG F25.
Errors detected in the Fileset File/Directory Allocation Map control information. (F)
Errors detected in the Fileset File/Directory Allocation Map. (F)
**Phase 8 - Verify Disk Allocation Maps
2044 consecutive blocks observed available but pmap (77829, 192, 4) indicates they are allocated.
Incorrect number free detected in dmap 77829.
Incorrect internal (0) value detected in DM page 77829.
Incorrect internal (4) value detected in DM page 77829.
Incorrect internal (17) value detected in DM page 77829.
Incorrect internal (18) value detected in DM page 77829.
Incorrect internal (19) value detected in DM page 77829.
Incorrect internal (69) value detected in DM page 77829.
Incorrect internal (70) value detected in DM page 77829.
Incorrect internal (71) value detected in DM page 77829.
Incorrect internal (73) value detected in DM page 77829.
Incorrect internal (77) value detected in DM page 77829.
Incorrect leaf (277) value detected in DM page 77829.
Incorrect leaf (278) value detected in DM page 77829.
Incorrect leaf (279) value detected in DM page 77829.
Incorrect leaf (281) value detected in DM page 77829.
Incorrect leaf (285) value detected in DM page 77829.
Incorrect leaf (293) value detected in DM page 77829.
Incorrect leaf (309) value detected in DM page 77829.
8192 consecutive blocks observed available but pmap (77830, 0, 0) indicates they are allocated.
Incorrect number free detected in dmap 77830.
Incorrect internal (0) value detected in DM page 77830.
Incorrect internal (1) value detected in DM page 77830.
Incorrect internal (5) value detected in DM page 77830.
Incorrect internal (21) value detected in DM page 77830.
Incorrect leaf (85) value detected in DM page 77830.
8192 consecutive blocks observed available but pmap (77831, 0, 0) indicates they are allocated.
Incorrect number free detected in dmap 77831.
Incorrect internal (0) value detected in DM page 77831.
Incorrect internal (1) value detected in DM page 77831.
Incorrect internal (5) value detected in DM page 77831.
Incorrect internal (21) value detected in DM page 77831.
Incorrect leaf (85) value detected in DM page 77831.
8192 consecutive blocks observed available but pmap (77832, 0, 0) indicates they are allocated.
Incorrect number free detected in dmap 77832.
Incorrect internal (0) value detected in DM page 77832.
Incorrect internal (1) value detected in DM page 77832.
Incorrect internal (5) value detected in DM page 77832.
Incorrect internal (21) value detected in DM page 77832.
Incorrect leaf (85) value detected in DM page 77832.
8192 consecutive blocks observed available but pmap (77833, 0, 0) indicates they are allocated.
Incorrect number free detected in dmap 77833.
Incorrect internal (0) value detected in DM page 77833.
Incorrect internal (1) value detected in DM page 77833.
Incorrect internal (5) value detected in DM page 77833.
Incorrect internal (21) value detected in DM page 77833.
Incorrect leaf (85) value detected in DM page 77833.
8192 consecutive blocks observed available but pmap (77834, 0, 0) indicates they are allocated.
Incorrect number free detected in dmap 77834.
Incorrect internal (0) value detected in DM page 77834.
Incorrect internal (1) value detected in DM page 77834.
Incorrect internal (5) value detected in DM page 77834.
Incorrect internal (21) value detected in DM page 77834.
Incorrect leaf (85) value detected in DM page 77834.
8192 consecutive blocks observed available but pmap (77835, 0, 0) indicates they are allocated.
Incorrect number free detected in dmap 77835.
Incorrect internal (0) value detected in DM page 77835.
Incorrect internal (1) value detected in DM page 77835.
Incorrect internal (5) value detected in DM page 77835.
Incorrect internal (21) value detected in DM page 77835.
Incorrect leaf (85) value detected in DM page 77835.
8192 consecutive blocks observed available but pmap (77836, 0, 0) indicates they are allocated.

Discussion

  • rwk

    rwk - 2009-03-10

    That should be:

    . file produced is 800GB at the time jfsCommit reaches 100%.

    not:

    . file produced is 800GB at the tile jfsCommit reaches 100%.

    Is there a way to edit submissions?

     

  • Nobody/Anonymous

    Nobody/Anonymous - 2009-03-10

    Results of running fsck.jfs after killing amanda and unmounting the filesystem. Amanda was not getting anywhere.

    # jfs_fsck -v -f -n /dev/evms/space
    fsck.jfs version 1.1.13, 17-Jul-2008
    processing started: 3/10/2009 7.18.48
    The current device is: /dev/evms/space
    Open(...READ/WRITE EXCLUSIVE...) returned rc = 0
    Primary superblock is valid.
    The type of file system for the device is JFS.
    Block size in bytes: 4096
    Filesystem size in blocks: 3303833599
    **Phase 0 - Replay Journal Log
    LOGREDO: Log already redone!
    logredo returned rc = 0
    **Phase 1 - Check Blocks, Files/Directories, and Directory Entries
    **Phase 2 - Count links
    **Phase 3 - Duplicate Block Rescan and Directory Connectedness
    **Phase 4 - Report Problems
    **Phase 5 - Check Connectivity
    **Phase 6 - Perform Approved Corrections
    **Phase 7 - Verify File/Directory Allocation Maps
    **Phase 8 - Verify Disk Allocation Maps
    Incorrect maximum active AGs detected in Block Map Control Page.
    Incorrect data detected in the Block Map Control Page.
    Incorrect data detected in disk allocation control structures.
    Filesystem Summary:
    Blocks in use for inodes: 207004
    Inode count: 1656032
    File count: 1506612
    Directory count: 99745
    Block count: 3303833599
    Free block count: 3089529890
    13215334396 kilobytes total disk space.
    410976 kilobytes in 99745 directories.
    854726058 kilobytes in 1506612 user files.
    0 kilobytes in extended attributes
    0 kilobytes in access control lists
    2899754 kilobytes reserved for system use.
    12358119560 kilobytes are available for use.
    File system checked READ ONLY.
    Filesystem is dirty.
    Filesystem is dirty but is marked clean. In its present state,
    the results of accessing /dev/evms/space (except by this utility) are undefined.
    processing terminated: 3/10/2009 7:27:24 with return code: 0 exit code: 4.

     

  • Nobody/Anonymous

    Nobody/Anonymous - 2009-03-10

    Without "n" option:

    # fsck -v -f /dev/evms/space
    fsck 1.39 (29-May-2006)
    fsck.jfs version 1.1.13, 17-Jul-2008
    processing started: 3/10/2009 7.32.33
    The current device is: /dev/evms/space
    Open(...READ/WRITE EXCLUSIVE...) returned rc = 0
    Primary superblock is valid.
    The type of file system for the device is JFS.
    Block size in bytes: 4096
    Filesystem size in blocks: 3303833599
    **Phase 0 - Replay Journal Log
    LOGREDO: Log already redone!
    logredo returned rc = 0
    **Phase 1 - Check Blocks, Files/Directories, and Directory Entries
    **Phase 2 - Count links
    **Phase 3 - Duplicate Block Rescan and Directory Connectedness
    **Phase 4 - Report Problems
    **Phase 5 - Check Connectivity
    **Phase 6 - Perform Approved Corrections
    **Phase 7 - Rebuild File/Directory Allocation Maps
    **Phase 8 - Rebuild Disk Allocation Maps
    Filesystem Summary:
    Blocks in use for inodes: 207004
    Inode count: 1656032
    File count: 1506612
    Directory count: 99745
    Block count: 3303833599
    Free block count: 3089530018
    13215334396 kilobytes total disk space.
    410976 kilobytes in 99745 directories.
    854726058 kilobytes in 1506612 user files.
    0 kilobytes in extended attributes
    0 kilobytes in access control lists
    2899242 kilobytes reserved for system use.
    12358120072 kilobytes are available for use.
    Filesystem is clean.
    All observed inconsistencies have been repaired.
    Filesystem has been marked clean.
    **** Filesystem was modified. ****
    processing terminated: 3/10/2009 7:39:51 with return code: 0 exit code: 0.

     

  • David Kleikamp

    David Kleikamp - 2009-03-10

    Can you "echo t > /proc/sysrq-trigger" when jfsCommit is chewing up the cpu, then send me the dmesg output (at least the part relevant to jfsCommit)?

    (Note: the 2.6.29 kernel will have /proc/<pid>/stack. That will be nice.)

    The problem found by fsck.jfs (after unmounting) is interesting, but it doesn't look like it would be related to the jfsCommit problem.

    ... and I don't know of a way to edit submissions

     

  • David Kleikamp

    David Kleikamp - 2009-03-10
    • assigned_to: nobody --> shaggyk
     

  • Nobody/Anonymous

    Nobody/Anonymous - 2009-03-11

    Here is the complete output from dmesg after running "echo t > /proc/sysrq-trigger".
    I include the whole thing in case there are any details about my system which are helpful.
    jfsCommit *was/is* running at 100%.

    Thanks!

    BIOS EBDA/lowmem at: 0009f400/0009f400
    Linux version 2.6.28-gentoo-r2 (root@dumphost) (gcc version 4.1.1 (Gentoo 4.1.1-r3)) #1 SMP Thu Mar 5 07:41:56 MST 2009
    KERNEL supported cpus:
    Intel GenuineIntel
    AMD AuthenticAMD
    Cyrix CyrixInstead
    NSC Geode by NSC
    Centaur CentaurHauls
    Transmeta GenuineTMx86
    Transmeta TransmetaCPU
    UMC UMC UMC UMC
    BIOS-provided physical RAM map:
    BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
    BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
    BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
    BIOS-e820: 0000000000100000 - 000000007fff0000 (usable)
    BIOS-e820: 000000007fff0000 - 000000007fff3000 (ACPI NVS)
    BIOS-e820: 000000007fff3000 - 0000000080000000 (ACPI data)
    BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
    DMI 2.3 present.
    last_pfn = 0x7fff0 max_arch_pfn = 0x100000
    kernel direct mapping tables up to 377fe000 @ 7000-d000
    RAMDISK: 37cef000 - 37fefaee
    Allocated new RAMDISK: 00731000 - 00a31aee
    Move RAMDISK from 0000000037cef000 - 0000000037fefaed to 00731000 - 00a31aed
    ACPI: RSDP 000F6BF0, 0014 (r0 VIA694)
    ACPI: RSDT 7FFF3000, 002C (r1 VIA694 AWRDACPI 42302E31 AWRD 0)
    ACPI: FACP 7FFF3040, 0074 (r1 VIA694 AWRDACPI 42302E31 AWRD 0)
    ACPI: DSDT 7FFF30C0, 2C69 (r1 VIA694 AWRDACPI 1000 MSFT 100000C)
    ACPI: FACS 7FFF0000, 0040
    ACPI: APIC 7FFF5D40, 005C (r1 VIA694 AWRDACPI 42302E31 AWRD 0)
    ACPI: Local APIC address 0xfee00000
    1159MB HIGHMEM available.
    887MB LOWMEM available.
    mapped low ram: 0 - 377fe000
    low ram: 00000000 - 377fe000
    bootmap 00009000 - 0000ff00
    (9 early reservations) ==> bootmem [0000000000 - 00377fe000]
    #0 [0000000000 - 0000001000] BIOS data page ==> [0000000000 - 0000001000]
    #1 [0000001000 - 0000002000] EX TRAMPOLINE ==> [0000001000 - 0000002000]
    #2 [0000006000 - 0000007000] TRAMPOLINE ==> [0000006000 - 0000007000]
    #3 [0000100000 - 000072d260] TEXT DATA BSS ==> [0000100000 - 000072d260]
    #4 [000072e000 - 0000731000] INIT_PG_TABLE ==> [000072e000 - 0000731000]
    #5 [000009f400 - 0000100000] BIOS reserved ==> [000009f400 - 0000100000]
    #6 [0000007000 - 0000009000] PGTABLE ==> [0000007000 - 0000009000]
    #7 [0000731000 - 0000a31aee] NEW RAMDISK ==> [0000731000 - 0000a31aee]
    #8 [0000009000 - 0000010000] BOOTMAP ==> [0000009000 - 0000010000]
    found SMP MP-table at [c00f5260] 000f5260
    Zone PFN ranges:
    DMA 0x00000000 -> 0x00001000
    Normal 0x00001000 -> 0x000377fe
    HighMem 0x000377fe -> 0x0007fff0
    Movable zone start PFN for each node
    early_node_map[2] active PFN ranges
    0: 0x00000000 -> 0x0000009f
    0: 0x00000100 -> 0x0007fff0
    On node 0 totalpages: 524175
    free_area_init_node: node 0, pgdat c0638980, node_mem_map c1000000
    DMA zone: 32 pages used for memmap
    DMA zone: 0 pages reserved
    DMA zone: 3967 pages, LIFO batch:0
    Normal zone: 1744 pages used for memmap
    Normal zone: 221486 pages, LIFO batch:31
    HighMem zone: 2320 pages used for memmap
    HighMem zone: 294626 pages, LIFO batch:31
    Movable zone: 0 pages used for memmap
    ACPI: PM-Timer IO Port: 0x4008
    ACPI: Local APIC address 0xfee00000
    ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
    ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
    ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
    IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
    ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
    ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 dfl dfl)
    ACPI: IRQ0 used by override.
    ACPI: IRQ2 used by override.
    ACPI: IRQ9 used by override.
    Enabling APIC mode: Flat. Using 1 I/O APICs
    Using ACPI (MADT) for SMP configuration information
    SMP: Allowing 2 CPUs, 0 hotplug CPUs
    Allocating PCI resources starting at 88000000 (gap: 80000000:7ec00000)
    PERCPU: Allocating 32768 bytes of per cpu data
    NR_CPUS: 8, nr_cpu_ids: 2, nr_node_ids 1
    Built 1 zonelists in Zone order, mobility grouping on. Total pages: 520079
    Kernel command line: root=/dev/evms/hda2 ramdisk=8192
    Enabling fast FPU save and restore... done.
    Enabling unmasked SIMD FPU exception support... done.
    Initializing CPU#0
    PID hash table entries: 4096 (order: 12, 16384 bytes)
    Fast TSC calibration using PIT
    Detected 1132.695 MHz processor.
    Console: colour VGA+ 80x25
    console [tty0] enabled
    Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
    Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
    Memory: 2068984k/2097088k available (3495k kernel code, 26860k reserved, 1975k data, 324k init, 1187784k highmem)
    virtual kernel memory layout:
    fixmap : 0xfff51000 - 0xfffff000 ( 696 kB)
    pkmap : 0xff800000 - 0xffc00000 (4096 kB)
    vmalloc : 0xf7ffe000 - 0xff7fe000 ( 120 MB)
    lowmem : 0xc0000000 - 0xf77fe000 ( 887 MB)
    .init : 0xc065f000 - 0xc06b0000 ( 324 kB)
    .data : 0xc0469de0 - 0xc0657a98 (1975 kB)
    .text : 0xc0100000 - 0xc0469de0 (3495 kB)
    Checking if this processor honours the WP bit even in supervisor mode...Ok.
    Calibrating delay loop (skipped), value calculated using timer frequency.. 2265.39 BogoMIPS (lpj=4530780)
    Mount-cache hash table entries: 512
    CPU: L1 I cache: 16K, L1 D cache: 16K
    CPU: L2 cache: 512K
    Intel machine check architecture supported.
    Intel machine check reporting enabled on CPU#0.
    Checking 'hlt' instruction... OK.
    ACPI: Core revision 20080926
    ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
    CPU0: Intel(R) Pentium(R) III CPU family 1133MHz stepping 01
    Booting processor 1 APIC 0x1 ip 0x6000
    Initializing CPU#1
    Calibrating delay using timer specific routine.. 2265.75 BogoMIPS (lpj=4531506)
    CPU: L1 I cache: 16K, L1 D cache: 16K
    CPU: L2 cache: 512K
    Intel machine check architecture supported.
    Intel machine check reporting enabled on CPU#1.
    CPU1: Intel(R) Pentium(R) III CPU family 1133MHz stepping 01
    checking TSC synchronization [CPU#0 -> CPU#1]: passed.
    Brought up 2 CPUs
    Total of 2 processors activated (4531.14 BogoMIPS).
    CPU0 attaching sched-domain:
    domain 0: span 0-1 level CPU
    groups: 0 1
    CPU1 attaching sched-domain:
    domain 0: span 0-1 level CPU
    groups: 1 0
    net_namespace: 492 bytes
    NET: Registered protocol family 16
    ACPI: bus type pci registered
    PCI: PCI BIOS revision 2.10 entry at 0xfb640, last bus=1
    PCI: Using configuration type 1 for base access
    mtrr: your CPUs had inconsistent fixed MTRR settings
    mtrr: your CPUs had inconsistent variable MTRR settings
    mtrr: probably your BIOS does not setup all CPUs.
    mtrr: corrected configuration.
    ACPI: EC: Look up EC in DSDT
    ACPI: Interpreter enabled
    ACPI: (supports S0 S1 S5)
    ACPI: Using IOAPIC for interrupt routing
    ACPI: No dock devices found.
    ACPI: PCI Root Bridge [PCI0] (0000:00)
    pci 0000:00:00.0: reg 10 32bit mmio: [0xf0000000-0xf3ffffff]
    pci 0000:00:01.0: supports D1
    pci 0000:00:07.0: reg 10 io port: [0xc000-0xc00f]
    pci 0000:00:07.0: reg 14 32bit mmio: [0xf83c1000-0xf83c100f]
    pci 0000:00:07.0: reg 18 32bit mmio: [0xf7000000-0xf77fffff]
    pci 0000:00:07.0: reg 30 32bit mmio: [0x000000-0x00ffff]
    pci 0000:00:07.0: supports D1
    pci 0000:00:08.0: reg 10 64bit mmio: [0xf8100000-0xf81fffff]
    pci 0000:00:08.0: reg 18 io port: [0xc400-0xc4ff]
    pci 0000:00:0a.0: reg 10 64bit mmio: [0xf8000000-0xf80fffff]
    pci 0000:00:0a.0: reg 18 io port: [0xc800-0xc8ff]
    pci 0000:00:0b.0: reg 10 64bit mmio: [0xf8380000-0xf839ffff]
    pci 0000:00:0b.0: reg 18 64bit mmio: [0xf8340000-0xf837ffff]
    pci 0000:00:0b.0: reg 20 io port: [0xcc00-0xcc3f]
    pci 0000:00:0b.0: reg 30 32bit mmio: [0x000000-0x03ffff]
    pci 0000:00:0b.0: PME# supported from D0 D3hot D3cold
    pci 0000:00:0b.0: PME# disabled
    pci 0000:00:0b.1: reg 10 64bit mmio: [0xf83a0000-0xf83bffff]
    pci 0000:00:0b.1: reg 18 64bit mmio: [0xf8300000-0xf833ffff]
    pci 0000:00:0b.1: reg 20 io port: [0xd000-0xd03f]
    pci 0000:00:0b.1: reg 30 32bit mmio: [0x000000-0x03ffff]
    pci 0000:00:0b.1: PME# supported from D0 D3hot D3cold
    pci 0000:00:0b.1: PME# disabled
    pci 0000:00:0d.0: reg 10 32bit mmio: [0xf83c0000-0xf83c0fff]
    pci 0000:00:0d.0: reg 14 io port: [0xd400-0xd43f]
    pci 0000:00:0d.0: reg 18 32bit mmio: [0xf8200000-0xf82fffff]
    pci 0000:00:0d.0: reg 30 32bit mmio: [0x000000-0x0fffff]
    pci 0000:00:0d.0: supports D1 D2
    pci 0000:00:0d.0: PME# supported from D0 D1 D2 D3hot D3cold
    pci 0000:00:0d.0: PME# disabled
    pci 0000:00:11.1: reg 20 io port: [0xd800-0xd80f]
    pci 0000:00:11.2: reg 20 io port: [0xdc00-0xdc1f]
    pci 0000:00:11.3: reg 20 io port: [0xe000-0xe01f]
    pci 0000:00:11.4: reg 20 io port: [0xe400-0xe41f]
    pci 0000:01:00.0: reg 10 32bit mmio: [0xf4000000-0xf4ffffff]
    pci 0000:01:00.0: reg 14 32bit mmio: [0xe8000000-0xefffffff]
    pci 0000:01:00.0: reg 30 32bit mmio: [0x000000-0x00ffff]
    pci 0000:00:01.0: bridge 32bit mmio: [0xf4000000-0xf5ffffff]
    pci 0000:00:01.0: bridge 32bit mmio pref: [0xe8000000-0xefffffff]
    bus 00 -> node 0
    ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
    ACPI: PCI Interrupt Link [LNKA] (IRQs 1 3 4 5 6 7 10 *11 12 14 15)
    ACPI: PCI Interrupt Link [LNKB] (IRQs 1 3 4 *5 6 7 10 11 12 14 15)
    ACPI: PCI Interrupt Link [LNKC] (IRQs 1 3 4 5 6 7 *10 11 12 14 15)
    ACPI: PCI Interrupt Link [LNKD] (IRQs 1 3 4 5 6 7 10 *11 12 14 15)
    SCSI subsystem initialized
    libata version 3.00 loaded.
    usbcore: registered new interface driver usbfs
    usbcore: registered new interface driver hub
    usbcore: registered new device driver usb
    PCI: Using ACPI for IRQ routing
    pnp: PnP ACPI init
    ACPI: bus type pnp registered
    pnp 00:00: mem resource (0xde800-0xdffff) overlaps 0000:00:0d.0 BAR 6 (0x0-0xfffff), disabling
    pnp 00:00: mem resource (0xf0000-0xf7fff) overlaps 0000:00:0d.0 BAR 6 (0x0-0xfffff), disabling
    pnp 00:00: mem resource (0xf8000-0xfbfff) overlaps 0000:00:0d.0 BAR 6 (0x0-0xfffff), disabling
    pnp 00:00: mem resource (0xfc000-0xfffff) overlaps 0000:00:0d.0 BAR 6 (0x0-0xfffff), disabling
    pnp 00:00: mem resource (0x0-0x9ffff) overlaps 0000:00:0d.0 BAR 6 (0x0-0xfffff), disabling
    pnp: PnP ACPI: found 12 devices
    ACPI: ACPI bus type pnp unregistered
    system 00:00: iomem range 0x7fff0000-0x7fffffff could not be reserved
    system 00:00: iomem range 0xffff0000-0xffffffff has been reserved
    system 00:00: iomem range 0x100000-0x7ffeffff could not be reserved
    system 00:00: iomem range 0xfec00000-0xfec00fff has been reserved
    system 00:00: iomem range 0xfee00000-0xfee00fff has been reserved
    system 00:00: iomem range 0xfff80000-0xfffeffff has been reserved
    system 00:02: ioport range 0x4d0-0x4d1 has been reserved
    system 00:02: ioport range 0x294-0x297 has been reserved
    pci 0000:00:01.0: PCI bridge, secondary bus 0000:01
    pci 0000:00:01.0: IO window: disabled
    pci 0000:00:01.0: MEM window: 0xf4000000-0xf5ffffff
    pci 0000:00:01.0: PREFETCH window: 0x000000e8000000-0x000000efffffff
    pci 0000:00:01.0: setting latency timer to 64
    bus: 00 index 0 io port: [0x00-0xffff]
    bus: 00 index 1 mmio: [0x000000-0xffffffff]
    bus: 01 index 0 mmio: [0x0-0x0]
    bus: 01 index 1 mmio: [0xf4000000-0xf5ffffff]
    bus: 01 index 2 mmio: [0xe8000000-0xefffffff]
    bus: 01 index 3 mmio: [0x0-0x0]
    NET: Registered protocol family 2
    IP route cache hash table entries: 32768 (order: 5, 131072 bytes)
    TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
    TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
    TCP: Hash tables configured (established 131072 bind 65536)
    TCP reno registered
    NET: Registered protocol family 1
    checking if image is initramfs...it isn't (no cpio magic); looks like an initrd
    Freeing initrd memory: 3074k freed
    Machine check exception polling timer started.
    apm: BIOS version 1.2 Flags 0x07 (Driver version 1.16ac)
    apm: disabled - APM is not SMP safe.
    Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
    alg: cipher: Test 1 failed on encryption for aes-asm
    00000000: 00 01 02 03 04 05 06 07 08 08 08 08 08 08 08 08
    highmem bounce pool size: 64 pages
    HugeTLB registered 4 MB page size, pre-allocated 0 pages
    Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
    JFS: nTxBlock = 8192, nTxLock = 65536
    msgmni has been set to 1728
    alg: No test for stdrng (krng)
    io scheduler noop registered
    io scheduler anticipatory registered (default)
    io scheduler deadline registered
    io scheduler cfq registered
    PCI: VIA PCI bridge detected.Disabling DAC.
    pci 0000:00:0d.0: Firmware left e100 interrupts enabled; disabling
    pci 0000:01:00.0: Boot video device
    Real Time Clock Driver v1.12ac
    Linux agpgart interface v0.103
    agpgart: Detected VIA Pro 266 chipset
    agpgart-via 0000:00:00.0: AGP aperture is 64M @ 0xf0000000
    Hangcheck: starting hangcheck timer 0.9.0 (tick is 180 seconds, margin is 60 seconds).
    Hangcheck: Using get_cycles().
    input: Power Button (FF) as /class/input/input0
    ACPI: Power Button (FF) [PWRF]
    input: Power Button (CM) as /class/input/input1
    ACPI: Power Button (CM) [PWRB]
    fan PNP0C0B:00: registered as cooling_device0
    ACPI: Fan [FAN] (on)
    processor ACPI_CPU:00: registered as cooling_device1
    processor ACPI_CPU:01: registered as cooling_device2
    thermal LNXTHERM:01: registered as thermal_zone0
    ACPI: Thermal Zone [THRM] (33 C)
    [drm] Initialized drm 1.1.0 20060810
    Serial: 8250/16550 driver4 ports, IRQ sharing disabled
    serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
    serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
    00:07: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
    00:08: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
    parport_pc 00:09: reported by Plug and Play ACPI
    parport0: PC-style at 0x378 (0x778), irq 7 [PCSPP(,...)]
    floppy0: no floppy controllers found
    brd: module loaded
    loop: module loaded
    Intel(R) PRO/1000 Network Driver - version 7.3.21-k3-NAPI
    Copyright (c) 1999-2006 Intel Corporation.
    e1000 0000:00:0b.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
    e1000 0000:00:0b.0: PCI: Disallowing DAC for device
    e1000: 0000:00:0b.0: e1000_probe: (PCI:33MHz:32-bit) 00:04:23:a6:54:20
    e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
    e1000 0000:00:0b.1: PCI INT B -> GSI 18 (level, low) -> IRQ 18
    e1000 0000:00:0b.1: PCI: Disallowing DAC for device
    e1000: 0000:00:0b.1: e1000_probe: (PCI:33MHz:32-bit) 00:04:23:a6:54:21
    e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
    Ethernet Channel Bonding Driver: v3.3.0 (June 10, 2008)
    bonding: Warning: either miimon or arp_interval and arp_ip_target module parameters must be specified, otherwise bonding will not detect link failures! see bonding.txt for details.
    e100: Intel(R) PRO/100 Network Driver, 3.5.23-k6-NAPI
    e100: Copyright(c) 1999-2006 Intel Corporation
    e100 0000:00:0d.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
    e100 0000:00:0d.0: PME# disabled
    e100: eth2: e100_probe: addr 0xf83c0000, irq 19, MAC addr 00:30:48:41:53:00
    sky2 driver version 1.22
    Uniform Multi-Platform E-IDE driver
    via82cxxx 0000:00:11.1: VIA vt8233 (rev 00) IDE UDMA100
    via82cxxx 0000:00:11.1: IDE controller (0x1106:0x0571 rev 0x06)
    via82cxxx 0000:00:11.1: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0xd800-0xd807
    ide1: BM-DMA at 0xd808-0xd80f
    Probing IDE interface ide0...
    hda: WDC WD5000AAKB-00UKA0, ATA DISK drive
    hdb: WDC WD5000AAKB-00UKA0, ATA DISK drive
    hda: host max PIO5 wanted PIO255(auto-tune) selected PIO4
    hda: UDMA/100 mode selected
    hdb: host max PIO5 wanted PIO255(auto-tune) selected PIO4
    hdb: UDMA/100 mode selected
    Probing IDE interface ide1...
    hdc: WDC WD5000AAKB-00UKA0, ATA DISK drive
    hdc: host max PIO5 wanted PIO255(auto-tune) selected PIO4
    hdc: UDMA/100 mode selected
    ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
    ide1 at 0x170-0x177,0x376 on irq 15
    ide_generic: please use "probe_mask=0x3f" module parameter for probing all legacy ISA IDE ports
    ide-gd driver 1.18
    hda: max request size: 512KiB
    hda: 976773168 sectors (500107 MB) w/16384KiB Cache, CHS=60801/255/63
    hda: cache flushes supported
    hda: hda1 hda2 hda3
    hdb: max request size: 512KiB
    hdb: 976773168 sectors (500107 MB) w/16384KiB Cache, CHS=60801/255/63
    hdb: cache flushes supported
    hdb: unknown partition table
    hdc: max request size: 512KiB
    hdc: 976773168 sectors (500107 MB) w/16384KiB Cache, CHS=60801/255/63
    hdc: cache flushes supported
    hdc: unknown partition table
    ide-cd driver 5.00
    3ware Storage Controller device driver for Linux v1.26.02.002.
    3w-xxxx 0000:00:07.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
    scsi0 : 3ware Storage Controller
    3w-xxxx: scsi0: Found a 3ware Storage Controller at 0xc000, IRQ: 17.
    scsi 0:0:0:0: Direct-Access 3ware Logical Disk 0 1.2 PQ: 0 ANSI: 0
    scsi 0:0:1:0: Direct-Access 3ware Logical Disk 1 1.2 PQ: 0 ANSI: 0
    scsi 0:0:2:0: Direct-Access 3ware Logical Disk 2 1.2 PQ: 0 ANSI: 0
    scsi 0:0:3:0: Direct-Access 3ware Logical Disk 3 1.2 PQ: 0 ANSI: 0
    scsi 0:0:4:0: Direct-Access 3ware Logical Disk 4 1.2 PQ: 0 ANSI: 0
    scsi 0:0:5:0: Direct-Access 3ware Logical Disk 5 1.2 PQ: 0 ANSI: 0
    scsi 0:0:6:0: Direct-Access 3ware Logical Disk 6 1.2 PQ: 0 ANSI: 0
    scsi 0:0:7:0: Direct-Access 3ware Logical Disk 7 1.2 PQ: 0 ANSI: 0
    Driver 'sd' needs updating - please use bus_type methods
    sd 0:0:0:0: [sda] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:0:0: [sda] Write Protect is off
    sd 0:0:0:0: [sda] Mode Sense: 00 00 00 00
    sd 0:0:0:0: [sda] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sd 0:0:0:0: [sda] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:0:0: [sda] Write Protect is off
    sd 0:0:0:0: [sda] Mode Sense: 00 00 00 00
    sd 0:0:0:0: [sda] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sda: unknown partition table
    sd 0:0:0:0: [sda] Attached SCSI disk
    sd 0:0:1:0: [sdb] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:1:0: [sdb] Write Protect is off
    sd 0:0:1:0: [sdb] Mode Sense: 00 00 00 00
    sd 0:0:1:0: [sdb] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sd 0:0:1:0: [sdb] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:1:0: [sdb] Write Protect is off
    sd 0:0:1:0: [sdb] Mode Sense: 00 00 00 00
    sd 0:0:1:0: [sdb] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sdb: unknown partition table
    sd 0:0:1:0: [sdb] Attached SCSI disk
    sd 0:0:2:0: [sdc] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:2:0: [sdc] Write Protect is off
    sd 0:0:2:0: [sdc] Mode Sense: 00 00 00 00
    sd 0:0:2:0: [sdc] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sd 0:0:2:0: [sdc] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:2:0: [sdc] Write Protect is off
    sd 0:0:2:0: [sdc] Mode Sense: 00 00 00 00
    sd 0:0:2:0: [sdc] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sdc: unknown partition table
    sd 0:0:2:0: [sdc] Attached SCSI disk
    sd 0:0:3:0: [sdd] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:3:0: [sdd] Write Protect is off
    sd 0:0:3:0: [sdd] Mode Sense: 00 00 00 00
    sd 0:0:3:0: [sdd] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sd 0:0:3:0: [sdd] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:3:0: [sdd] Write Protect is off
    sd 0:0:3:0: [sdd] Mode Sense: 00 00 00 00
    sd 0:0:3:0: [sdd] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sdd: unknown partition table
    sd 0:0:3:0: [sdd] Attached SCSI disk
    sd 0:0:4:0: [sde] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:4:0: [sde] Write Protect is off
    sd 0:0:4:0: [sde] Mode Sense: 00 00 00 00
    sd 0:0:4:0: [sde] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sd 0:0:4:0: [sde] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:4:0: [sde] Write Protect is off
    sd 0:0:4:0: [sde] Mode Sense: 00 00 00 00
    sd 0:0:4:0: [sde] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sde: unknown partition table
    sd 0:0:4:0: [sde] Attached SCSI disk
    sd 0:0:5:0: [sdf] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:5:0: [sdf] Write Protect is off
    sd 0:0:5:0: [sdf] Mode Sense: 00 00 00 00
    sd 0:0:5:0: [sdf] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sd 0:0:5:0: [sdf] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:5:0: [sdf] Write Protect is off
    sd 0:0:5:0: [sdf] Mode Sense: 00 00 00 00
    sd 0:0:5:0: [sdf] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sdf: unknown partition table
    sd 0:0:5:0: [sdf] Attached SCSI disk
    sd 0:0:6:0: [sdg] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:6:0: [sdg] Write Protect is off
    sd 0:0:6:0: [sdg] Mode Sense: 00 00 00 00
    sd 0:0:6:0: [sdg] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sd 0:0:6:0: [sdg] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:6:0: [sdg] Write Protect is off
    sd 0:0:6:0: [sdg] Mode Sense: 00 00 00 00
    sd 0:0:6:0: [sdg] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sdg: unknown partition table
    sd 0:0:6:0: [sdg] Attached SCSI disk
    sd 0:0:7:0: [sdh] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:7:0: [sdh] Write Protect is off
    sd 0:0:7:0: [sdh] Mode Sense: 00 00 00 00
    sd 0:0:7:0: [sdh] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sd 0:0:7:0: [sdh] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 0:0:7:0: [sdh] Write Protect is off
    sd 0:0:7:0: [sdh] Mode Sense: 00 00 00 00
    sd 0:0:7:0: [sdh] Write cache: enabled, read cache: disabled, supports DPO and FUA
    sdh: unknown partition table
    sd 0:0:7:0: [sdh] Attached SCSI disk
    Driver 'sr' needs updating - please use bus_type methods
    sd 0:0:0:0: Attached scsi generic sg0 type 0
    sd 0:0:1:0: Attached scsi generic sg1 type 0
    sd 0:0:2:0: Attached scsi generic sg2 type 0
    sd 0:0:3:0: Attached scsi generic sg3 type 0
    sd 0:0:4:0: Attached scsi generic sg4 type 0
    sd 0:0:5:0: Attached scsi generic sg5 type 0
    sd 0:0:6:0: Attached scsi generic sg6 type 0
    sd 0:0:7:0: Attached scsi generic sg7 type 0
    sata_mv 0000:00:08.0: version 1.24
    sata_mv 0000:00:08.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
    sata_mv 0000:00:08.0: PCI: Disallowing DAC for device
    sata_mv 0000:00:08.0: Gen-II 32 slots 8 ports SCSI mode IRQ via INTx
    scsi1 : sata_mv
    scsi2 : sata_mv
    scsi3 : sata_mv
    scsi4 : sata_mv
    scsi5 : sata_mv
    scsi6 : sata_mv
    scsi7 : sata_mv
    scsi8 : sata_mv
    ata1: SATA max UDMA/133 mmio m1048576@0xf8100000 port 0xf8122000 irq 18
    ata2: SATA max UDMA/133 mmio m1048576@0xf8100000 port 0xf8124000 irq 18
    ata3: SATA max UDMA/133 mmio m1048576@0xf8100000 port 0xf8126000 irq 18
    ata4: SATA max UDMA/133 mmio m1048576@0xf8100000 port 0xf8128000 irq 18
    ata5: SATA max UDMA/133 mmio m1048576@0xf8100000 port 0xf8132000 irq 18
    ata6: SATA max UDMA/133 mmio m1048576@0xf8100000 port 0xf8134000 irq 18
    ata7: SATA max UDMA/133 mmio m1048576@0xf8100000 port 0xf8136000 irq 18
    ata8: SATA max UDMA/133 mmio m1048576@0xf8100000 port 0xf8138000 irq 18
    ata1: SATA link down (SStatus 0 SControl 300)
    ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    ata2.00: ATA-7: ST3750640AS, 3.AAD, max UDMA/133
    ata2.00: 1465149168 sectors, multi 0: LBA48 NCQ (depth 31/32)
    ata2.00: max_sectors limited to 256 for NCQ
    ata2.00: max_sectors limited to 256 for NCQ
    ata2.00: configured for UDMA/133
    ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    ata3.00: ATA-7: ST3750640AS, 3.AAC, max UDMA/133
    ata3.00: 1465149168 sectors, multi 0: LBA48 NCQ (depth 31/32)
    ata3.00: max_sectors limited to 256 for NCQ
    ata3.00: max_sectors limited to 256 for NCQ
    ata3.00: configured for UDMA/133
    ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    ata4.00: ATA-7: ST3750640AS, 3.AAD, max UDMA/133
    ata4.00: 1465149168 sectors, multi 0: LBA48 NCQ (depth 31/32)
    ata4.00: max_sectors limited to 256 for NCQ
    ata4.00: max_sectors limited to 256 for NCQ
    ata4.00: configured for UDMA/133
    ata5: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
    ata5.00: ATA-7: ST3750640AS, 3.AAK, max UDMA/133
    ata5.00: 1465149168 sectors, multi 0: LBA48 NCQ (depth 31/32)
    ata5.00: max_sectors limited to 256 for NCQ
    ata5.00: max_sectors limited to 256 for NCQ
    ata5.00: configured for UDMA/133
    ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    ata6.00: ATA-7: ST3750640AS, 3.AAJ, max UDMA/133
    ata6.00: 1465149168 sectors, multi 0: LBA48 NCQ (depth 31/32)
    ata6.00: max_sectors limited to 256 for NCQ
    ata6.00: max_sectors limited to 256 for NCQ
    ata6.00: configured for UDMA/133
    ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    ata7.00: ATA-7: Maxtor 6H500F0, HA431BU0, max UDMA/133
    ata7.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
    ata7.00: max_sectors limited to 256 for NCQ
    ata7.00: max_sectors limited to 256 for NCQ
    ata7.00: configured for UDMA/133
    ata8: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    ata8.00: ATA-7: Maxtor 6H500F0, HA431BU0, max UDMA/133
    ata8.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
    ata8.00: max_sectors limited to 256 for NCQ
    ata8.00: max_sectors limited to 256 for NCQ
    ata8.00: configured for UDMA/133
    scsi 2:0:0:0: Direct-Access ATA ST3750640AS 3.AA PQ: 0 ANSI: 5
    sd 2:0:0:0: [sdi] 1465149168 512-byte hardware sectors: (750 GB/698 GiB)
    sd 2:0:0:0: [sdi] Write Protect is off
    sd 2:0:0:0: [sdi] Mode Sense: 00 3a 00 00
    sd 2:0:0:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sd 2:0:0:0: [sdi] 1465149168 512-byte hardware sectors: (750 GB/698 GiB)
    sd 2:0:0:0: [sdi] Write Protect is off
    sd 2:0:0:0: [sdi] Mode Sense: 00 3a 00 00
    sd 2:0:0:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sdi: unknown partition table
    sd 2:0:0:0: [sdi] Attached SCSI disk
    sd 2:0:0:0: Attached scsi generic sg8 type 0
    scsi 3:0:0:0: Direct-Access ATA ST3750640AS 3.AA PQ: 0 ANSI: 5
    sd 3:0:0:0: [sdj] 1465149168 512-byte hardware sectors: (750 GB/698 GiB)
    sd 3:0:0:0: [sdj] Write Protect is off
    sd 3:0:0:0: [sdj] Mode Sense: 00 3a 00 00
    sd 3:0:0:0: [sdj] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sd 3:0:0:0: [sdj] 1465149168 512-byte hardware sectors: (750 GB/698 GiB)
    sd 3:0:0:0: [sdj] Write Protect is off
    sd 3:0:0:0: [sdj] Mode Sense: 00 3a 00 00
    sd 3:0:0:0: [sdj] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sdj: unknown partition table
    sd 3:0:0:0: [sdj] Attached SCSI disk
    sd 3:0:0:0: Attached scsi generic sg9 type 0
    scsi 4:0:0:0: Direct-Access ATA ST3750640AS 3.AA PQ: 0 ANSI: 5
    sd 4:0:0:0: [sdk] 1465149168 512-byte hardware sectors: (750 GB/698 GiB)
    sd 4:0:0:0: [sdk] Write Protect is off
    sd 4:0:0:0: [sdk] Mode Sense: 00 3a 00 00
    sd 4:0:0:0: [sdk] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sd 4:0:0:0: [sdk] 1465149168 512-byte hardware sectors: (750 GB/698 GiB)
    sd 4:0:0:0: [sdk] Write Protect is off
    sd 4:0:0:0: [sdk] Mode Sense: 00 3a 00 00
    sd 4:0:0:0: [sdk] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sdk: unknown partition table
    sd 4:0:0:0: [sdk] Attached SCSI disk
    sd 4:0:0:0: Attached scsi generic sg10 type 0
    scsi 5:0:0:0: Direct-Access ATA ST3750640AS 3.AA PQ: 0 ANSI: 5
    sd 5:0:0:0: [sdl] 1465149168 512-byte hardware sectors: (750 GB/698 GiB)
    sd 5:0:0:0: [sdl] Write Protect is off
    sd 5:0:0:0: [sdl] Mode Sense: 00 3a 00 00
    sd 5:0:0:0: [sdl] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sd 5:0:0:0: [sdl] 1465149168 512-byte hardware sectors: (750 GB/698 GiB)
    sd 5:0:0:0: [sdl] Write Protect is off
    sd 5:0:0:0: [sdl] Mode Sense: 00 3a 00 00
    sd 5:0:0:0: [sdl] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sdl: unknown partition table
    sd 5:0:0:0: [sdl] Attached SCSI disk
    sd 5:0:0:0: Attached scsi generic sg11 type 0
    scsi 6:0:0:0: Direct-Access ATA ST3750640AS 3.AA PQ: 0 ANSI: 5
    sd 6:0:0:0: [sdm] 1465149168 512-byte hardware sectors: (750 GB/698 GiB)
    sd 6:0:0:0: [sdm] Write Protect is off
    sd 6:0:0:0: [sdm] Mode Sense: 00 3a 00 00
    sd 6:0:0:0: [sdm] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sd 6:0:0:0: [sdm] 1465149168 512-byte hardware sectors: (750 GB/698 GiB)
    sd 6:0:0:0: [sdm] Write Protect is off
    sd 6:0:0:0: [sdm] Mode Sense: 00 3a 00 00
    sd 6:0:0:0: [sdm] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sdm: unknown partition table
    sd 6:0:0:0: [sdm] Attached SCSI disk
    sd 6:0:0:0: Attached scsi generic sg12 type 0
    scsi 7:0:0:0: Direct-Access ATA Maxtor 6H500F0 HA43 PQ: 0 ANSI: 5
    sd 7:0:0:0: [sdn] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 7:0:0:0: [sdn] Write Protect is off
    sd 7:0:0:0: [sdn] Mode Sense: 00 3a 00 00
    sd 7:0:0:0: [sdn] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sd 7:0:0:0: [sdn] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 7:0:0:0: [sdn] Write Protect is off
    sd 7:0:0:0: [sdn] Mode Sense: 00 3a 00 00
    sd 7:0:0:0: [sdn] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sdn: unknown partition table
    sd 7:0:0:0: [sdn] Attached SCSI disk
    sd 7:0:0:0: Attached scsi generic sg13 type 0
    scsi 8:0:0:0: Direct-Access ATA Maxtor 6H500F0 HA43 PQ: 0 ANSI: 5
    sd 8:0:0:0: [sdo] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 8:0:0:0: [sdo] Write Protect is off
    sd 8:0:0:0: [sdo] Mode Sense: 00 3a 00 00
    sd 8:0:0:0: [sdo] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sd 8:0:0:0: [sdo] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 8:0:0:0: [sdo] Write Protect is off
    sd 8:0:0:0: [sdo] Mode Sense: 00 3a 00 00
    sd 8:0:0:0: [sdo] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sdo: unknown partition table
    sd 8:0:0:0: [sdo] Attached SCSI disk
    sd 8:0:0:0: Attached scsi generic sg14 type 0
    sata_mv 0000:00:0a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
    sata_mv 0000:00:0a.0: PCI: Disallowing DAC for device
    sata_mv 0000:00:0a.0: Gen-II 32 slots 8 ports SCSI mode IRQ via INTx
    scsi9 : sata_mv
    scsi10 : sata_mv
    scsi11 : sata_mv
    scsi12 : sata_mv
    scsi13 : sata_mv
    scsi14 : sata_mv
    scsi15 : sata_mv
    scsi16 : sata_mv
    ata9: SATA max UDMA/133 mmio m1048576@0xf8000000 port 0xf8022000 irq 16
    ata10: SATA max UDMA/133 mmio m1048576@0xf8000000 port 0xf8024000 irq 16
    ata11: SATA max UDMA/133 mmio m1048576@0xf8000000 port 0xf8026000 irq 16
    ata12: SATA max UDMA/133 mmio m1048576@0xf8000000 port 0xf8028000 irq 16
    ata13: SATA max UDMA/133 mmio m1048576@0xf8000000 port 0xf8032000 irq 16
    ata14: SATA max UDMA/133 mmio m1048576@0xf8000000 port 0xf8034000 irq 16
    ata15: SATA max UDMA/133 mmio m1048576@0xf8000000 port 0xf8036000 irq 16
    ata16: SATA max UDMA/133 mmio m1048576@0xf8000000 port 0xf8038000 irq 16
    ata9: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    ata9.00: ATA-8: WDC WD4001ABYS-01YNA0, 59.01D01, max UDMA/133
    ata9.00: 781422768 sectors, multi 0: LBA48 NCQ (depth 31/32)
    ata9.00: max_sectors limited to 256 for NCQ
    ata9.00: max_sectors limited to 256 for NCQ
    ata9.00: configured for UDMA/133
    ata10: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    ata10.00: ATA-7: WDC WD5000KS-75MNB0, 08.02E08, max UDMA/133
    ata10.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
    ata10.00: max_sectors limited to 256 for NCQ
    ata10.00: max_sectors limited to 256 for NCQ
    ata10.00: configured for UDMA/133
    ata11: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    ata11.00: ATA-7: WDC WD5000KS-75MNB0, 08.02E08, max UDMA/133
    ata11.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
    ata11.00: max_sectors limited to 256 for NCQ
    ata11.00: max_sectors limited to 256 for NCQ
    ata11.00: configured for UDMA/133
    ata12: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    ata12.00: ATA-7: WDC WD4000YS-01MPB1, 09.02E09, max UDMA/133
    ata12.00: 781422768 sectors, multi 0: LBA48 NCQ (depth 31/32)
    ata12.00: max_sectors limited to 256 for NCQ
    ata12.00: max_sectors limited to 256 for NCQ
    ata12.00: configured for UDMA/133
    ata13: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    ata13.00: ATA-7: WDC WD5000KS-75MNB0, 08.02E08, max UDMA/133
    ata13.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
    ata13.00: max_sectors limited to 256 for NCQ
    ata13.00: max_sectors limited to 256 for NCQ
    ata13.00: configured for UDMA/133
    ata14: SATA link down (SStatus 0 SControl 300)
    ata15: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    ata15.00: ATA-7: WDC WD5000KS-75MNB0, 08.02E08, max UDMA/133
    ata15.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
    ata15.00: max_sectors limited to 256 for NCQ
    ata15.00: max_sectors limited to 256 for NCQ
    ata15.00: configured for UDMA/133
    ata16: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    ata16.00: ATA-7: WDC WD5000KS-75MNB0, 08.02E08, max UDMA/133
    ata16.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
    ata16.00: max_sectors limited to 256 for NCQ
    ata16.00: max_sectors limited to 256 for NCQ
    ata16.00: configured for UDMA/133
    scsi 9:0:0:0: Direct-Access ATA WDC WD4001ABYS-0 59.0 PQ: 0 ANSI: 5
    sd 9:0:0:0: [sdp] 781422768 512-byte hardware sectors: (400 GB/372 GiB)
    sd 9:0:0:0: [sdp] Write Protect is off
    sd 9:0:0:0: [sdp] Mode Sense: 00 3a 00 00
    sd 9:0:0:0: [sdp] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sd 9:0:0:0: [sdp] 781422768 512-byte hardware sectors: (400 GB/372 GiB)
    sd 9:0:0:0: [sdp] Write Protect is off
    sd 9:0:0:0: [sdp] Mode Sense: 00 3a 00 00
    sd 9:0:0:0: [sdp] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sdp: unknown partition table
    sd 9:0:0:0: [sdp] Attached SCSI disk
    sd 9:0:0:0: Attached scsi generic sg15 type 0
    scsi 10:0:0:0: Direct-Access ATA WDC WD5000KS-75M 08.0 PQ: 0 ANSI: 5
    sd 10:0:0:0: [sdq] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 10:0:0:0: [sdq] Write Protect is off
    sd 10:0:0:0: [sdq] Mode Sense: 00 3a 00 00
    sd 10:0:0:0: [sdq] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sd 10:0:0:0: [sdq] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 10:0:0:0: [sdq] Write Protect is off
    sd 10:0:0:0: [sdq] Mode Sense: 00 3a 00 00
    sd 10:0:0:0: [sdq] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sdq: unknown partition table
    sd 10:0:0:0: [sdq] Attached SCSI disk
    sd 10:0:0:0: Attached scsi generic sg16 type 0
    scsi 11:0:0:0: Direct-Access ATA WDC WD5000KS-75M 08.0 PQ: 0 ANSI: 5
    sd 11:0:0:0: [sdr] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 11:0:0:0: [sdr] Write Protect is off
    sd 11:0:0:0: [sdr] Mode Sense: 00 3a 00 00
    sd 11:0:0:0: [sdr] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sd 11:0:0:0: [sdr] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 11:0:0:0: [sdr] Write Protect is off
    sd 11:0:0:0: [sdr] Mode Sense: 00 3a 00 00
    sd 11:0:0:0: [sdr] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sdr: unknown partition table
    sd 11:0:0:0: [sdr] Attached SCSI disk
    sd 11:0:0:0: Attached scsi generic sg17 type 0
    scsi 12:0:0:0: Direct-Access ATA WDC WD4000YS-01M 09.0 PQ: 0 ANSI: 5
    sd 12:0:0:0: [sds] 781422768 512-byte hardware sectors: (400 GB/372 GiB)
    sd 12:0:0:0: [sds] Write Protect is off
    sd 12:0:0:0: [sds] Mode Sense: 00 3a 00 00
    sd 12:0:0:0: [sds] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sd 12:0:0:0: [sds] 781422768 512-byte hardware sectors: (400 GB/372 GiB)
    sd 12:0:0:0: [sds] Write Protect is off
    sd 12:0:0:0: [sds] Mode Sense: 00 3a 00 00
    sd 12:0:0:0: [sds] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sds: unknown partition table
    sd 12:0:0:0: [sds] Attached SCSI disk
    sd 12:0:0:0: Attached scsi generic sg18 type 0
    scsi 13:0:0:0: Direct-Access ATA WDC WD5000KS-75M 08.0 PQ: 0 ANSI: 5
    sd 13:0:0:0: [sdt] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 13:0:0:0: [sdt] Write Protect is off
    sd 13:0:0:0: [sdt] Mode Sense: 00 3a 00 00
    sd 13:0:0:0: [sdt] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sd 13:0:0:0: [sdt] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 13:0:0:0: [sdt] Write Protect is off
    sd 13:0:0:0: [sdt] Mode Sense: 00 3a 00 00
    sd 13:0:0:0: [sdt] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sdt: unknown partition table
    sd 13:0:0:0: [sdt] Attached SCSI disk
    sd 13:0:0:0: Attached scsi generic sg19 type 0
    scsi 15:0:0:0: Direct-Access ATA WDC WD5000KS-75M 08.0 PQ: 0 ANSI: 5
    sd 15:0:0:0: [sdu] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 15:0:0:0: [sdu] Write Protect is off
    sd 15:0:0:0: [sdu] Mode Sense: 00 3a 00 00
    sd 15:0:0:0: [sdu] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sd 15:0:0:0: [sdu] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 15:0:0:0: [sdu] Write Protect is off
    sd 15:0:0:0: [sdu] Mode Sense: 00 3a 00 00
    sd 15:0:0:0: [sdu] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sdu: unknown partition table
    sd 15:0:0:0: [sdu] Attached SCSI disk
    sd 15:0:0:0: Attached scsi generic sg20 type 0
    scsi 16:0:0:0: Direct-Access ATA WDC WD5000KS-75M 08.0 PQ: 0 ANSI: 5
    sd 16:0:0:0: [sdv] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 16:0:0:0: [sdv] Write Protect is off
    sd 16:0:0:0: [sdv] Mode Sense: 00 3a 00 00
    sd 16:0:0:0: [sdv] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sd 16:0:0:0: [sdv] 976773168 512-byte hardware sectors: (500 GB/465 GiB)
    sd 16:0:0:0: [sdv] Write Protect is off
    sd 16:0:0:0: [sdv] Mode Sense: 00 3a 00 00
    sd 16:0:0:0: [sdv] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
    sdv: unknown partition table
    sd 16:0:0:0: [sdv] Attached SCSI disk
    sd 16:0:0:0: Attached scsi generic sg21 type 0
    I2O subsystem v1.325
    i2o: max drivers = 8
    I2O Configuration OSM v1.323
    I2O Bus Adapter OSM v1.317
    I2O Block Device OSM v1.325
    I2O SCSI Peripheral OSM v1.316
    I2O ProcFS OSM v1.316
    Fusion MPT base driver 3.04.07
    Copyright (c) 1999-2008 LSI Corporation
    Fusion MPT SPI Host driver 3.04.07
    ieee1394: raw1394: /dev/raw1394 device initialized
    usbmon: debugfs is not available
    ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
    ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
    uhci_hcd: USB Universal Host Controller Interface driver
    ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
    uhci_hcd 0000:00:11.2: PCI INT D -> Link[LNKD] -> GSI 11 (level, low) -> IRQ 11
    uhci_hcd 0000:00:11.2: UHCI Host Controller
    uhci_hcd 0000:00:11.2: new USB bus registered, assigned bus number 1
    uhci_hcd 0000:00:11.2: irq 11, io base 0x0000dc00
    usb usb1: configuration #1 chosen from 1 choice
    hub 1-0:1.0: USB hub found
    hub 1-0:1.0: 2 ports detected
    uhci_hcd 0000:00:11.3: PCI INT D -> Link[LNKD] -> GSI 11 (level, low) -> IRQ 11
    uhci_hcd 0000:00:11.3: UHCI Host Controller
    uhci_hcd 0000:00:11.3: new USB bus registered, assigned bus number 2
    uhci_hcd 0000:00:11.3: irq 11, io base 0x0000e000
    usb usb2: configuration #1 chosen from 1 choice
    hub 2-0:1.0: USB hub found
    hub 2-0:1.0: 2 ports detected
    uhci_hcd 0000:00:11.4: PCI INT D -> Link[LNKD] -> GSI 11 (level, low) -> IRQ 11
    uhci_hcd 0000:00:11.4: UHCI Host Controller
    uhci_hcd 0000:00:11.4: new USB bus registered, assigned bus number 3
    uhci_hcd 0000:00:11.4: irq 11, io base 0x0000e400
    usb usb3: configuration #1 chosen from 1 choice
    hub 3-0:1.0: USB hub found
    hub 3-0:1.0: 2 ports detected
    usbcore: registered new interface driver usblp
    Initializing USB Mass Storage driver...
    usbcore: registered new interface driver usb-storage
    USB Mass Storage support registered.
    PNP: PS/2 Controller [PNP0303:PS2K,PNP0f13:PS2M] at 0x60,0x64 irq 1,12
    serio: i8042 KBD port at 0x60,0x64 irq 1
    serio: i8042 AUX port at 0x60,0x64 irq 12
    mice: PS/2 mouse device common for all mice
    i2c /dev entries driver
    device-mapper: ioctl: 4.14.0-ioctl (2008-04-23) initialised: dm-devel@redhat.com
    cpuidle: using governor ladder
    padlock: VIA PadLock not detected.
    usbcore: registered new interface driver usbhid
    usbhid: v2.6:USB HID core driver
    Advanced Linux Sound Architecture Driver Version 1.0.18rc3.
    ALSA device list:
    No soundcards found.
    oprofile: using NMI interrupt.
    TCP cubic registered
    NET: Registered protocol family 10
    IPv6 over IPv4 tunneling driver
    NET: Registered protocol family 17
    RPC: Registered udp transport module.
    RPC: Registered tcp transport module.
    Using IPI No-Shortcut mode
    input: AT Translated Set 2 keyboard as /class/input/input2
    input: PS/2 Generic Mouse as /class/input/input3
    RAMDISK: Compressed image found at block 0
    VFS: Mounted root (ext2 filesystem).
    kjournald starting. Commit interval 5 seconds
    EXT3-fs: mounted filesystem with ordered data mode.
    Freeing unused kernel memory: 324k freed
    EXT3 FS on dm-1, internal journal
    Adding 2097136k swap on /swapfile. Priority:-1 extents:849 across:3103356k
    hda: UDMA/100 mode selected
    hdb: UDMA/100 mode selected
    hdc: UDMA/100 mode selected
    bonding: bond0: setting mode to balance-alb (6).
    bonding: bond0: Setting MII monitoring interval to 100.
    ADDRCONF(NETDEV_UP): bond0: link is not ready
    ADDRCONF(NETDEV_UP): eth7: link is not ready
    e1000: eth7: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
    ADDRCONF(NETDEV_CHANGE): eth7: link becomes ready
    ADDRCONF(NETDEV_UP): eth6: link is not ready
    e1000: eth6: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
    ADDRCONF(NETDEV_CHANGE): eth6: link becomes ready
    ADDRCONF(NETDEV_UP): eth5: link is not ready
    e100: eth5: e100_watchdog: link up, 100Mbps, full-duplex
    ADDRCONF(NETDEV_CHANGE): eth5: link becomes ready
    bonding: bond0: enslaving eth7 as an active interface with a down link.
    bonding: bond0: enslaving eth6 as an active interface with a down link.
    bonding: bond0: enslaving eth5 as an active interface with a down link.
    e100: eth5: e100_watchdog: link up, 100Mbps, full-duplex
    bonding: bond0: link status definitely up for interface eth5.
    bonding: bond0: making interface eth5 the new active one.
    device eth5 entered promiscuous mode
    bonding: bond0: first active interface up!
    ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
    e1000: eth7: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
    bonding: bond0: link status definitely up for interface eth7.
    e1000: eth6: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
    bonding: bond0: link status definitely up for interface eth6.
    device eth5 left promiscuous mode
    bond0: no IPv6 routers present
    SysRq : Show State
    task PC stack pid father
    init S f703fbc0 0 1 0
    f69176c0 00000086 c2014968 f703fbc0 f703fd14 c2017600 00000001 f7041b48
    00000282 c0133ed0 c2014968 00000000 00000282 1d7f2b13 00003d24 1d7f2b13
    00003d24 00000000 f7041b48 c0465b03 00000000 00000001 f66b3bc8 f66bdb48
    Call Trace:
    [<c0133ed0>] hrtimer_start_range_ns+0x10a/0x116
    [<c0465b03>] schedule_hrtimeout_range+0xcb/0xf2
    [<c0133b36>] hrtimer_wakeup+0x0/0x18
    [<c016be74>] do_select+0x390/0x407
    [<c016c23e>] __pollwait+0x0/0xac
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c03f7124>] tcp_rcv_established+0x383/0x4af
    [<c03fb9d5>] tcp_v4_do_rcv+0x1f/0x159
    [<c03d04d6>] __alloc_skb+0x49/0xf5
    [<c0106ca4>] nommu_map_single+0x2b/0x3a
    [<c02a810e>] e1000_alloc_rx_buffers+0x15d/0x23f
    [<c02a7f88>] e1000_clean_rx_irq+0x3aa/0x3d3
    [<c02a8800>] e1000_clean+0x369/0x47a
    [<c032b64f>] tw_interrupt+0x17/0x6b6
    [<c03d642c>] net_rx_action+0x5c/0x10f
    [<c0126ad0>] __do_softirq+0x6f/0x10f
    [<c01469e9>] get_page_from_freelist+0x8e/0x360
    [<c016d3f3>] __d_lookup+0x9c/0xd1
    [<c016c18e>] core_sys_select+0x2a3/0x2c4
    [<c0168a92>] __link_path_walk+0x935/0xa66
    [<c017106b>] mntput_no_expire+0x13/0xbd
    [<c0168c2a>] path_walk+0x67/0x70
    [<c0168d3f>] do_path_lookup+0xd8/0xf1
    [<c0229dc6>] copy_to_user+0x25/0x3b
    [<c0163870>] cp_new_stat64+0xfc/0x10e
    [<c0106cda>] read_tsc+0x6/0x22
    [<c01363a7>] getnstimeofday+0x4a/0xd0
    [<c012649c>] timespec_add_safe+0x1f/0x3e
    [<c016c21d>] sys_select+0x6e/0x8f
    [<c0102c19>] sysenter_do_call+0x12/0x25
    kthreadd S f703f850 0 2 0
    f62b4a00 00000046 f0b03f84 f703f850 f703f9a4 c2017600 00000001 00000003
    f0b03f8c f0b03f88 f0b03f8c 00000282 00000000 c011a792 00000000 c05f05d4
    00002cd3 f0b03f6c 00000000 c01317b0 00000000 000000ff 000000ff c0131745
    Call Trace:
    [<c011a792>] complete+0x28/0x36
    [<c01317b0>] kthreadd+0x6b/0x13b
    [<c0131745>] kthreadd+0x0/0x13b
    [<c010399b>] kernel_thread_helper+0x7/0x10
    migration/0 S f703f4e0 0 3 2
    f6281960 00000046 f710e0a0 f703f4e0 f703f634 c200f600 00000000 00000003
    ed877f68 ed877f64 ed877f68 00000203 c04713c0 c011a792 00000000 c200fa0c
    c2017600 c200f600 c04713c0 c012019d 00000000 00000001 00000000 c0120067
    Call Trace:
    [<c011a792>] complete+0x28/0x36
    [<c012019d>] migration_thread+0x136/0x1ae
    [<c0120067>] migration_thread+0x0/0x1ae
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    ksoftirqd/0 S f703f170 0 4 2
    f5c47d60 00000046 c05e9340 f703f170 f703f2c4 c200f600 00000000 00000001
    c0645a9c 0000000a c01419ef c0126ad0 00000000 00000246 c0126cc2 00000000
    c0126cc2 00000000 00000000 c0126ce5 00000000 c013195f c0131927 00000000
    Call Trace:
    [<c01419ef>] rcu_process_callbacks+0x34/0x3b
    [<c0126ad0>] __do_softirq+0x6f/0x10f
    [<c0126cc2>] ksoftirqd+0x0/0xa6
    [<c0126cc2>] ksoftirqd+0x0/0xa6
    [<c0126ce5>] ksoftirqd+0x23/0xa6
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    watchdog/0 S f703ee00 0 5 2
    f5c47d60 00000046 b525c8bb f703ee00 f703ef54 c200f600 00000000 c200fa80
    c013511d 00000246 01960000 00000380 0000f470 00000246 c06ac9b8 f68040c0
    c05e9340 00000380 0000f470 c013f656 00000000 00000063 00000000 c013f61c
    Call Trace:
    [<c013511d>] sched_clock_cpu+0x13e/0x149
    [<c013f656>] watchdog+0x3a/0x1bc
    [<c013f61c>] watchdog+0x0/0x1bc
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    migration/1 S f703ea90 0 6 2
    f5ccb760 00000046 203a0571 f703ea90 f703ebe4 c2017600 00000001 00000003
    ed877f68 ed877f64 ed877f68 00000206 c04713c0 c011a792 00000000 c2017a0c
    c200f600 c2017600 c04713c0 c012019d 00000001 00000000 00000001 c0120067
    Call Trace:
    [<c011a792>] complete+0x28/0x36
    [<c012019d>] migration_thread+0x136/0x1ae
    [<c0120067>] migration_thread+0x0/0x1ae
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    ksoftirqd/1 S f703e720 0 7 2
    f5c47d60 00000046 f703e040 f703e720 f703e874 c2017600 00000001 00000001
    c0645a9c 0000000a c01419ef c0126ad0 00000001 00000246 c0126cc2 00000001
    c0126cc2 00000000 00000000 c0126ce5 00000001 c013195f c0131927 00000000
    Call Trace:
    [<c01419ef>] rcu_process_callbacks+0x34/0x3b
    [<c0126ad0>] __do_softirq+0x6f/0x10f
    [<c0126cc2>] ksoftirqd+0x0/0xa6
    [<c0126cc2>] ksoftirqd+0x0/0xa6
    [<c0126ce5>] ksoftirqd+0x23/0xa6
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    watchdog/1 S f703e3b0 0 8 2
    f5c47d60 00000046 b536111d f703e3b0 f703e504 c2017600 00000001 c0471360
    c013f61c 00000246 01968000 00000000 00000000 00000246 c06ac9b8 00000001
    c013f61c 00000000 00000000 c013f656 00000001 00000063 00000001 c013f61c
    Call Trace:
    [<c013f61c>] watchdog+0x0/0x1bc
    [<c013f61c>] watchdog+0x0/0x1bc
    [<c013f656>] watchdog+0x3a/0x1bc
    [<c013f61c>] watchdog+0x0/0x1bc
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    events/0 S f7075be0 0 9 2
    f5c476e0 00000046 00000000 f7075be0 f7075d34 c200f600 00000000 ffffffff
    c012f604 f703bc40 c200cce0 c014cafc 00000000 00000246 c0131b8d f703bc40
    c012f756 f703bc4c 00000000 c012f7dc 00000000 f7075be0 c0131a1d f703bc50
    Call Trace:
    [<c012f604>] queue_delayed_work_on+0x8b/0x97
    [<c014cafc>] vmstat_update+0x0/0x24
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    events/1 S f7075870 0 10 2
    f5c47d60 00000046 00000000 f7075870 f70759c4 c2017600 00000001 ffffffff
    c012f604 f703bbc0 c2014ce0 c014cafc 00000000 00000246 c0131b8d f703bbc0
    c012f756 f703bbcc 00000000 c012f7dc 00000000 f7075870 c0131a1d f703bbd0
    Call Trace:
    [<c012f604>] queue_delayed_work_on+0x8b/0x97
    [<c014cafc>] vmstat_update+0x0/0x24
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    khelper S f7075500 0 11 2
    f6917380 00000046 00000000 f7075500 f7075654 c200f600 00000000 00000246
    00000000 c011a792 00000000 00000000 f703bb40 00000246 c0131b8d f703bb40
    c012f756 f703bb4c 00000000 c012f7dc 00000000 f7075500 c0131a1d f703bb50
    Call Trace:
    [<c011a792>] complete+0x28/0x36
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kstop/0 S f7075190 0 14 2
    c05e91a0 00000046 00000000 f7075190 f70752e4 c200f600 00000000 f7075190
    f7041df0 c0465313 f703fbc0 f7075190 f70752e4 00000246 c0131b8d f705aea0
    c012f756 f705aeac 00000000 c012f7dc 00000000 f7075190 c0131a1d f705aeb0
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kstop/1 S f7074e20 0 15 2
    c05e91a0 00000046 00000001 f7074e20 f7074f74 c2017600 00000001 f7074e20
    00000000 c0465313 f703e040 f7074e20 f7074f74 00000246 c0131b8d f705ae20
    c012f756 f705ae2c 00000000 c012f7dc 00000000 f7074e20 c0131a1d f705ae30
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kblockd/0 S f7074ab0 0 170 2
    f5c476e0 00000046 c0302f2e f7074ab0 f7074c04 c200f600 00000000 f68ac5f0
    c021a64a 00000000 c021bbff f68ac558 c021be3a 00000246 c0131b8d f705a120
    c012f756 f705a12c 00000000 c012f7dc 00000000 f7074ab0 c0131a1d f705a130
    Call Trace:
    [<c0302f2e>] scsi_request_fn+0x2bc/0x359
    [<c021a64a>] blk_unplug_work+0x0/0xc
    [<c021bbff>] __generic_unplug_device+0x1a/0x1c
    [<c021be3a>] generic_unplug_device+0x1e/0x29
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kblockd/1 S f7074740 0 171 2
    f5c476e0 00000046 c0302f2e f7074740 f7074894 c2017600 00000001 f68a2650
    c021a64a 00000000 c021bbff f68a25b8 c021be3a 00000246 c0131b8d f705a0a0
    c012f756 f705a0ac 00000000 c012f7dc 00000000 f7074740 c0131a1d f705a0b0
    Call Trace:
    [<c0302f2e>] scsi_request_fn+0x2bc/0x359
    [<c021a64a>] blk_unplug_work+0x0/0xc
    [<c021bbff>] __generic_unplug_device+0x1a/0x1c
    [<c021be3a>] generic_unplug_device+0x1e/0x29
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kacpid S f70743d0 0 174 2
    c05e91a0 00000046 00000001 f70743d0 f7074524 c2017600 00000001 f70743d0
    00000000 c0465313 f703e040 f70743d0 f7074524 00000246 c0131b8d f70b8c40
    c012f756 f70b8c4c 00000000 c012f7dc 00000000 f70743d0 c0131a1d f70b8c50
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kacpi_notify S f7074060 0 175 2
    c05e91a0 00000046 00000000 f7074060 f70741b4 c200f600 00000000 f7074060
    f7041ddc c0465313 f703fbc0 f7074060 f70741b4 00000246 c0131b8d f70b8b40
    c012f756 f70b8b4c 00000000 c012f7dc 00000000 f7074060 c0131a1d f70b8b50
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    ata/0 S f707d1b0 0 231 2
    c05e91a0 00000046 00000000 f707d1b0 f707d304 c200f600 00000000 00000009
    f6aa12dc 00000000 c0341e1f 00000001 f70eece0 00000246 c0131b8d f70eece0
    c012f756 f70eecec 00000000 c012f7dc 00000000 f707d1b0 c0131a1d f70eecf0
    Call Trace:
    [<c0341e1f>] ata_pio_task+0xa6/0xb3
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    ata/1 S f707d520 0 232 2
    c05e91a0 00000046 00000001 f707d520 f707d674 c2017600 00000001 f707d520
    00000000 c0465313 f703e040 f707d520 f707d674 00000246 c0131b8d f70eec60
    c012f756 f70eec6c 00000000 c012f7dc 00000000 f707d520 c0131a1d f70eec70
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    ata_aux S f707dc00 0 233 2
    c05e91a0 00000046 00000000 f707dc00 f707dd54 c200f600 00000000 f707dc00
    f7041dec c0465313 f703fbc0 f707dc00 f707dd54 00000246 c0131b8d f70eebe0
    c012f756 f70eebec 00000000 c012f7dc 00000000 f707dc00 c0131a1d f70eebf0
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    ksuspend_usbd S f710fc20 0 234 2
    c05e91a0 00000046 00000001 f710fc20 f710fd74 c2017600 00000001 f710fc20
    00000000 c0465313 f703e040 f710fc20 f710fd74 00000246 c0131b8d f70eeae0
    c012f756 f70eeaec 00000000 c012f7dc 00000000 f710fc20 c0131a1d f70eeaf0
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    khubd S f710f8b0 0 240 2
    c05e91a0 00000046 f7075870 f710f8b0 f710fa04 c2017600 00000001 00000000
    c011a5a8 00001a4f 00000000 c2017634 f710f8b0 00000246 c0131b8d c062a4e8
    f689d0c0 f7119fbc 00000000 c036f3c6 f6be821c f6be5c00 00000001 00000000
    Call Trace:
    [<c011a5a8>] update_curr+0x5c/0x95
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c036f3c6>] hub_thread+0x9ed/0xaaf
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c011a792>] complete+0x28/0x36
    [<c036e9d9>] hub_thread+0x0/0xaaf
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kseriod S f710f540 0 243 2
    c05e91a0 00000046 f70ee6a0 f710f540 f710f694 c200f600 00000000 c062dfa8
    c0296d41 00000000 c029657b f70ee69c 00000000 00000246 c0131b8d f6bf33e0
    f6bf33e0 f7117fc8 00000246 c038ac81 00000000 f710f540 c0131a1d c062d764
    Call Trace:
    [<c0296d41>] __driver_attach+0x0/0x67
    [<c029657b>] bus_for_each_dev+0x4c/0x55
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c038ac81>] serio_thread+0x25c/0x2c7
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c038aa25>] serio_thread+0x0/0x2c7
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kswapd0 S f710f1d0 0 299 2
    f62b4a00 00000046 f7169f08 f710f1d0 f710f324 c2017600 00000001 000000d0
    c014b4dd 00000000 00000000 00000000 00000018 00000246 c0131b8d 00000001
    00000000 00000000 ffffffff c014bcf0 00000000 c0638980 0000000c 00078d75
    Call Trace:
    [<c014b4dd>] shrink_slab+0x126/0x132
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c014bcf0>] kswapd+0xc3/0x49f
    [<c0149ebd>] isolate_pages_global+0x0/0x1bc
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c011a792>] complete+0x28/0x36
    [<c014bc2d>] kswapd+0x0/0x49f
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    aio/0 S f710ee60 0 300 2
    c05e91a0 00000046 00000000 f710ee60 f710efb4 c200f600 00000000 f710ee60
    c0643000 c0465313 c05e9340 f710ee60 f710efb4 00000246 c0131b8d f7089e60
    c012f756 f7089e6c 00000000 c012f7dc 00000000 f710ee60 c0131a1d f7089e70
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    aio/1 S f710eaf0 0 301 2
    c05e91a0 00000046 00000001 f710eaf0 f710ec44 c2017600 00000001 f710eaf0
    f7041dec c0465313 f703fbc0 f710eaf0 f710ec44 00000246 c0131b8d f7089de0
    c012f756 f7089dec 00000000 c012f7dc 00000000 f710eaf0 c0131a1d f7089df0
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    nfsiod S f710e780 0 302 2
    c05e91a0 00000046 00000000 f710e780 f710e8d4 c200f600 00000000 f710e780
    c0643000 c0465313 c05e9340 f710e780 f710e8d4 00000246 c0131b8d f7329d20
    c012f756 f7329d2c 00000000 c012f7dc 00000000 f710e780 c0131a1d f7329d30
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    jfsIO S f710e410 0 304 2
    f62b4a00 00000046 00000001 f710e410 f710e564 c2017600 00000001 00000000
    c021bf2c 01000000 f63f1ea0 f5c35e80 00000000 00000000 c017b70f 00000000
    c020423d 00000000 00000000 c02042a1 c013195f c0131927 00000000 c010399b
    Call Trace:
    [<c021bf2c>] submit_bio+0x92/0x97
    [<c017b70f>] bio_alloc+0xb/0x17
    [<c020423d>] jfsIOWait+0x0/0x7c
    [<c02042a1>] jfsIOWait+0x64/0x7c
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    jfsCommit R running 0 305 2
    f6e9c658 628e5000 00000000 f23a5d98 c0142db8 c15437c0 00000000 f4981840
    c02027cd 00001000 0000c160 f6e9c5b0 00000000 00000000 00000000 00000003
    f4981c78 c1542280 00002000 0000c161 00002000 efcb69f0 c01fac44 00000286
    Call Trace:
    [<c0142db8>] read_cache_page+0xa/0x3a
    [<c02027cd>] __get_metapage+0x339/0x347
    [<c01fac44>] dbUpdatePMap+0x170/0x413
    [<c02049a1>] txAllocPMap+0xdb/0xed
    [<c0204d3a>] txUpdateMap+0x86/0x1b8
    [<c02061cb>] txEnd+0xf7/0xfc
    [<c02068ae>] jfs_lazycommit+0x82/0x1de
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c020682c>] jfs_lazycommit+0x0/0x1de
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    jfsCommit S f710e0a0 0 306 2
    f5c47d60 00000046 f710e410 f710e0a0 f710e1f4 c2017600 00000001 00000000
    00000000 00000003 f5c35e80 f8041050 0000090c 00000286 00000046 f5c35e80
    00000246 f7375fc8 f5c35d80 c02069db 00000000 f710e0a0 c011c8f9 c05f51f4
    Call Trace:
    [<c02069db>] jfs_lazycommit+0x1af/0x1de
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c020682c>] jfs_lazycommit+0x0/0x1de
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    jfsSync S f7377c40 0 307 2
    c05e91a0 00000046 c06af600 f7377c40 f7377d94 c200f600 00000000 f7377c40
    f7377d94 c200f600 00000000 00000003 f7041e44 f7041e40 f7041e44 00000000
    c0206561 00000000 00000000 c02066bc c0206561 00000000 c0206561 c013195f
    Call Trace:
    [<c0206561>] jfs_sync+0x0/0x17a
    [<c02066bc>] jfs_sync+0x15b/0x17a
    [<c0206561>] jfs_sync+0x0/0x17a
    [<c0206561>] jfs_sync+0x0/0x17a
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    bond0 S f73efc00 0 991 2
    f5c47d60 00000046 c02b4c52 f73efc00 f73efd54 c2017600 00000001 f7145498
    00000000 00000003 00000000 f687dbe0 f71456a4 00000246 c0131b8d f687dbe0
    c012f756 f687dbec 00000000 c012f7dc 00000000 f73efc00 c0131a1d f687dbf0
    Call Trace:
    [<c02b4c52>] bond_mii_monitor+0x0/0x3f8
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_0 S f7377560 0 1035 2
    c05e91a0 00000046 00000000 f7377560 f73776b4 c2017600 00000001 00000000
    f7377560 13007002 c06af600 f7377560 13007002 c0465313 f703fbc0 f7158000
    f7158000 00000000 00000000 c0301035 f7041d90 f7041d8c f7041d90 00000282
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_1 S f73f1540 0 1091 2
    c05e91a0 00000046 00000000 f73f1540 f73f1694 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f68ad400 00000000 00000000 00000246
    f68ad400 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_2 S f73f18b0 0 1093 2
    c05e91a0 00000046 00000000 f73f18b0 f73f1a04 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f68ad000 00000000 00000000 00000246
    f68ad000 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_3 S f73f1c20 0 1095 2
    c05e91a0 00000046 00000000 f73f1c20 f73f1d74 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f6a54c00 00000000 00000000 00000246
    f6a54c00 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_4 S f73dc080 0 1097 2
    c05e91a0 00000046 00000000 f73dc080 f73dc1d4 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f6a54800 00000000 00000000 00000246
    f6a54800 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_5 S f73dc760 0 1099 2
    c05e91a0 00000046 00000000 f73dc760 f73dc8b4 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f6a54400 00000000 00000000 00000246
    f6a54400 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_6 S f73dc3f0 0 1101 2
    c05e91a0 00000046 00000000 f73dc3f0 f73dc544 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f6a54000 00000000 00000000 00000246
    f6a54000 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_7 S f73fa450 0 1103 2
    c05e91a0 00000046 00000000 f73fa450 f73fa5a4 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f6a56c00 00000000 00000000 00000246
    f6a56c00 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_8 S f73fa0e0 0 1105 2
    c05e91a0 00000046 00000000 f73fa0e0 f73fa234 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f6a56800 00000000 00000000 00000246
    f6a56800 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_9 S f7376b10 0 1149 2
    c05e91a0 00000046 00000000 f7376b10 f7376c64 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f73a8c00 00000000 00000000 00000246
    f73a8c00 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_10 S f73771f0 0 1151 2
    c05e91a0 00000046 00000000 f73771f0 f7377344 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f73a8800 00000000 00000000 00000246
    f73a8800 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_11 S f7376e80 0 1153 2
    c05e91a0 00000046 00000000 f7376e80 f7376fd4 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f73a8400 00000000 00000000 00000246
    f73a8400 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_12 S f7376430 0 1155 2
    c05e91a0 00000046 00000000 f7376430 f7376584 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f73a8000 00000000 00000000 00000246
    f73a8000 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_13 S f73760c0 0 1157 2
    c05e91a0 00000046 00000000 f73760c0 f7376214 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f6b94c00 00000000 00000000 00000246
    f6b94c00 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_14 S f73fab30 0 1159 2
    c05e91a0 00000046 00000000 f73fab30 f73fac84 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f6b94800 00000000 00000000 00000246
    f6b94800 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_15 S f707ce40 0 1161 2
    c05e91a0 00000046 00000000 f707ce40 f707cf94 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f6b94400 00000000 00000000 00000246
    f6b94400 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    scsi_eh_16 S f707c760 0 1163 2
    c05e91a0 00000046 00000000 f707c760 f707c8b4 c200f600 00000000 00000000
    00000246 c02fe22a 00000246 00000246 f6b94000 00000000 00000000 00000246
    f6b94000 00000000 00000000 c0301035 f7041d14 f7041d10 f7041d14 00000292
    Call Trace:
    [<c02fe22a>] __scsi_iterate_devices+0x44/0x57
    [<c0301035>] scsi_error_handler+0x22/0x234
    [<c011a792>] complete+0x28/0x36
    [<c0301013>] scsi_error_handler+0x0/0x234
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    exec-osm/0 S f73dce40 0 1211 2
    c05e91a0 00000046 00000000 f73dce40 f73dcf94 c200f600 00000000 f73dce40
    f7043fb8 c0465313 f703f850 f73dce40 f73dcf94 00000246 c0131b8d f68ba1a0
    c012f756 f68ba1ac 00000000 c012f7dc 00000000 f73dce40 c0131a1d f68ba1b0
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    exec-osm/1 S f73dd1b0 0 1212 2
    c05e91a0 00000046 00000001 f73dd1b0 f73dd304 c2017600 00000001 f73dd1b0
    f7041dd0 c0465313 f703fbc0 f73dd1b0 f73dd304 00000246 c0131b8d f68ba120
    c012f756 f68ba12c 00000000 c012f7dc 00000000 f73dd1b0 c0131a1d f68ba130
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    block-osm/0 S f68051f0 0 1218 2
    c05e91a0 00000046 00000000 f68051f0 f6805344 c200f600 00000000 f68051f0
    00000000 c0465313 f703f170 f68051f0 f6805344 00000246 c0131b8d f6bafe40
    c012f756 f6bafe4c 00000000 c012f7dc 00000000 f68051f0 c0131a1d f6bafe50
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    block-osm/1 S f6804e80 0 1219 2
    c05e91a0 00000046 00000001 f6804e80 f6804fd4 c2017600 00000001 f6804e80
    f7041dd0 c0465313 f703fbc0 f6804e80 f6804fd4 00000246 c0131b8d f6bafdc0
    c012f756 f6bafdcc 00000000 c012f7dc 00000000 f6804e80 c0131a1d f6bafdd0
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    khpsbpkt S f68047a0 0 1224 2
    c05e91a0 00000046 c0643000 f68047a0 f68048f4 c200f600 00000000 c05e9340
    f68047a0 f68048f4 c200f600 00000000 00000003 f7041e40 f7041e3c f6be1fc8
    f6be1fac 00000000 00000000 c035b0ff f6be1fc8 f6be1fc8 00000000 c035b048
    Call Trace:
    [<c035b0ff>] hpsbpkt_thread+0xb7/0xd8
    [<c035b048>] hpsbpkt_thread+0x0/0xd8
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kpsmoused S f73f0af0 0 1269 2
    c05e91a0 00000046 00000000 f73f0af0 f73f0c44 c200f600 00000000 f73f0af0
    c0643000 c0465313 c05e9340 f73f0af0 f73f0c44 00000246 c0131b8d f6bbc360
    c012f756 f6bbc36c 00000000 c012f7dc 00000000 f73f0af0 c0131a1d f6bbc370
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kstriped S f73f0e60 0 1276 2
    c05e91a0 00000046 00000000 f73f0e60 f73f0fb4 c200f600 00000000 f73f0e60
    00000000 c0465313 f703f170 f73f0e60 f73f0fb4 00000246 c0131b8d f6bf9da0
    c012f756 f6bf9dac 00000000 c012f7dc 00000000 f73f0e60 c0131a1d f6bf9db0
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kondemand/0 S f73807c0 0 1278 2
    c05e91a0 00000046 00000000 f73807c0 f7380914 c200f600 00000000 f73807c0
    c0643000 c0465313 c05e9340 f73807c0 f7380914 00000246 c0131b8d f6bf9ca0
    c012f756 f6bf9cac 00000000 c012f7dc 00000000 f73807c0 c0131a1d f6bf9cb0
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kondemand/1 S f7380b30 0 1279 2
    c05e91a0 00000046 00000001 f7380b30 f7380c84 c2017600 00000001 f7380b30
    f7041df0 c0465313 f703fbc0 f7380b30 f7380c84 00000246 c0131b8d f6bf9c20
    c012f756 f6bf9c2c 00000000 c012f7dc 00000000 f7380b30 c0131a1d f6bf9c30
    Call Trace:
    [<c0465313>] schedule+0x586/0x5d6
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    hid_compat S f7381210 0 1281 2
    c05e91a0 00000046 00000000 f7381210 f7381364 c200f600 00000000 c0559be7
    f6409f44 00000000 f6409fa8 f6bf9b20 c062ff3c 00000246 c0131b8d f6bf9b20
    c012f756 f6bf9b2c 00000000 c012f7dc 00000000 f7381210 c0131a1d f6bf9b30
    Call Trace:
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    rpciod/0 S f7381580 0 1310 2
    f5cfa920 00000046 c03cce38 f7381580 f73816d4 c200f600 00000000 f5ca0800
    c043d228 00000000 c043c669 f5ca0a90 c043d253 00000246 c0131b8d f6434f60
    c012f756 f6434f6c 00000000 c012f7dc 00000000 f7381580 c0131a1d f6434f70
    Call Trace:
    [<c03cce38>] release_sock+0xf/0x84
    [<c043d228>] xprt_autoclose+0x0/0x34
    [<c043c669>] xprt_release_xprt+0x39/0x66
    [<c043d253>] xprt_autoclose+0x2b/0x34
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    rpciod/1 S f73eee40 0 1311 2
    f6281e40 00000046 c03ca852 f73eee40 f73eef94 c2017600 00000001 f5ca0800
    f5ca0b14 c043f758 00000000 c043f812 f6ed7b60 00000246 c0131b8d f6434ee0
    c012f756 f6434eec 00000000 c012f7dc 00000000 f73eee40 c0131a1d f6434ef0
    Call Trace:
    [<c03ca852>] kernel_connect+0xb/0xe
    [<c043f758>] xs_tcp_connect_worker4+0x0/0x12e
    [<c043f812>] xs_tcp_connect_worker4+0xba/0x12e
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kdmflush S f6449c40 0 1361 2
    f5c476e0 00000046 00000000 f6449c40 f6449d94 c2017600 00000001 00000282
    00000000 c011a792 00000000 00000000 f68941e0 00000246 c0131b8d f68941e0
    c012f756 f68941ec 00000000 c012f7dc 00000000 f6449c40 c0131a1d f68941f0
    Call Trace:
    [<c011a792>] complete+0x28/0x36
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kdmflush S f6448b10 0 1365 2
    f5c476e0 00000046 00000000 f6448b10 f6448c64 c2017600 00000001 00000282
    00000000 c011a792 00000000 00000000 f632aea0 00000246 c0131b8d f632aea0
    c012f756 f632aeac 00000000 c012f7dc 00000000 f6448b10 c0131a1d f632aeb0
    Call Trace:
    [<c011a792>] complete+0x28/0x36
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kdmflush S f64480c0 0 1369 2
    f5c476e0 00000046 00000000 f64480c0 f6448214 c200f600 00000000 00000282
    00000000 c011a792 00000000 00000000 f632ada0 00000246 c0131b8d f632ada0
    c012f756 f632adac 00000000 c012f7dc 00000000 f64480c0 c0131a1d f632adb0
    Call Trace:
    [<c011a792>] complete+0x28/0x36
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kdmflush S f634fc60 0 1373 2
    f5c476e0 00000046 00000000 f634fc60 f634fdb4 c2017600 00000001 00000282
    00000000 c011a792 00000000 00000000 f632aca0 00000246 c0131b8d f632aca0
    c012f756 f632acac 00000000 c012f7dc 00000000 f634fc60 c0131a1d f632acb0
    Call Trace:
    [<c011a792>] complete+0x28/0x36
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kdmflush S f6448430 0 1377 2
    f5c476e0 00000046 00000000 f6448430 f6448584 c2017600 00000001 00000282
    00000000 c011a792 00000000 00000000 f632aba0 00000246 c0131b8d f632aba0
    c012f756 f632abac 00000000 c012f7dc 00000000 f6448430 c0131a1d f632abb0
    Call Trace:
    [<c011a792>] complete+0x28/0x36
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c012f756>] worker_thread+0x0/0xbd
    [<c012f7dc>] worker_thread+0x86/0xbd
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    kjournald S f64498d0 0 1384 2
    f5c47d60 00000046 00000000 f64498d0 f6449a24 c200f600 00000000 00000000
    00000001 00000286 c011a807 00000000 00000000 00000246 c0131b8d f6916a00
    00000000 f6916a68 f6916a14 c01a94ce f6916a50 f6916aec 00000000 f64498d0
    Call Trace:
    [<c011a807>] __wake_up+0x29/0x39
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c01a94ce>] kjournald+0x154/0x1c2
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c01a937a>] kjournald+0x0/0x1c2
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    udevd S f73ef890 0 1497 1
    f5c47bc0 00000086 c05e9340 f73ef890 f73ef9e4 c200f600 00000000 00000000
    c0104740 c04b6280 f691fe40 00000008 00000003 c0103767 c04b6280 00000000
    00000008 00000000 00000000 c0465a69 00000000 0000007b 0000007b 00000246
    Call Trace:
    [<c0104740>] do_IRQ+0x8e/0xa2
    [<c0103767>] common_interrupt+0x23/0x28
    [<c0465a69>] schedule_hrtimeout_range+0x31/0xf2
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c04158ee>] unix_peer_get+0xe/0x24
    [<c0416d81>] unix_dgram_poll+0xb3/0x123
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c046582c>] mutex_lock+0xb/0x1a
    [<c016be74>] do_select+0x390/0x407
    [<c016c23e>] __pollwait+0x0/0xac
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c02290ce>] vsnprintf+0x4c0/0x502
    [<c02290ce>] vsnprintf+0x4c0/0x502
    [<c02286bc>] number+0x13d/0x217
    [<c01469e9>] get_page_from_freelist+0x8e/0x360
    [<c02290ce>] vsnprintf+0x4c0/0x502
    [<c014e794>] do_wp_page+0x3be/0x427
    [<c014f862>] handle_mm_fault+0x4a0/0x4ee
    [<c0468269>] do_page_fault+0x2b8/0x5c3
    [<c016c18e>] core_sys_select+0x2a3/0x2c4
    [<c016d75e>] dput+0x21/0xf3
    [<c0167191>] __follow_mount+0x1e/0x60
    [<c016725e>] do_lookup+0x53/0x153
    [<c0168af8>] __link_path_walk+0x99b/0xa66
    [<c017106b>] mntput_no_expire+0x13/0xbd
    [<c0168c2a>] path_walk+0x67/0x70
    [<c016d3f3>] __d_lookup+0x9c/0xd1
    [<c016cebe>] shrink_dcache_parent+0x4e/0xc7
    [<c0166f4e>] inode_permission+0x57/0x6a
    [<c01741de>] simple_empty+0xe/0x4c
    [<c0131bcd>] remove_wait_queue+0xf/0x34
    [<c012511a>] do_wait+0x1e7/0x299
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c016c21d>] sys_select+0x6e/0x8f
    [<c0125273>] sys_waitpid+0x13/0x17
    [<c0102c19>] sysenter_do_call+0x12/0x25
    syslog-ng S f6590ea0 0 6622 1
    f5ccb420 00000082 c200c968 f6590ea0 f6590ff4 c200f600 00000000 f6209bc8
    00000282 c0133ed0 c200c968 00000000 00000282 dac05d44 00003e39 dac05d44
    00003e39 00000000 f6209bc8 c0465b03 00000000 f66bff59 00000000 00000000
    Call Trace:
    [<c0133ed0>] hrtimer_start_range_ns+0x10a/0x116
    [<c0465b03>] schedule_hrtimeout_range+0xcb/0xf2
    [<c0133b36>] hrtimer_wakeup+0x0/0x18
    [<c016b83e>] do_sys_poll+0x253/0x323
    [<c016c23e>] __pollwait+0x0/0xac
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c01989f0>] ext3_mark_inode_dirty+0x29/0x30
    [<c011a807>] __wake_up+0x29/0x39
    [<c01a52a7>] journal_stop+0x144/0x14d
    [<c019f296>] __ext3_journal_stop+0x19/0x34
    [<c019a5f2>] ext3_ordered_write_end+0xf2/0x100
    [<c0143123>] generic_file_buffered_write+0x12c/0x254
    [<c016d3f3>] __d_lookup+0x9c/0xd1
    [<c016725e>] do_lookup+0x53/0x153
    [<c03cfa69>] skb_dequeue+0x42/0x48
    [<c0416aff>] unix_stream_recvmsg+0x389/0x428
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cfa69>] skb_dequeue+0x42/0x48
    [<c0106cda>] read_tsc+0x6/0x22
    [<c01363a7>] getnstimeofday+0x4a/0xd0
    [<c012649c>] timespec_add_safe+0x1f/0x3e
    [<c016b56f>] poll_select_set_timeout+0x4c/0x66
    [<c016ba5a>] sys_poll+0x46/0x91
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    portmap S f65900e0 0 7177 1
    f645fa40 00000082 c0400459 f65900e0 f6590234 c2017600 00000001 f6916600
    f5c62824 00000024 c01987a4 00001000 f7731680 00005eea 0100007f 00000000
    f63cbe90 00000000 00000000 c0465a69 00000000 f5c62810 f600b980 00000246
    Call Trace:
    [<c0400459>] __udp4_lib_rcv+0x495/0x6f0
    [<c01987a4>] __ext3_get_inode_loc+0xcf/0x284
    [<c0465a69>] schedule_hrtimeout_range+0x31/0xf2
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c03d20c1>] datagram_poll+0x14/0xa8
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c03eefe1>] tcp_poll+0x14/0x130
    [<c016b83e>] do_sys_poll+0x253/0x323
    [<c016c23e>] __pollwait+0x0/0xac
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c03ea160>] ip_finish_output+0x1bc/0x1f3
    [<c03e8d9a>] ip_cork_release+0x2a/0x36
    [<c03e90bc>] ip_push_pending_frames+0x2a3/0x2ac
    [<c03ffc4a>] udp_push_pending_frames+0x29d/0x302
    [<c046674d>] _spin_lock_bh+0x8/0x1e
    [<c03cce38>] release_sock+0xf/0x84
    [<c0400c35>] udp_sendmsg+0x489/0x539
    [<c04058ac>] inet_sendmsg+0x35/0x3f
    [<c03cb559>] sock_sendmsg+0xb8/0xd1
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c0229bd3>] copy_from_user+0x23/0x4f
    [<c0229bd3>] copy_from_user+0x23/0x4f
    [<c03d16d7>] verify_iovec+0x3e/0x6f
    [<c0229bd3>] copy_from_user+0x23/0x4f
    [<c03cb701>] sys_sendmsg+0x18f/0x1f2
    [<c0146600>] __pagevec_free+0x18/0x22
    [<c0148fb8>] release_pages+0x15f/0x167
    [<c016ba5a>] sys_poll+0x46/0x91
    [<c0102c19>] sysenter_do_call+0x12/0x25
    rpc.statd S f7380450 0 7239 1
    f5cfa5e0 00000082 f62c71b0 f7380450 f73805a4 c2017600 00000001 f78c95e8
    c200f650 f65900e0 f738047c c200f600 c0120f17 00000001 00000001 00000000
    0000000a 00000000 00000000 c0465a69 00000000 c03cdae3 f63ef400 00000246
    Call Trace:
    [<c0120f17>] check_preempt_wakeup+0x108/0x121
    [<c0465a69>] schedule_hrtimeout_range+0x31/0xf2
    [<c03cdae3>] sock_queue_rcv_skb+0xaa/0xb1
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c03d20c1>] datagram_poll+0x14/0xa8
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c03eefe1>] tcp_poll+0x14/0x130
    [<c016be74>] do_select+0x390/0x407
    [<c016c23e>] __pollwait+0x0/0xac
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c0127167>] local_bh_enable+0x6e/0x7b
    [<c03d6e7e>] dev_queue_xmit+0x394/0x3c6
    [<c03ea160>] ip_finish_output+0x1bc/0x1f3
    [<c03e8d9a>] ip_cork_release+0x2a/0x36
    [<c03e90bc>] ip_push_pending_frames+0x2a3/0x2ac
    [<c03ffc4a>] udp_push_pending_frames+0x29d/0x302
    [<c046674d>] _spin_lock_bh+0x8/0x1e
    [<c03cce38>] release_sock+0xf/0x84
    [<c0400c35>] udp_sendmsg+0x489/0x539
    [<c04058ac>] inet_sendmsg+0x35/0x3f
    [<c03cb559>] sock_sendmsg+0xb8/0xd1
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016d3f3>] __d_lookup+0x9c/0xd1
    [<c016c18e>] core_sys_select+0x2a3/0x2c4
    [<c03d3e9e>] __dev_get_by_name+0x68/0x73
    [<c03d4e21>] netdev_run_todo+0x35/0x188
    [<c0229dc6>] copy_to_user+0x25/0x3b
    [<c0404e51>] devinet_ioctl+0x4f6/0x524
    [<c03cae6f>] sock_ioctl+0x19c/0x1c0
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c046674d>] _spin_lock_bh+0x8/0x1e
    [<c03ccf35>] lock_sock_nested+0x88/0x90
    [<c016a715>] vfs_ioctl+0x1c/0x5d
    [<c016c21d>] sys_select+0x6e/0x8f
    [<c0102c19>] sysenter_do_call+0x12/0x25
    rpc.mountd S f6539540 0 7299 1
    f5c883a0 00000086 00000000 f6539540 f6539694 c200f600 00000000 024c97af
    00000000 c02f0c44 024c97af c02f0d42 ffffffff f2cc7f8a 00000007 00000000
    00000008 00000000 00000000 c0465a69 00000000 00000000 c2017634 00000246
    Call Trace:
    [<c02f0c44>] ide_do_request+0x6d1/0x81e
    [<c02f0d42>] ide_do_request+0x7cf/0x81e
    [<c0465a69>] schedule_hrtimeout_range+0x31/0xf2
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c03d20c1>] datagram_poll+0x14/0xa8
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c03eefe1>] tcp_poll+0x14/0x130
    [<c016be74>] do_select+0x390/0x407
    [<c016c23e>] __pollwait+0x0/0xac
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c012a174>] __mod_timer+0x8d/0x97
    [<c022953a>] delay_tsc+0x1b/0x37
    [<c02f67d1>] dma_timer_expiry+0x0/0x61
    [<c02294f6>] __delay+0x6/0x7
    [<c02f1c92>] ide_execute_command+0x50/0x65
    [<c02f67d1>] dma_timer_expiry+0x0/0x61
    [<c02f6727>] ide_dma_intr+0x0/0x7d
    [<c02f67ce>] ide_dma_exec_cmd+0x17/0x1a
    [<c02f67d1>] dma_timer_expiry+0x0/0x61
    [<c02f4fc1>] do_rw_taskfile+0x19e/0x1af
    [<c02f9fbc>] ide_do_rw_disk+0x212/0x25f
    [<c02f1594>] ide_tf_load+0x124/0x12c
    [<c0146c5f>] get_page_from_freelist+0x304/0x360
    [<c02f0c44>] ide_do_request+0x6d1/0x81e
    [<c0146d8d>] __alloc_pages_internal+0xc1/0x34c
    [<c014f583>] handle_mm_fault+0x1c1/0x4ee
    [<c0468269>] do_page_fault+0x2b8/0x5c3
    [<c016d3f3>] __d_lookup+0x9c/0xd1
    [<c016725e>] do_lookup+0x53/0x153
    [<c016c18e>] core_sys_select+0x2a3/0x2c4
    [<c01428d7>] find_get_page+0x1d/0x7a
    [<c0142aad>] find_lock_page+0x1b/0x48
    [<c0144603>] filemap_fault+0x1e2/0x343
    [<c015ffb3>] nameidata_to_filp+0x1c/0x2c
    [<c014e394>] __do_fault+0x2e9/0x32b
    [<c0467fb1>] do_page_fault+0x0/0x5c3
    [<c014f629>] handle_mm_fault+0x267/0x4ee
    [<c016c21d>] sys_select+0x6e/0x8f
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    lockd S f634e0e0 0 7301 2
    f62b4860 00000046 c2017634 f634e0e0 f634e234 c2017600 00000001 c04713c0
    00a94c15 00000001 f6303bec c2017634 00000202 c0141951 00000101 7fffffff
    f63ecd90 f5d47f64 00000000 c04655a2 00000001 00000246 f63ecd90 f5d47f64
    Call Trace:
    [<c0141951>] __rcu_process_callbacks+0xfb/0x165
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c044afc4>] svc_recv+0x2d1/0x699
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c01de278>] lockd+0xa1/0x152
    [<c0465313>] schedule+0x586/0x5d6
    [<c011a792>] complete+0x28/0x36
    [<c01de1d7>] lockd+0x0/0x152
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    nfsd S f65d03b0 0 7302 2
    f62b4a00 00000046 f73efc00 f65d03b0 f65d0504 c2017600 00000001 c012a06d
    f6091f68 0103f8b4 f6091f68 00000286 c012a174 00000000 00000286 0103f8b4
    0103f8b4 f6091f9c f639d000 c04655fa f60d1f68 f666ff68 0103f8b4 c0129f53
    Call Trace:
    [<c012a06d>] lock_timer_base+0x19/0x35
    [<c012a174>] __mod_timer+0x8d/0x97
    [<c04655fa>] schedule_timeout+0x6b/0x86
    [<c0129f53>] process_timeout+0x0/0x5
    [<c044afc4>] svc_recv+0x2d1/0x699
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c01d0cc0>] nfsd+0xc9/0x249
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    nfsd S f7381c60 0 7303 2
    f62b4a00 00000046 f65d03b0 f7381c60 f7381db4 c2017600 00000001 c012a06d
    f60d1f68 0103f8b4 f60d1f68 00000286 c012a174 00000000 00000286 0103f8b4
    0103f8b4 f60d1f9c f6086000 c04655fa f612ff68 f6091f68 0103f8b4 c0129f53
    Call Trace:
    [<c012a06d>] lock_timer_base+0x19/0x35
    [<c012a174>] __mod_timer+0x8d/0x97
    [<c04655fa>] schedule_timeout+0x6b/0x86
    [<c0129f53>] process_timeout+0x0/0x5
    [<c044afc4>] svc_recv+0x2d1/0x699
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c01d0cc0>] nfsd+0xc9/0x249
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    nfsd S f73dd520 0 7304 2
    f62b4a00 00000046 f7381c60 f73dd520 f73dd674 c2017600 00000001 c012a06d
    f612ff68 0103f8b4 f612ff68 00000286 c012a174 00000000 00000286 0103f8b4
    0103f8b4 f612ff9c f60cb000 c04655fa f616ff68 f60d1f68 0103f8b4 c0129f53
    Call Trace:
    [<c012a06d>] lock_timer_base+0x19/0x35
    [<c012a174>] __mod_timer+0x8d/0x97
    [<c04655fa>] schedule_timeout+0x6b/0x86
    [<c0129f53>] process_timeout+0x0/0x5
    [<c044afc4>] svc_recv+0x2d1/0x699
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c01d0cc0>] nfsd+0xc9/0x249
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    nfsd S f5c77560 0 7305 2
    f62b4a00 00000046 f73dd520 f5c77560 f5c776b4 c2017600 00000001 c012a06d
    f616ff68 0103f8b4 f616ff68 00000286 c012a174 00000000 00000286 0103f8b4
    0103f8b4 f616ff9c f6110000 c04655fa f61aff68 f612ff68 0103f8b4 c0129f53
    Call Trace:
    [<c012a06d>] lock_timer_base+0x19/0x35
    [<c012a174>] __mod_timer+0x8d/0x97
    [<c04655fa>] schedule_timeout+0x6b/0x86
    [<c0129f53>] process_timeout+0x0/0x5
    [<c044afc4>] svc_recv+0x2d1/0x699
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c01d0cc0>] nfsd+0xc9/0x249
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    nfsd S f62c7520 0 7306 2
    f62b4a00 00000046 f5c77560 f62c7520 f62c7674 c2017600 00000001 c012a06d
    f61aff68 0103f8b4 f61aff68 00000286 c012a174 00000000 00000286 0103f8b4
    0103f8b4 f61aff9c f6155000 c04655fa f61eff68 f616ff68 0103f8b4 c0129f53
    Call Trace:
    [<c012a06d>] lock_timer_base+0x19/0x35
    [<c012a174>] __mod_timer+0x8d/0x97
    [<c04655fa>] schedule_timeout+0x6b/0x86
    [<c0129f53>] process_timeout+0x0/0x5
    [<c044afc4>] svc_recv+0x2d1/0x699
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c01d0cc0>] nfsd+0xc9/0x249
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    nfsd S f73ddc00 0 7307 2
    f62b4a00 00000046 f62c7520 f73ddc00 f73ddd54 c2017600 00000001 c012a06d
    f61eff68 0103f8b4 f61eff68 00000286 c012a174 00000000 00000286 0103f8b4
    0103f8b4 f61eff9c f619a000 c04655fa f662ff68 f61aff68 0103f8b4 c0129f53
    Call Trace:
    [<c012a06d>] lock_timer_base+0x19/0x35
    [<c012a174>] __mod_timer+0x8d/0x97
    [<c04655fa>] schedule_timeout+0x6b/0x86
    [<c0129f53>] process_timeout+0x0/0x5
    [<c044afc4>] svc_recv+0x2d1/0x699
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c01d0cc0>] nfsd+0xc9/0x249
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    nfsd S f62c7890 0 7308 2
    f62b4a00 00000046 f73ddc00 f62c7890 f62c79e4 c2017600 00000001 c012a06d
    f662ff68 0103f8b4 f662ff68 00000286 c012a174 00000000 00000286 0103f8b4
    0103f8b4 f662ff9c f61df000 c04655fa f704ca84 f61eff68 0103f8b4 c0129f53
    Call Trace:
    [<c012a06d>] lock_timer_base+0x19/0x35
    [<c012a174>] __mod_timer+0x8d/0x97
    [<c04655fa>] schedule_timeout+0x6b/0x86
    [<c0129f53>] process_timeout+0x0/0x5
    [<c044afc4>] svc_recv+0x2d1/0x699
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c01d0cc0>] nfsd+0xc9/0x249
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    nfsd S f65d0720 0 7309 2
    f62b4a00 00000046 f62c2060 f65d0720 f65d0874 c2017600 00000001 c012a06d
    f666ff68 0103f8b4 f666ff68 00000286 c012a174 00000000 00000286 0103f8b4
    0103f8b4 f666ff9c f6624000 c04655fa f6091f68 f704ca84 0103f8b4 c0129f53
    Call Trace:
    [<c012a06d>] lock_timer_base+0x19/0x35
    [<c012a174>] __mod_timer+0x8d/0x97
    [<c04655fa>] schedule_timeout+0x6b/0x86
    [<c0129f53>] process_timeout+0x0/0x5
    [<c044afc4>] svc_recv+0x2d1/0x699
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c01d0cc0>] nfsd+0xc9/0x249
    [<c01d0bf7>] nfsd+0x0/0x249
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    ntpd S f62c3870 0 7415 1
    f6281140 00200082 c2014968 f62c3870 f62c39c4 c2017600 00000001 f66b3bc8
    00200282 c0133ed0 c2014968 00000000 00200282 b4bf01e9 00003d27 b4bf01e9
    00003d27 00000000 f66b3bc8 c0465b03 00000000 f7041b49 00000000 00000000
    Call Trace:
    [<c0133ed0>] hrtimer_start_range_ns+0x10a/0x116
    [<c0465b03>] schedule_hrtimeout_range+0xcb/0xf2
    [<c0133b36>] hrtimer_wakeup+0x0/0x18
    [<c016b83e>] do_sys_poll+0x253/0x323
    [<c016c23e>] __pollwait+0x0/0xac
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c03d6e7e>] dev_queue_xmit+0x394/0x3c6
    [<c03db350>] neigh_resolve_output+0x203/0x22d
    [<c03ea160>] ip_finish_output+0x1bc/0x1f3
    [<c03e8d9a>] ip_cork_release+0x2a/0x36
    [<c03e90bc>] ip_push_pending_frames+0x2a3/0x2ac
    [<c03ffc4a>] udp_push_pending_frames+0x29d/0x302
    [<c0229dc6>] copy_to_user+0x25/0x3b
    [<c03d1676>] memcpy_toiovec+0x27/0x4a
    [<c03d1c2d>] skb_copy_datagram_iovec+0x41/0x18b
    [<c046674d>] _spin_lock_bh+0x8/0x1e
    [<c046674d>] _spin_lock_bh+0x8/0x1e
    [<c03cce38>] release_sock+0xf/0x84
    [<c040140b>] udp_recvmsg+0x1c3/0x22c
    [<c03cc96b>] sock_common_recvmsg+0x2f/0x45
    [<c03cb486>] sock_recvmsg+0xc8/0xe3
    [<c03cb559>] sock_sendmsg+0xb8/0xd1
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c0301e7c>] scsi_init_sgtable+0x58/0x80
    [<c0302040>] scsi_init_io+0x11/0x92
    [<c032e4b9>] sd_prep_fn+0x57/0x567
    [<c0219c3a>] elv_next_request+0x143/0x14f
    [<c03cc09d>] sys_recvfrom+0xb4/0x11d
    [<c021b4ea>] blk_invoke_request_fn+0x63/0xf9
    [<c02a8800>] e1000_clean+0x369/0x47a
    [<c0106cda>] read_tsc+0x6/0x22
    [<c01363a7>] getnstimeofday+0x4a/0xd0
    [<c012649c>] timespec_add_safe+0x1f/0x3e
    [<c016b56f>] poll_select_set_timeout+0x4c/0x66
    [<c016ba5a>] sys_poll+0x46/0x91
    [<c0102c19>] sysenter_do_call+0x12/0x25
    ntpd S f65380a0 0 7416 1
    f5c0b8e0 00200082 c200c968 f65380a0 f65381f4 c200f600 00000000 f6469bc8
    00200282 c0133ed0 c200c968 00000000 00200282 8931b176 00003d25 8931b176
    00003d25 00000000 f6469bc8 c0465b03 00000000 f66bff59 00000000 00000000
    Call Trace:
    [<c0133ed0>] hrtimer_start_range_ns+0x10a/0x116
    [<c0465b03>] schedule_hrtimeout_range+0xcb/0xf2
    [<c0133b36>] hrtimer_wakeup+0x0/0x18
    [<c016b83e>] do_sys_poll+0x253/0x323
    [<c016c23e>] __pollwait+0x0/0xac
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c03df65a>] __qdisc_run+0xdb/0x187
    [<c03d6e7e>] dev_queue_xmit+0x394/0x3c6
    [<c02b476b>] bond_dev_queue_xmit+0x116/0x133
    [<c02b9e98>] bond_alb_xmit+0x47e/0x4ae
    [<c03d4c69>] dev_hard_start_xmit+0x1e8/0x246
    [<c03d6e7e>] dev_queue_xmit+0x394/0x3c6
    [<c011a5a8>] update_curr+0x5c/0x95
    [<c011a5a8>] update_curr+0x5c/0x95
    [<c011a8ac>] task_rq_lock+0x28/0x4b
    [<c011c8f0>] try_to_wake_up+0x135/0x13e
    [<c0119e5c>] __wake_up_common+0x2e/0x58
    [<c011a7ca>] __wake_up_sync+0x2a/0x3e
    [<c03ce54e>] sock_def_readable+0x2f/0x58
    [<c04172f6>] unix_stream_sendmsg+0x210/0x2d0
    [<c03caaec>] sock_aio_write+0xb3/0xc0
    [<c0120f17>] check_preempt_wakeup+0x108/0x121
    [<c02a8800>] e1000_clean+0x369/0x47a
    [<c02a8800>] e1000_clean+0x369/0x47a
    [<c0126ad0>] __do_softirq+0x6f/0x10f
    [<c0106cda>] read_tsc+0x6/0x22
    [<c01363a7>] getnstimeofday+0x4a/0xd0
    [<c012649c>] timespec_add_safe+0x1f/0x3e
    [<c016b56f>] poll_select_set_timeout+0x4c/0x66
    [<c016ba5a>] sys_poll+0x46/0x91
    [<c0102c19>] sysenter_do_call+0x12/0x25
    smbd S f62c6760 0 7472 1
    f5ccb900 00000086 00000246 f62c6760 f62c68b4 c200f600 00000000 f629a220
    00000020 c03d04d6 000005f2 2c1d6812 f70db458 f89e80d0 c0106ca4 00000000
    00000016 00000000 00000000 c0465a69 00000000 ffffffff f687ace0 f7360000
    Call Trace:
    [<c03d04d6>] __alloc_skb+0x49/0xf5
    [<c0106ca4>] nommu_map_single+0x2b/0x3a
    [<c0465a69>] schedule_hrtimeout_range+0x31/0xf2
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c0165d0b>] pipe_poll+0x24/0x7c
    [<c016be74>] do_select+0x390/0x407
    [<c016c23e>] __pollwait+0x0/0xac
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c0104740>] do_IRQ+0x8e/0xa2
    [<c0103767>] common_interrupt+0x23/0x28
    [<c0228474>] put_dec+0xca/0x1d5
    [<c03ce54e>] sock_def_readable+0x2f/0x58
    [<c02286bc>] number+0x13d/0x217
    [<c03d04d6>] __alloc_skb+0x49/0xf5
    [<c0106ca4>] nommu_map_single+0x2b/0x3a
    [<c02290ce>] vsnprintf+0x4c0/0x502
    [<c02a8800>] e1000_clean+0x369/0x47a
    [<c0146c5f>] get_page_from_freelist+0x304/0x360
    [<c016c18e>] core_sys_select+0x2a3/0x2c4
    [<c016d447>] d_lookup+0x1f/0x2f
    [<c018c960>] proc_flush_task+0x119/0x213
    [<c02a8800>] e1000_clean+0x369/0x47a
    [<c016c21d>] sys_select+0x6e/0x8f
    [<c0104740>] do_IRQ+0x8e/0xa2
    [<c0102c19>] sysenter_do_call+0x12/0x25
    smbd S f73818f0 0 7476 7472
    f5c373e0 00000082 f5c373e0 f73818f0 f7381a44 c2017600 00000001 b7fb30a0
    f73818f0 00000007 c0468269 00000001 f63bffb8 f5c37414 f5c373e0 b7fb2e04
    b7fba420 00000013 f63be000 c012bfeb c0102c19 b7fb2e04 00000837 b8093c00
    Call Trace:
    [<c0468269>] do_page_fault+0x2b8/0x5c3
    [<c012bfeb>] sys_pause+0x11/0x17
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    nmbd S f6302a90 0 7482 1
    f5c88bc0 00000086 c2014968 f6302a90 f6302be4 c2017600 00000001 f66bdb48
    00000282 c0133ed0 c2014968 00000000 00000282 bce12e88 00003d23 bce12e88
    00003d23 00000000 f66bdb48 c0465b03 00000000 f7041b49 00000000 00000000
    Call Trace:
    [<c0133ed0>] hrtimer_start_range_ns+0x10a/0x116
    [<c0465b03>] schedule_hrtimeout_range+0xcb/0xf2
    [<c0133b36>] hrtimer_wakeup+0x0/0x18
    [<c016be74>] do_select+0x390/0x407
    [<c016c23e>] __pollwait+0x0/0xac
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c03d6e7e>] dev_queue_xmit+0x394/0x3c6
    [<c02b476b>] bond_dev_queue_xmit+0x116/0x133
    [<c02b9e98>] bond_alb_xmit+0x47e/0x4ae
    [<c03d4c69>] dev_hard_start_xmit+0x1e8/0x246
    [<c03d6e7e>] dev_queue_xmit+0x394/0x3c6
    [<c03ea160>] ip_finish_output+0x1bc/0x1f3
    [<c03e8e17>] ip_local_out+0x15/0x17
    [<c03ea771>] ip_queue_xmit+0x24e/0x291
    [<c046674d>] _spin_lock_bh+0x8/0x1e
    [<c03cce38>] release_sock+0xf/0x84
    [<c040140b>] udp_recvmsg+0x1c3/0x22c
    [<c03fc297>] tcp_v4_send_check+0x6e/0xa4
    [<c03f805c>] tcp_transmit_skb+0x56c/0x59f
    [<c03ce52e>] sock_def_readable+0xf/0x58
    [<c03f7124>] tcp_rcv_established+0x383/0x4af
    [<c03fb9d5>] tcp_v4_do_rcv+0x1f/0x159
    [<c016c18e>] core_sys_select+0x2a3/0x2c4
    [<c02a7f88>] e1000_clean_rx_irq+0x3aa/0x3d3
    [<c02a8800>] e1000_clean+0x369/0x47a
    [<c0106cda>] read_tsc+0x6/0x22
    [<c01363a7>] getnstimeofday+0x4a/0xd0
    [<c012649c>] timespec_add_safe+0x1f/0x3e
    [<c016c21d>] sys_select+0x6e/0x8f
    [<c0102c19>] sysenter_do_call+0x12/0x25
    sshd S f5c77c40 0 7539 1
    f5c47a20 00200082 f634eea0 f5c77c40 f5c77d94 c2017600 00000001 000005f2
    c05ea880 f65dc720 2dac1012 c02a810e 00000002 ffffffff f687ace0 00000000
    00000004 00000000 00000000 c0465a69 00000000 000000d0 000005ee f5d8f080
    Call Trace:
    [<c02a810e>] e1000_alloc_rx_buffers+0x15d/0x23f
    [<c0465a69>] schedule_hrtimeout_range+0x31/0xf2
    [<c02a7f88>] e1000_clean_rx_irq+0x3aa/0x3d3
    [<c0131bcd>] remove_wait_queue+0xf/0x34
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c03eefe1>] tcp_poll+0x14/0x130
    [<c016be74>] do_select+0x390/0x407
    [<c016c23e>] __pollwait+0x0/0xac
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c021e016>] blk_recount_segments+0x27/0x3f
    [<c01448b7>] mempool_alloc+0x28/0xc7
    [<c01448b7>] mempool_alloc+0x28/0xc7
    [<c017b469>] bvec_alloc_bs+0x7d/0xc2
    [<c017b67f>] bio_alloc_bioset+0x52/0x97
    [<c01448b7>] mempool_alloc+0x28/0xc7
    [<c0399612>] clone_bio+0x3f/0x6f
    [<c0399856>] __map_bio+0x31/0x8c
    [<c022223e>] as_add_request+0x93/0xbf
    [<c0219d00>] elv_insert+0xba/0x14e
    [<c021c256>] __make_request+0x325/0x357
    [<c021b06e>] generic_make_request+0x1f9/0x22f
    [<c011a5a8>] update_curr+0x5c/0x95
    [<c0465313>] schedule+0x586/0x5d6
    [<c016c18e>] core_sys_select+0x2a3/0x2c4
    [<c01428d7>] find_get_page+0x1d/0x7a
    [<c0142aad>] find_lock_page+0x1b/0x48
    [<c0144603>] filemap_fault+0x1e2/0x343
    [<c01175b1>] ptep_set_access_flags+0x20/0x27
    [<c014e556>] do_wp_page+0x180/0x427
    [<c016c21d>] sys_select+0x6e/0x8f
    [<c015fc97>] filp_close+0x4d/0x53
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    cron S f73778d0 0 7595 1
    f645f700 00000086 c200c968 f73778d0 f7377a24 c200f600 00000000 f66bff58
    00000282 c0133ed0 c200c968 00000000 00000282 0000c350 00000001 0000c350
    00000001 f66bff58 0000c350 c0465b87 00000000 00000001 0000c350 bfd57104
    Call Trace:
    [<c0133ed0>] hrtimer_start_range_ns+0x10a/0x116
    [<c0465b87>] do_nanosleep+0x54/0x8c
    [<c0133f4c>] hrtimer_nanosleep+0x70/0xcd
    [<c0133b36>] hrtimer_wakeup+0x0/0x18
    [<c0133fea>] sys_nanosleep+0x41/0x51
    [<c0102c19>] sysenter_do_call+0x12/0x25
    xinetd S f64487a0 0 7653 1
    f645fd80 00000082 f5c760c0 f64487a0 f64488f4 c200f600 00000000 f65ba000
    c0140d1f 00000012 00000000 f714a598 00000040 c200d274 000000ec 00000000
    00000006 00000000 00000000 c0465a69 00000000 0000000a c0126ad0 00000246
    Call Trace:
    [<c0140d1f>] handle_fasteoi_irq+0x94/0x9b
    [<c0465a69>] schedule_hrtimeout_range+0x31/0xf2
    [<c0126ad0>] __do_softirq+0x6f/0x10f
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c03d20c1>] datagram_poll+0x14/0xa8
    [<c0400fb0>] udp_poll+0x10/0xd5
    [<c016be74>] do_select+0x390/0x407
    [<c016c23e>] __pollwait+0x0/0xac
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c0106c59>] nommu_map_sg+0x5b/0x7b
    [<c0106c6c>] nommu_map_sg+0x6e/0x7b
    [<c012a06d>] lock_timer_base+0x19/0x35
    [<c012a174>] __mod_timer+0x8d/0x97
    [<c022953a>] delay_tsc+0x1b/0x37
    [<c02f67d1>] dma_timer_expiry+0x0/0x61
    [<c02294f6>] __delay+0x6/0x7
    [<c02f1c92>] ide_execute_command+0x50/0x65
    [<c02f67d1>] dma_timer_expiry+0x0/0x61
    [<c02f6727>] ide_dma_intr+0x0/0x7d
    [<c02286bc>] number+0x13d/0x217
    [<c011a5a8>] update_curr+0x5c/0x95
    [<c0110f74>] xapic_wait_icr_idle+0x11/0x17
    [<c011a729>] resched_task+0x56/0x58
    [<c011c8f0>] try_to_wake_up+0x135/0x13e
    [<c03d1f61>] __skb_recv_datagram+0xa9/0x1d4
    [<c04012ab>] udp_recvmsg+0x63/0x22c
    [<c03ce54e>] sock_def_readable+0x2f/0x58
    [<c04172f6>] unix_stream_sendmsg+0x210/0x2d0
    [<c03cc96b>] sock_common_recvmsg+0x2f/0x45
    [<c03cb486>] sock_recvmsg+0xc8/0xe3
    [<c03cb559>] sock_sendmsg+0xb8/0xd1
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016c18e>] core_sys_select+0x2a3/0x2c4
    [<c03cc09d>] sys_recvfrom+0xb4/0x11d
    [<c0168d3f>] do_path_lookup+0xd8/0xf1
    [<c0229dc6>] copy_to_user+0x25/0x3b
    [<c0163870>] cp_new_stat64+0xfc/0x10e
    [<c0131bcd>] remove_wait_queue+0xf/0x34
    [<c012511a>] do_wait+0x1e7/0x299
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c016c21d>] sys_select+0x6e/0x8f
    [<c0125273>] sys_waitpid+0x13/0x17
    [<c0102c19>] sysenter_do_call+0x12/0x25
    agetty S f6303170 0 7716 1
    f5c37720 00000082 20000002 f6303170 f63032c4 c2017600 00000001 00000007
    000000ff 00000000 00000000 00000020 c20136d4 c0146600 c1fcd340 7fffffff
    f5c96400 f66bbf48 bfb0decb c04655a2 00000246 c0123621 00000246 00000246
    Call Trace:
    [<c0146600>] __pagevec_free+0x18/0x22
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0123621>] release_console_sem+0x169/0x182
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c023ba22>] n_tty_read+0x311/0x557
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c023b711>] n_tty_read+0x0/0x557
    [<c023968f>] tty_read+0x5b/0x9a
    [<c0239634>] tty_read+0x0/0x9a
    [<c016188c>] vfs_read+0x81/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    agetty S f6302040 0 7717 1
    f62b4520 00000082 20000002 f6302040 f6302194 c2017600 00000001 00000007
    000000ff 00000000 00000000 00000020 c20136d4 c0146600 c1fcd340 7fffffff
    f5c9e000 f6063f48 bf98aecb c04655a2 00000246 c0123621 00000246 00000246
    Call Trace:
    [<c0146600>] __pagevec_free+0x18/0x22
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0123621>] release_console_sem+0x169/0x182
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c023ba22>] n_tty_read+0x311/0x557
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c023b711>] n_tty_read+0x0/0x557
    [<c023968f>] tty_read+0x5b/0x9a
    [<c0239634>] tty_read+0x0/0x9a
    [<c016188c>] vfs_read+0x81/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    agetty S f65d14e0 0 7718 1
    f5c37580 00000082 20000002 f65d14e0 f65d1634 c2017600 00000001 00000007
    000000ff 00000000 00000000 00000020 c20136d4 c0146600 c1fcd340 7fffffff
    f5c5a000 f66c1f48 bfe8943b c04655a2 00000246 c0123621 00000246 00000246
    Call Trace:
    [<c0146600>] __pagevec_free+0x18/0x22
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0123621>] release_console_sem+0x169/0x182
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c023ba22>] n_tty_read+0x311/0x557
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c023b711>] n_tty_read+0x0/0x557
    [<c023968f>] tty_read+0x5b/0x9a
    [<c0239634>] tty_read+0x0/0x9a
    [<c016188c>] vfs_read+0x81/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    agetty S f6538af0 0 7719 1
    f62b41e0 00000082 20000002 f6538af0 f6538c44 c200f600 00000000 00000007
    000000ff 00000000 00000000 00000020 c200b6d4 c0146600 c1fd6940 7fffffff
    f5c9d400 f66c3f48 bff0743b c04655a2 00000246 c0123621 00000246 00000246
    Call Trace:
    [<c0146600>] __pagevec_free+0x18/0x22
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0123621>] release_console_sem+0x169/0x182
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c023ba22>] n_tty_read+0x311/0x557
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c023b711>] n_tty_read+0x0/0x557
    [<c023968f>] tty_read+0x5b/0x9a
    [<c0239634>] tty_read+0x0/0x9a
    [<c016188c>] vfs_read+0x81/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    agetty S f6539c20 0 7720 1
    f645f560 00000086 20000002 f6539c20 f6539d74 c2017600 00000001 00000007
    000000ff 00000000 00000000 00000020 c20136d4 c0146600 c1fcd340 7fffffff
    f5c56400 f66c9f48 bfe6339b c04655a2 00000246 c0123621 00000246 00000246
    Call Trace:
    [<c0146600>] __pagevec_free+0x18/0x22
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0123621>] release_console_sem+0x169/0x182
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c023ba22>] n_tty_read+0x311/0x557
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c023b711>] n_tty_read+0x0/0x557
    [<c023968f>] tty_read+0x5b/0x9a
    [<c0239634>] tty_read+0x0/0x9a
    [<c016188c>] vfs_read+0x81/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    agetty S f6448e80 0 7721 1
    f5c47880 00000082 20000002 f6448e80 f6448fd4 c2017600 00000001 00000007
    000000ff 00000000 00000000 00000020 c20136d4 c0146600 c1fcd340 7fffffff
    f63fec00 f66cbf48 bfb1284b c04655a2 00000246 c0123621 00000246 00000246
    Call Trace:
    [<c0146600>] __pagevec_free+0x18/0x22
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0123621>] release_console_sem+0x169/0x182
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c023ba22>] n_tty_read+0x311/0x557
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c023b711>] n_tty_read+0x0/0x557
    [<c023968f>] tty_read+0x5b/0x9a
    [<c0239634>] tty_read+0x0/0x9a
    [<c016188c>] vfs_read+0x81/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    bash S f73f00a0 0 7879 1
    f5c88200 00000086 b7edfb00 f73f00a0 f73f01f4 c200f600 00000000 00000200
    f6538410 00000000 f73f00a0 00000004 c0124ffc ef9d1f6c 00000003 f73f0098
    00000000 f73f00a0 00000004 c01250f1 00000000 00000003 00000014 00000006
    Call Trace:
    [<c0124ffc>] do_wait+0xc9/0x299
    [<c01250f1>] do_wait+0x1be/0x299
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c012524d>] sys_wait4+0x81/0x94
    [<c0125273>] sys_waitpid+0x13/0x17
    [<c0102c19>] sysenter_do_call+0x12/0x25
    su S f6538410 0 7880 7879
    f5cfa2a0 00000086 f644fd90 f6538410 f6538564 c200f600 00000000 b7f64018
    f65391d0 00000000 f6538410 00000006 c0124ffc ef9d7f6c 00000003 f6538408
    00000000 f6538410 00000006 c01250f1 00000000 00000003 00000014 00000006
    Call Trace:
    [<c0124ffc>] do_wait+0xc9/0x299
    [<c01250f1>] do_wait+0x1be/0x299
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c012524d>] sys_wait4+0x81/0x94
    [<c0125273>] sys_waitpid+0x13/0x17
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    amdump S f65391d0 0 7881 7880
    f5cfae00 00000086 f6286390 f65391d0 f6539324 c200f600 00000000 080e4c78
    f73eead0 00000000 f65391d0 00000004 c0124ffc efa69f6c 00000003 f65391c8
    00000000 f65391d0 00000004 c01250f1 00000000 00000003 00000014 00000006
    Call Trace:
    [<c0124ffc>] do_wait+0xc9/0x299
    [<c01250f1>] do_wait+0x1be/0x299
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c012524d>] sys_wait4+0x81/0x94
    [<c0125273>] sys_waitpid+0x13/0x17
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    perl S f73eead0 0 7901 7881
    f6917ba0 00000086 00000008 f73eead0 f73eec24 c2017600 00000001 081c3cdc
    f65398b0 00000000 f73eead0 00000004 c0124ffc efd83f6c 00000003 f73eeac8
    00000000 f73eead0 00000004 c01250f1 00000000 00000003 00000014 00000006
    Call Trace:
    [<c0124ffc>] do_wait+0xc9/0x299
    [<c01250f1>] do_wait+0x1be/0x299
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c012524d>] sys_wait4+0x81/0x94
    [<c0125273>] sys_waitpid+0x13/0x17
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    driver S f65398b0 0 7903 7901
    f5c47060 00200086 f62c3190 f65398b0 f6539a04 c2017600 00000001 00200282
    c0133d05 f0287bc8 ffffffff 00200282 c0133dab 00200282 f0287bc8 00000000
    f0287e98 00000000 00000000 c0465a69 00000000 f66bdb48 00000000 00000000
    Call Trace:
    [<c0133d05>] lock_hrtimer_base+0x15/0x2f
    [<c0133dab>] hrtimer_try_to_cancel+0x55/0x5c
    [<c0465a69>] schedule_hrtimeout_range+0x31/0xf2
    [<c0131c33>] add_wait_queue+0x12/0x33
    [<c0416e08>] unix_poll+0x17/0x8b
    [<c016b83e>] do_sys_poll+0x253/0x323
    [<c016c23e>] __pollwait+0x0/0xac
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c01a55ac>] do_get_write_access+0x2fc/0x335
    [<c01987a4>] __ext3_get_inode_loc+0xcf/0x284
    [<c01a2a73>] __ext3_journal_dirty_metadata+0x13/0x32
    [<c019866f>] ext3_mark_iloc_dirty+0x27b/0x2e1
    [<c01989f0>] ext3_mark_inode_dirty+0x29/0x30
    [<c019f296>] __ext3_journal_stop+0x19/0x34
    [<c01755f2>] __mark_inode_dirty+0x21/0x12a
    [<c01a52a7>] journal_stop+0x144/0x14d
    [<c019f296>] __ext3_journal_stop+0x19/0x34
    [<c019a5f2>] ext3_ordered_write_end+0xf2/0x100
    [<c0143123>] generic_file_buffered_write+0x12c/0x254
    [<c014316b>] generic_file_buffered_write+0x174/0x254
    [<c0143873>] __generic_file_aio_write_nolock+0x448/0x4ab
    [<c0143b20>] generic_file_aio_write+0x58/0xa6
    [<c0197335>] ext3_file_write+0x19/0x83
    [<c016109a>] do_sync_write+0xbf/0x100
    [<c0110dde>] flush_tlb_page+0x50/0x54
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c0106cda>] read_tsc+0x6/0x22
    [<c01363a7>] getnstimeofday+0x4a/0xd0
    [<c012649c>] timespec_add_safe+0x1f/0x3e
    [<c0106cda>] read_tsc+0x6/0x22
    [<c01363a7>] getnstimeofday+0x4a/0xd0
    [<c016ba5a>] sys_poll+0x46/0x91
    [<c012673a>] sys_gettimeofday+0x27/0x53
    [<c0102c19>] sysenter_do_call+0x12/0x25
    taper S f6538780 0 7904 7903
    f5c476e0 00000082 c017f657 f6538780 f65388d4 c200f600 00000000 b728d000
    c0151c14 b728dbd8 b728d000 efde9e90 c0138b6d f5c47714 00000246 efde9e78
    00000000 c0707ac4 f5c47714 c0138df5 00002993 b728dbd8 00002993 00000000
    Call Trace:
    [<c017f657>] mpage_end_io_read+0x0/0x51
    [<c0151c14>] find_extend_vma+0x12/0x4d
    [<c0138b6d>] get_futex_key+0x6c/0x105
    [<c0138df5>] futex_wait+0x198/0x36a
    [<c011a807>] __wake_up+0x29/0x39
    [<c0138fe3>] wake_futex+0x1c/0x26
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c0139c63>] do_futex+0x68/0x6e3
    [<c0465313>] schedule+0x586/0x5d6
    [<c013a3af>] sys_futex+0xd1/0xe6
    [<c010165c>] sys_clone+0x28/0x2d
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    taper D f73ee3f0 0 10642 7903
    f5c476e0 00000082 f632b760 f73ee3f0 f73ee544 c2017600 00000001 0d81b6cb
    c10ffdc0 35c2cb94 00000000 00020000 00000003 f632b6c0 c1afc8c0 c2017600
    c1afc8c0 c2005794 ec189e54 c0465577 ec189e4c c0142a79 c0143c42 c046567c
    Call Trace:
    [<c0465577>] io_schedule+0x1a/0x23
    [<c0142a79>] sync_page+0x3e/0x41
    [<c0143c42>] sync_page_killable+0x5/0x2a
    [<c046567c>] __wait_on_bit_lock+0x31/0x6b
    [<c0143c3d>] sync_page_killable+0x0/0x2a
    [<c01429d9>] __lock_page_killable+0x51/0x57
    [<c0131a4a>] wake_bit_function+0x0/0x3c
    [<c014403b>] generic_file_aio_read+0x3ae/0x54f
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c013929e>] futex_wake+0xc2/0xcc
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03029a9>] scsi_io_completion+0x174/0x35e
    [<c030223e>] scsi_device_unbusy+0x61/0x78
    [<c01610db>] do_sync_read+0x0/0xfe
    [<c016188c>] vfs_read+0x81/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    taper S f73f0410 0 10643 7903
    f5c476e0 00000082 c0480f00 f73f0410 f73f0564 c2017600 00000001 b6900000
    c0151c14 b690046c b6900000 eda3fe90 c0138b6d f5c47714 00000246 eda3fe78
    00000000 c070786c f5c47714 c0138df5 00acfd31 b690046c 00acfd31 00000000
    Call Trace:
    [<c0151c14>] find_extend_vma+0x12/0x4d
    [<c0138b6d>] get_futex_key+0x6c/0x105
    [<c0138df5>] futex_wait+0x198/0x36a
    [<c0143b20>] generic_file_aio_write+0x58/0xa6
    [<c016109a>] do_sync_write+0xbf/0x100
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c0139c63>] do_futex+0x68/0x6e3
    [<c032b64f>] tw_interrupt+0x17/0x6b6
    [<c013a3af>] sys_futex+0xd1/0xe6
    [<c0161bbf>] sys_write+0x3c/0x63
    [<c0161bde>] sys_write+0x5b/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    dumper R running 0 7905 7903
    f5c47d60 00000086 e86b3500 f73dd890 f73dd9e8 c2017600 00000001 c03f7219
    ec3cb834 e86b3520 f03b20a0 e86b3500 f03b20a0 00000000 c03fb9d5 f03b20a0
    00000000 f03b20cc 00000000 c011fd15 c0465390 c03cce67 f03b20a0 f73dd890
    Call Trace:
    [<c03f7219>] tcp_rcv_established+0x478/0x4af
    [<c03fb9d5>] tcp_v4_do_rcv+0x1f/0x159
    [<c011fd15>] __cond_resched+0x13/0x2f
    [<c0465390>] cond_resched_softirq+0x2d/0x3b
    [<c03cce67>] release_sock+0x3e/0x84
    [<c03f0679>] tcp_recvmsg+0x2c3/0x6ce
    [<c03cc96b>] sock_common_recvmsg+0x2f/0x45
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c0104c2a>] profile_pc+0x24/0x4e
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c0106cda>] read_tsc+0x6/0x22
    [<c01363a7>] getnstimeofday+0x4a/0xd0
    [<c012649c>] timespec_add_safe+0x1f/0x3e
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    dumper S f73f0780 0 7906 7903
    f5cfa920 00000082 f77da460 f73f0780 f73f08d4 c200f600 00000000 00000000
    f6c76ac0 f6c27828 f6c27828 00000000 f6c27828 f6c39fa8 c01a55ac 7fffffff
    00000000 f6039e54 f73be200 c04655a2 f6916600 f77da460 00000246 c0131b8d
    Call Trace:
    [<c01a55ac>] do_get_write_access+0x2fc/0x335
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    dumper S f73dcad0 0 7907 7903
    f5cfaac0 00000086 f77da460 f73dcad0 f73dcc24 c2017600 00000001 00000000
    f6c76ac0 f6ef8a20 f6ef8a20 00000000 f6ef8a20 e80c94e4 c01a55ac 7fffffff
    00000000 f66f7e54 efb01e40 c04655a2 f6916600 f77da460 00000246 c0131b8d
    Call Trace:
    [<c01a55ac>] do_get_write_access+0x2fc/0x335
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    dumper S f64491f0 0 7908 7903
    f6917d40 00000082 f77da460 f64491f0 f6449344 c200f600 00000000 00000000
    f6c76ac0 f6ef8a20 f6ef8a20 00000000 f6ef8a20 e80c9928 c01a55ac 7fffffff
    00000000 efaf9e54 efb01ac0 c04655a2 f6916600 f77da460 00000246 c0131b8d
    Call Trace:
    [<c01a55ac>] do_get_write_access+0x2fc/0x335
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c014f862>] handle_mm_fault+0x4a0/0x4ee
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    dumper S f6449560 0 7909 7903
    f6917380 00000082 c05e9340 f6449560 f64496b4 c200f600 00000000 00000000
    f6c76ac0 f6ef8a20 f6ef8a20 00000000 f6ef8a20 e80c9c9c c01a55ac 7fffffff
    00000000 efc7fe54 efb01740 c04655a2 f6916600 f77da460 00000246 c0131b8d
    Call Trace:
    [<c01a55ac>] do_get_write_access+0x2fc/0x335
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    dumper S f65d1bc0 0 7910 7903
    f6917860 00000086 f77da460 f65d1bc0 f65d1d14 c2017600 00000001 00000000
    f6c76ac0 f6c27828 f6c27828 00000000 f6c27828 e80c9274 c01a55ac 7fffffff
    00000000 efc7be54 efb013c0 c04655a2 f6916600 f77da460 00000246 c0131b8d
    Call Trace:
    [<c01a55ac>] do_get_write_access+0x2fc/0x335
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    dumper S f65d1170 0 7911 7903
    f69171e0 00000082 00000000 f65d1170 f65d12c4 c2017600 00000001 000000cb
    00000000 00000000 c014316b 00000000 efb03ee4 f63a0ce0 f6fc7d64 7fffffff
    00000000 efb03e54 efb01040 c04655a2 000000cb c04796a0 00000246 c0131b8d
    Call Trace:
    [<c014316b>] generic_file_buffered_write+0x174/0x254
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c014f629>] handle_mm_fault+0x267/0x4ee
    [<c01341be>] hrtimer_run_pending+0x1a/0x94
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    dumper S f65d0a90 0 7912 7903
    f6917520 00000082 00000000 f65d0a90 f65d0be4 c2017600 00000001 000000cb
    00000000 00000000 c014316b 00000000 efb0bee4 f631a7e0 f6fc77d0 7fffffff
    00000000 efb0be54 f027bc80 c04655a2 000000cb c04796a0 00000246 c0131b8d
    Call Trace:
    [<c014316b>] generic_file_buffered_write+0x174/0x254
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c014f629>] handle_mm_fault+0x267/0x4ee
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    dumper S f65d1850 0 7913 7903
    f5cfa440 00000082 00000000 f65d1850 f65d19a4 c2017600 00000001 000000cb
    00000000 00000000 c014316b 00000000 efe0bee4 efb27920 f6fc7b88 7fffffff
    00000000 efe0be54 f027b900 c04655a2 000000cb c04796a0 00000246 c0131b8d
    Call Trace:
    [<c014316b>] generic_file_buffered_write+0x174/0x254
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c014f629>] handle_mm_fault+0x267/0x4ee
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    dumper S f65d0040 0 7914 7903
    f6917040 00000082 00000000 f65d0040 f65d0194 c200f600 00000000 000000cb
    00000000 00000000 c014316b 00000000 efe17ee4 efe852e0 f6fbaac8 7fffffff
    00000000 efe17e54 f027b580 c04655a2 000000cb c04796a0 00000246 c0131b8d
    Call Trace:
    [<c014316b>] generic_file_buffered_write+0x174/0x254
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c014f629>] handle_mm_fault+0x267/0x4ee
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    dumper S f707cad0 0 7915 7903
    f6917a00 00000086 00000000 f707cad0 f707cc24 c2017600 00000001 000000cb
    00000000 00000000 c014316b 00000000 efecbee4 f02159e0 f6fc75f4 7fffffff
    00000000 efecbe54 f027b200 c04655a2 000000cb c04796a0 00000246 c0131b8d
    Call Trace:
    [<c014316b>] generic_file_buffered_write+0x174/0x254
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c014f629>] handle_mm_fault+0x267/0x4ee
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    dumper S f73767a0 0 7916 7903
    f5ccb280 00000082 00000000 f73767a0 f73768f4 c2017600 00000001 000000cb
    00000000 00000000 c014316b 00000000 f66cdee4 efb274a0 f6fc7418 7fffffff
    00000000 f66cde54 efa77e40 c04655a2 000000cb c04796a0 00000246 c0131b8d
    Call Trace:
    [<c014316b>] generic_file_buffered_write+0x174/0x254
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c014f629>] handle_mm_fault+0x267/0x4ee
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    dumper S f62c2e20 0 7917 7903
    f5c0b0c0 00000086 00000000 f62c2e20 f62c2f74 c200f600 00000000 000000cb
    00000000 00000000 c014316b 00000000 efe1bee4 f69238a0 f6e7c3f8 7fffffff
    00000000 efe1be54 efa77ac0 c04655a2 000000cb c04796a0 00000246 c0131b8d
    Call Trace:
    [<c014316b>] generic_file_buffered_write+0x174/0x254
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c014f629>] handle_mm_fault+0x267/0x4ee
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    dumper S f6303bc0 0 7918 7903
    f5c0b400 00000086 00000000 f6303bc0 f6303d14 c200f600 00000000 000000cb
    00000000 00000000 c014316b 00000000 efebbee4 efb213e0 f6e7c21c 7fffffff
    00000000 efebbe54 efa77740 c04655a2 000000cb c04796a0 00000246 c0131b8d
    Call Trace:
    [<c014316b>] generic_file_buffered_write+0x174/0x254
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c014f629>] handle_mm_fault+0x267/0x4ee
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    dumper S f6302720 0 7919 7903
    f5c0b5a0 00000086 00000000 f6302720 f6302874 c2017600 00000001 000000cb
    00000000 00000000 c014316b 00000000 efeb5ee4 f689f6e0 f6fbae80 7fffffff
    00000000 efeb5e54 efa773c0 c04655a2 000000cb c04796a0 00000246 c0131b8d
    Call Trace:
    [<c014316b>] generic_file_buffered_write+0x174/0x254
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c014f629>] handle_mm_fault+0x267/0x4ee
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    dumper S f6303850 0 7920 7903
    f62b46c0 00000086 00000000 f6303850 f63039a4 c200f600 00000000 000000cb
    00000000 00000000 c014316b 00000000 ef98bee4 efb21960 f6fbaca4 7fffffff
    00000000 ef98be54 efa77040 c04655a2 000000cb c04796a0 00000246 c0131b8d
    Call Trace:
    [<c014316b>] generic_file_buffered_write+0x174/0x254
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c03cacd3>] sock_ioctl+0x0/0x1c0
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016abe2>] do_vfs_ioctl+0x3b4/0x3fe
    [<c014f629>] handle_mm_fault+0x267/0x4ee
    [<c01341be>] hrtimer_run_pending+0x1a/0x94
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    chg-glue S f63034e0 0 7922 7904
    f5c0b260 00200086 00200096 f63034e0 f6303634 c200f600 00000000 f64d1df8
    00000001 00000000 c0131a2a 00000000 efde9e70 00200246 c0131b8d f64d1200
    00001000 f0269e84 f0269fa0 c0166535 00000000 f63034e0 c0131a1d f64d1204
    Call Trace:
    [<c0131a2a>] autoremove_wake_function+0xd/0x2d
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0166535>] pipe_wait+0x51/0x6f
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c0166c0a>] pipe_read+0x2bf/0x329
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c0110dde>] flush_tlb_page+0x50/0x54
    [<c0106cda>] read_tsc+0x6/0x22
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c011dcd6>] run_rebalance_domains+0x80/0x3af
    [<c01341be>] hrtimer_run_pending+0x1a/0x94
    [<c01610db>] do_sync_read+0x0/0xfe
    [<c016188c>] vfs_read+0x81/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    chg-glue S f6538e60 0 7927 7904
    f5c0b260 00200086 f703e720 f6538e60 f6538fb4 c2017600 00000001 f5c0b260
    00030001 00000000 effa1f68 c0467fb1 00000001 00200246 c0131b8d f630e000
    00000014 effa1e84 effa1fa0 c0166535 00000000 f6538e60 c0131a1d f630e004
    Call Trace:
    [<c0467fb1>] do_page_fault+0x0/0x5c3
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0166535>] pipe_wait+0x51/0x6f
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c0166c0a>] pipe_read+0x2bf/0x329
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c014f862>] handle_mm_fault+0x4a0/0x4ee
    [<c01610db>] do_sync_read+0x0/0xfe
    [<c016188c>] vfs_read+0x81/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    sshd S f634eea0 0 10327 7539
    f5c37240 00000082 c0646580 f634eea0 f634eff4 c2017600 00000001 c0646580
    00000011 0c120349 1401a8c0 e870b680 a0392279 0000000c e81f2cf0 7fffffff
    00000000 ec1b5e54 f63b0200 c04655a2 00000000 c03ce52e 00000246 c0131b8d
    Call Trace:
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c03ce52e>] sock_def_readable+0xf/0x58
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0416956>] unix_stream_recvmsg+0x1e0/0x428
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03029a9>] scsi_io_completion+0x174/0x35e
    [<c030223e>] scsi_device_unbusy+0x61/0x78
    [<c03d642c>] net_rx_action+0x5c/0x10f
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    sshd S f634f8f0 0 10329 10327
    f5c473a0 00000086 f73fbc60 f634f8f0 f634fa44 c2017600 00000001 c0126ad0
    00000001 00000046 00000049 c0646580 00000046 00000046 00000003 00000000
    0000000b 00000000 00000000 c0465a69 00000000 00000000 00000000 00000003
    Call Trace:
    [<c0126ad0>] __do_softirq+0x6f/0x10f
    [<c0465a69>] schedule_hrtimeout_range+0x31/0xf2
    [<c023cc48>] tty_ldisc_deref+0x4c/0x5b
    [<c023b5ee>] n_tty_poll+0x0/0x123
    [<c02378f9>] tty_poll+0x55/0x62
    [<c016be74>] do_select+0x390/0x407
    [<c016c23e>] __pollwait+0x0/0xac
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c03ea771>] ip_queue_xmit+0x24e/0x291
    [<c03d4c69>] dev_hard_start_xmit+0x1e8/0x246
    [<c03d6e7e>] dev_queue_xmit+0x394/0x3c6
    [<c03ea160>] ip_finish_output+0x1bc/0x1f3
    [<c03e8e17>] ip_local_out+0x15/0x17
    [<c03ea771>] ip_queue_xmit+0x24e/0x291
    [<c03f805c>] tcp_transmit_skb+0x56c/0x59f
    [<c012a06d>] lock_timer_base+0x19/0x35
    [<c012a174>] __mod_timer+0x8d/0x97
    [<c03ccf62>] sk_reset_timer+0xc/0x16
    [<c03f9606>] __tcp_push_pending_frames+0x630/0x6cb
    [<c046674d>] _spin_lock_bh+0x8/0x1e
    [<c03cce38>] release_sock+0xf/0x84
    [<c03f02b4>] tcp_sendmsg+0x803/0x905
    [<c016c18e>] core_sys_select+0x2a3/0x2c4
    [<c03caaec>] sock_aio_write+0xb3/0xc0
    [<c011c8f0>] try_to_wake_up+0x135/0x13e
    [<c016109a>] do_sync_write+0xbf/0x100
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c02396ac>] tty_read+0x78/0x9a
    [<c016c21d>] sys_select+0x6e/0x8f
    [<c0161bbf>] sys_write+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    bash S f634e450 0 10330 10329
    f645f220 00000082 00005410 f634e450 f634e5a4 c200f600 00000000 bf953ffc
    f73faea0 00000000 f634e450 0000000e c0124ffc eb5ddf6c 00000003 f634e448
    00000000 f634e450 0000000e c01250f1 00000000 00000003 00000014 0000000e
    Call Trace:
    [<c0124ffc>] do_wait+0xc9/0x299
    [<c01250f1>] do_wait+0x1be/0x299
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c012524d>] sys_wait4+0x81/0x94
    [<c0125273>] sys_waitpid+0x13/0x17
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    pdflush S f73800e0 0 10619 2
    f5c476e0 00000046 c01486f9 f73800e0 f7380234 c200f600 00000000 00000000
    00000400 00000000 00000000 00000000 00000000 00000000 00000021 f0b03fc4
    c01486f9 00000000 00000000 c01487b7 f73800e0 00000000 00000000 f66f3fc4
    Call Trace:
    [<c01486f9>] pdflush+0x0/0x1aa
    [<c01486f9>] pdflush+0x0/0x1aa
    [<c01487b7>] pdflush+0xbe/0x1aa
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    chunker S f62c2060 0 10640 7903
    f62b4a00 00000086 00000020 f62c2060 f62c21b4 c2017600 00000001 00000000
    c017a283 00001000 00001000 28df7000 00000050 e8094e20 efc7c120 7fffffff
    f66904c0 ea29be08 ea29be38 c04655a2 28df7000 00000050 c046674d f66904c0
    Call Trace:
    [<c017a283>] nobh_write_end+0x108/0x119
    [<c04655a2>] schedule_timeout+0x13/0x86
    [<c046674d>] _spin_lock_bh+0x8/0x1e
    [<c03cce38>] release_sock+0xf/0x84
    [<c03cd6cd>] sk_wait_data+0x64/0x98
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c03f068f>] tcp_recvmsg+0x2d9/0x6ce
    [<c0143873>] __generic_file_aio_write_nolock+0x448/0x4ab
    [<c03cc96b>] sock_common_recvmsg+0x2f/0x45
    [<c03cabb2>] sock_aio_read+0xb9/0xc7
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c016189d>] vfs_read+0x92/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    gzip S f6591580 0 10641 7905
    f5c88d60 00000086 f73dd890 f6591580 f65916d4 c2017600 00000001 00000246
    f5d98a00 00000680 f5d98a00 00000020 c034661e 00000246 c0131b8d f0019600
    00003c00 eb44fe84 eb44ffa0 c0166535 00000000 f6591580 c0131a1d f0019604
    Call Trace:
    [<c034661e>] mv_interrupt+0x15/0x5a1
    [<c0131b8d>] prepare_to_wait+0x12/0x43
    [<c0166535>] pipe_wait+0x51/0x6f
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c0166c0a>] pipe_read+0x2bf/0x329
    [<c016119a>] do_sync_read+0xbf/0xfe
    [<c0131a1d>] autoremove_wake_function+0x0/0x2d
    [<c032b64f>] tw_interrupt+0x17/0x6b6
    [<c011a62a>] sched_slice+0x49/0x4f
    [<c01610db>] do_sync_read+0x0/0xfe
    [<c016188c>] vfs_read+0x81/0xf4
    [<c0161b5c>] sys_read+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    pdflush S f68040c0 0 11475 2
    f62b4a00 00000046 c01486f9 f68040c0 f6804214 c200f600 00000000 00000000
    f66f3fa8 00000400 00000000 00000000 00000000 00000000 00000000 f66f3fc4
    c01486f9 00000000 00000000 c01487b7 f68040c0 00000000 00000000 c05f1394
    Call Trace:
    [<c01486f9>] pdflush+0x0/0x1aa
    [<c01486f9>] pdflush+0x0/0x1aa
    [<c01487b7>] pdflush+0xbe/0x1aa
    [<c013195f>] kthread+0x38/0x5d
    [<c0131927>] kthread+0x0/0x5d
    [<c010399b>] kernel_thread_helper+0x7/0x10
    su S f73faea0 0 11501 10330
    f645f8a0 00000086 00000008 f73faea0 f73faff4 c200f600 00000000 b7fa1018
    f73fbc60 00000000 f73faea0 00000006 c0124ffc ec273f6c 00000003 f73fae98
    00000000 f73faea0 00000006 c01250f1 00000000 00000003 00000014 00000006
    Call Trace:
    [<c0124ffc>] do_wait+0xc9/0x299
    [<c01250f1>] do_wait+0x1be/0x299
    [<c011c8f9>] default_wake_function+0x0/0x8
    [<c012524d>] sys_wait4+0x81/0x94
    [<c0125273>] sys_waitpid+0x13/0x17
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    bash R running 0 11502 11501
    00000000 c0558b39 00000000 ffffffff f73fbdb4 c2017600 c057a5aa c0573033
    00000000 00000096 f6201c74 f6913ce8 00000001 00000000 c0119e5c 7fffffff
    f6392400 7fffffff f63924e4 00000001 f63924f0 f63924f0 00000000 00000046
    Call Trace:
    [<c0119e5c>] __wake_up_common+0x2e/0x58
    [<c0134694>] up+0x9/0x2a
    [<c0123621>] release_console_sem+0x169/0x182
    [<c0123621>] release_console_sem+0x169/0x182
    [<c0123621>] release_console_sem+0x169/0x182
    [<c0123621>] release_console_sem+0x169/0x182
    [<c0123621>] release_console_sem+0x169/0x182
    [<c01047ba>] print_trace_address+0x0/0x21
    [<c0123a8e>] printk+0x14/0x18
    [<c0104792>] printk_address+0x1c/0x20
    [<c01300bb>] __kernel_text_address+0x18/0x23
    [<c01300bb>] __kernel_text_address+0x18/0x23
    [<c0104854>] dump_trace+0x79/0xc4
    [<c01048cd>] show_trace_log_lvl+0x2e/0x36
    [<c0104963>] show_stack_log_lvl+0x8e/0x96
    [<c0104b14>] show_stack+0x10/0x13
    [<c0120305>] show_state_filter+0x39/0x78
    [<c02475eb>] __handle_sysrq+0x83/0xfa
    [<c0247662>] write_sysrq_trigger+0x0/0x29
    [<c0247685>] write_sysrq_trigger+0x23/0x29
    [<c018b196>] proc_reg_write+0x56/0x69
    [<c018b140>] proc_reg_write+0x0/0x69
    [<c0161798>] vfs_write+0x83/0xf6
    [<c0161bbf>] sys_write+0x3c/0x63
    [<c0102c19>] sysenter_do_call+0x12/0x25
    [<c0460000>] init_intel+0x50/0x25f
    Sched Debug Version: v0.07, 2.6.28-gentoo-r2 #1
    now at 67222524.346315 msecs
    .sysctl_sched_latency : 40.000000
    .sysctl_sched_min_granularity : 8.000000
    .sysctl_sched_wakeup_granularity : 10.000000
    .sysctl_sched_child_runs_first : 0.000001
    .sysctl_sched_features : 24191

    cpu#0, 1132.695 MHz
    .nr_running : 1
    .load : 3121
    .nr_switches : 60351160
    .nr_load_updates : 16803869
    .nr_uninterruptible : 7197
    .jiffies : 16728869
    .next_balance : 16.728894
    .curr->pid : 305
    .clock : 67222521.586110
    .cpu_load[0] : 3121
    .cpu_load[1] : 3121
    .cpu_load[2] : 3121
    .cpu_load[3] : 3113
    .cpu_load[4] : 2955

    cfs_rq[0]:
    .exec_clock : 0.000000
    .MIN_vruntime : 0.000001
    .min_vruntime : 21812758.288813
    .max_vruntime : 0.000001
    .spread : 0.000000
    .spread0 : 0.000000
    .nr_running : 1
    .load : 3121
    .nr_spread_over : 0

    rt_rq[0]:
    .rt_nr_running : 0
    .rt_throttled : 0
    .rt_time : 0.000000
    .rt_runtime : 950.000000

    runnable tasks:
    task PID tree-key switches prio exec-runtime sum-exec sum-sleep
    ----------------------------------------------------------------------------------------------------------
    R jfsCommit 305 21812758.288813 469483 115 0 0 0.000000 0.000000 0.000000

    cpu#1, 1132.695 MHz
    .nr_running : 2
    .load : 2048
    .nr_switches : 61792074
    .nr_load_updates : 16803824
    .nr_uninterruptible : 4294960100
    .jiffies : 16728869
    .next_balance : 16.728865
    .curr->pid : 11502
    .clock : 67222509.893694
    .cpu_load[0] : 1024
    .cpu_load[1] : 807
    .cpu_load[2] : 922
    .cpu_load[3] : 966
    .cpu_load[4] : 1092

    cfs_rq[1]:
    .exec_clock : 0.000000
    .MIN_vruntime : 23378434.680689
    .min_vruntime : 23378434.625886
    .max_vruntime : 23378434.680689
    .spread : 0.000000
    .spread0 : 1565676.337073
    .nr_running : 2
    .load : 2048
    .nr_spread_over : 0

    rt_rq[1]:
    .rt_nr_running : 0
    .rt_throttled : 0
    .rt_time : 0.000000
    .rt_runtime : 950.000000

    runnable tasks:
    task PID tree-key switches prio exec-runtime sum-exec sum-sleep
    ----------------------------------------------------------------------------------------------------------
    dumper 7905 23378434.680689 26937303 120 0 0 0.000000 0.000000 0.000000
    R bash 11502 23378394.740313 17 120 0 0 0.000000 0.000000 0.000000

     

  • rwk

    rwk - 2009-03-12

    Can you tell me what the prospects are for fixing this?
    I am stuck at the moment unless I abandon JFS for an
    alternative filesystem. I would prefer to keep using JFS
    but need to know what king of time frame is likely for
    a fix.

    Is there any additional feedback I can provide?

    Thanks!

     

  • houkouonchi

    houkouonchi - 2009-03-21

    I actually just posted about this same issue on jfs-discussion as I didn't see anything when I did a search. I just thought I would add I have this exact same problem and run into it a lot quicker (usually after writing files that are getting 30+GB). A guess of a possible cause is fragmentation since it seems to do it worse on files that are written and are more fragmented.

    Also maybe the length of time/size of write required before this issue surfaces has to do with the write speed? I know I can writes files at 600-700 MB/sec but it seemed to not creep up as often/soon when writing at much slower speeds.

    I really have no clue what causes it so I am just guessing here.

     
MongoDB Logo MongoDB