Re: [SSI-devel] SSI-1.1.1-FC2 supplied kernel oops
Brought to you by:
brucewalker,
rogertsang
From: David B Z. <dav...@hp...> - 2004-11-29 17:33:21
|
This doesn't LOOK at all SSI related. I can't comment on the DRBD device configuration issues. David On Nov 28, 2004, at 2:27 AM, Roger Tsang wrote: > Hi guys. I am getting repeatable kernel oops during the updatedb and > disk > IO stuff that happens during cron.daily. I thought it was my bad > kernel, > so I reverted back to the original SSI-1.1.1-FC2 kernel. To my > surprise > I'm still getting kernel oops. Consequently my SSI NFS server stuck > in IO > forever and I had to reboot the initnode. > > I did a forced fsck on /dev/drbd/0 and /dev/drbd/1 revealing no ext3 > filesystem problems. I wonder what is this kernel attempt to access > beyond end of device was the cause of all these kernel oops. Could it > be > because my underlying physical device is not big enough for drbd and > drbd > metadata has overlapped the end of my drbd device? I checked the > difference in the number of 4K blocks between the drbd device and its > underlying physical device to be 32768 which works out to 128MB. > > Nov 28 04:02:12 node1 syslogd 1.4.1: restart. > Nov 28 04:02:12 node1 logrotate: ALERT exited abnormally with [1] > Nov 28 04:04:51 node1 kernel: <6>attempt to access beyond end of > device > Nov 28 04:04:51 node1 kernel: 93:00: rw=0, want=820404740, > limit=71550956 > Nov 28 04:04:51 node1 kernel: EXT3-fs error (device drbd1(147,0)): > ext3_readdir: directory #5652542 contains a hole at offset 0 > Nov 28 04:04:51 node1 kernel: Unable to handle kernel NULL pointer > dereference at virtual address 0000003a > Nov 28 04:04:51 node1 kernel: printing eip: > Nov 28 04:04:51 node1 kernel: c019d146 > Nov 28 04:04:51 node1 kernel: *pde = 0a52f001 > Nov 28 04:04:51 node1 kernel: *pte = 0a517067 > Nov 28 04:04:51 node1 kernel: Oops: 0000 > Nov 28 04:04:51 node1 kernel: r128 agpgart nfsd autofs4 ipt_REJECT > ipt_multiport ipt_state ip_conntrack iptable_filter ip_tables ide-cd > sr_mod cdrom floppy dm-mod keybdev mousedev hid inpu > Nov 28 04:04:51 node1 kernel: CPU: 0 > Nov 28 04:04:51 node1 kernel: EIP: 0060:[<c019d146>] Not tainted > Nov 28 04:04:51 node1 kernel: EFLAGS: 00013202 > Nov 28 04:04:52 node1 kernel: > Nov 28 04:04:52 node1 kernel: EIP is at ext3_handle_error [kernel] 0x26 > (2.4.22-1.2199.nptl_ssi_5develsmp) > Nov 28 04:04:52 node1 kernel: eax: 00000002 ebx: f6e99c00 ecx: > 00000001 edx: f5942000 > Nov 28 04:04:52 node1 kernel: esi: 00000000 edi: f358aa80 ebp: > e8719a40 esp: e8719a2c > Nov 28 04:04:52 node1 kernel: ds: 0068 es: 0068 ss: 0068 > Nov 28 04:04:52 node1 kernel: Process nfsd (pid: 68336, > stackpage=e8719000) > Nov 28 04:04:52 node1 kernel: Stack: c03e89a5 e8719a58 f6e99c00 > f6e99c00 > 00000000 e8719a5c c019d24a f6e99c00 > Nov 28 04:04:52 node1 kernel: c06e6600 c03e42bb c06e7820 > e8719b18 > e8719afc c0192936 f6e99c00 c03e42bb > Nov 28 04:04:52 node1 kernel: c03e83b0 0056403e 00000000 > c07259c0 > 00000000 e55cd59c 00000001 00000000 > Nov 28 04:04:52 node1 kernel: Call Trace: > Nov 28 04:04:52 node1 kernel: [<c019d24a>] ext3_error [kernel] 0x5a > (0xe8719a44)Nov 28 04:04:53 node1 kernel: [<c0192936>] ext3_readdir > [kernel] 0x3d6 (0xe8719a60) > Nov 28 04:04:53 node1 kernel: [<c01222d8>] recalc_task_prio [kernel] > 0xa8 > (0xe8719aa0) > Nov 28 04:04:53 node1 kernel: [<c025363e>] cfsd_open [kernel] 0xce > (0xe8719ae0) > Nov 28 04:04:53 node1 kernel: [<c024c530>] cfs_local_readdir [kernel] > 0xc0 > (0xe8719b00) > Nov 28 04:04:53 node1 kernel: [<f8d30400>] filldir_one [nfsd] 0x0 > (0xe8719b0c) > Nov 28 04:04:53 node1 kernel: [<c024cad3>] cfs_readdir [kernel] 0x553 > (0xe8719b98) > Nov 28 04:04:54 node1 kernel: [<f8d30400>] filldir_one [nfsd] 0x0 > (0xe8719ba4) > Nov 28 04:04:54 node1 kernel: [<c0256f94>] hfind [kernel] 0x14 > (0xe8719bf8) > Nov 28 04:04:54 node1 kernel: [<c0257088>] svrtok_lookup [kernel] 0x78 > (0xe8719c18) > Nov 28 04:04:54 node1 kernel: [<c019ab80>] ext3_lookup [kernel] 0x110 > (0xe8719c20) > Nov 28 04:04:55 node1 kernel: [<c016d49d>] vfs_readdir [kernel] 0xad > (0xe8719c4c) > Nov 28 04:04:55 node1 kernel: [<f8d30400>] filldir_one [nfsd] 0x0 > (0xe8719c58) > Nov 28 04:04:56 node1 kernel: [<f8d30400>] filldir_one [nfsd] 0x0 > (0xe8719c68) > Nov 28 04:04:56 node1 kernel: [<f8d30600>] nfsd_get_name [nfsd] 0x190 > (0xe8719c70) > Nov 28 04:04:56 node1 kernel: [<f8d30400>] filldir_one [nfsd] 0x0 > (0xe8719c78) > Nov 28 04:04:57 node1 kernel: [<c024745c>] cfs_hpget [kernel] 0xec > (0xe8719c90) > Nov 28 04:04:57 node1 kernel: [<f8d30b90>] splice [nfsd] 0x30 > (0xe8719d48) > Nov 28 04:04:58 node1 kernel: [<c02481e5>] cfs_fh_to_dentry [kernel] > 0xe5 > (0xe8719dbc) > Nov 28 04:04:58 node1 kernel: [<c025a4f7>] cfstok_relse [kernel] 0x47 > (0xe8719dd4) > Nov 28 04:04:58 node1 kernel: [<c024cff5>] cfs_lookup [kernel] 0x125 > (0xe8719e00) > Nov 28 04:04:59 node1 kernel: [<c0172d01>] dput [kernel] 0x31 > (0xe8719e3c) > Nov 28 04:04:59 node1 kernel: [<f8d30abd>] nfsd_findparent [nfsd] 0xbd > (0xe8719e54) > Nov 28 04:04:59 node1 kernel: [<f8d3defc>] .rodata.str1.1 [nfsd] 0x160 > (0xe8719e60) > Nov 28 04:05:00 node1 kernel: [<f8d30ed7>] find_fh_dentry [nfsd] 0x1d7 > (0xe8719e80) > Nov 28 04:05:00 node1 kernel: [<f8d31269>] fh_verify [nfsd] 0x199 > (0xe8719eb4) > Nov 28 04:05:00 node1 kernel: [<c039dd6b>] svc_sock_enqueue [kernel] > 0x13b > (0xe8719ef0) > Nov 28 04:05:00 node1 kernel: [<f8d396aa>] nfsd3_proc_getattr [nfsd] > 0x6a > (0xe8719f10) > Nov 28 04:05:01 node1 kernel: [<f8d41804>] nfsd_procedures3 [nfsd] 0x24 > (0xe8719f3c) > Nov 28 04:05:01 node1 kernel: [<f8d2e639>] nfsd_dispatch [nfsd] 0xc9 > (0xe8719f48) > Nov 28 04:05:02 node1 kernel: [<f8d2e570>] nfsd_dispatch [nfsd] 0x0 > (0xe8719f60)Nov 28 04:05:02 node1 kernel: [<c039da25>] svc_process > [kernel] 0x355 (0xe8719f68) > Nov 28 04:05:02 node1 kernel: [<f8d41804>] nfsd_procedures3 [nfsd] 0x24 > (0xe8719f8c) > Nov 28 04:05:03 node1 kernel: [<f8d41118>] nfsd_version3 [nfsd] 0x0 > (0xe8719f90)Nov 28 04:05:03 node1 kernel: [<f8d41138>] nfsd_program > [nfsd] > 0x0 (0xe8719f94) > Nov 28 04:05:03 node1 kernel: [<f8d2e41b>] nfsd [nfsd] 0x1fb > (0xe8719fb0) > Nov 28 04:05:04 node1 kernel: [<f8d2e220>] nfsd [nfsd] 0x0 (0xe8719fe0) > Nov 28 04:05:04 node1 kernel: [<c01077ed>] kernel_thread_helper > [kernel] > 0x5 (0xe8719ff0) > Nov 28 04:05:04 node1 kernel: > Nov 28 04:05:04 node1 kernel: Code: 0f b7 46 3a 83 c8 02 66 89 46 3a > f6 43 > 34 01 75 53 89 1c 24 > Nov 28 04:05:34 node1 kernel: kernel BUG in header file at line 325 > Nov 28 04:05:34 node1 kernel: ------------[ cut here ]------------ > Nov 28 04:05:34 node1 kernel: kernel BUG at panic.c:297! > Nov 28 04:05:34 node1 kernel: invalid operand: 0000 > Nov 28 04:05:34 node1 kernel: r128 agpgart nfsd autofs4 ipt_REJECT > ipt_multiport ipt_state ip_conntrack iptable_filter ip_tables ide-cd > sr_mod cdrom floppy dm-mod keybdev mousedev hid inpu > Nov 28 04:05:34 node1 kernel: CPU: 0 > Nov 28 04:05:34 node1 kernel: EIP: 0060:[<c0128a39>] Not tainted > Nov 28 04:05:34 node1 kernel: EFLAGS: 00010286 > Nov 28 04:05:34 node1 kernel: > Nov 28 04:05:34 node1 kernel: EIP is at __out_of_line_bug [kernel] 0x19 > (2.4.22-1.2199.nptl_ssi_5develsmp) > Nov 28 04:05:34 node1 kernel: eax: 00000026 ebx: d17e6e80 ecx: > 00000000 edx: c066d980 > Nov 28 04:05:34 node1 kernel: esi: d17e6bbc edi: d17e6ee4 ebp: > cd12dc90 esp: cd12dc88 > Nov 28 04:05:34 node1 kernel: ds: 0068 es: 0068 ss: 0068 > Nov 28 04:05:34 node1 kernel: Process updatedb (pid: 74835, > stackpage=cd12d000) > Nov 28 04:05:34 node1 kernel: Stack: c03e1f1c 00000145 cd12dcb0 > c0173859 > 00000145 000001f0 d17e6bbc fffffff4 > Nov 28 04:05:34 node1 kernel: e5df6c80 e763a080 cd12dcd4 > c0168298 > e5df6c80 d17e6bbc 00000000 00000000 > Nov 28 04:05:34 node1 kernel: dfbb4dc0 f5a67a80 e5df6c80 > cd12dce8 > c016836e d17e6bbc e5df6c80 00000000 > Nov 28 04:05:34 node1 kernel: Call Trace: > Nov 28 04:05:34 node1 kernel: [<c0173859>] d_alloc [kernel] 0x199 > (0xcd12dc94) > Nov 28 04:05:34 node1 kernel: [<c0168298>] lookup_hash_it [kernel] 0x88 > (0xcd12dcb4) > Nov 28 04:05:34 node1 kernel: [<c016836e>] lookup_hash [kernel] 0x1e > (0xcd12dcd8) > Nov 28 04:05:34 node1 kernel: [<c0253044>] cfsd_lookup [kernel] 0x54 > (0xcd12dcec) > Nov 28 04:05:34 node1 kernel: [<c024e8c3>] cfs_proc_lookup [kernel] > 0x1b3 > (0xcd12dd24) > Nov 28 04:05:34 node1 kernel: [<c0174adb>] __mark_inode_dirty [kernel] > 0xbb (0xcd12dd34) > Nov 28 04:05:35 node1 kernel: [<c0176ecb>] update_atime [kernel] 0x6b > (0xcd12dd4c) > Nov 28 04:05:35 node1 kernel: [<c0192864>] ext3_readdir [kernel] 0x304 > (0xcd12dd60) > Nov 28 04:05:35 node1 kernel: [<c023f667>] tok_hold [kernel] 0x87 > (0xcd12ddd0) > Nov 28 04:05:35 node1 kernel: [<c025a157>] cfstok_req [kernel] 0x127 > (0xcd12ddf0) > Nov 28 04:05:35 node1 kernel: [<c013b4e5>] in_group_p [kernel] 0x25 > (0xcd12de34)Nov 28 04:05:35 node1 kernel: [<c0166b1a>] vfs_permission > [kernel] 0x8a (0xcd12de40) > Nov 28 04:05:35 node1 kernel: [<c024cfb1>] cfs_lookup [kernel] 0xe1 > (0xcd12de68)Nov 28 04:05:35 node1 kernel: [<c0166f5a>] real_lookup > [kernel] 0x19a (0xcd12dea4) > Nov 28 04:05:35 node1 kernel: [<c0166d41>] cached_lookup [kernel] 0x21 > (0xcd12deb0) > Nov 28 04:05:35 node1 kernel: [<c016786c>] link_path_walk_it [kernel] > 0x5bc (0xcd12decc) > Nov 28 04:05:35 node1 kernel: [<c0167e1e>] path_walk_it [kernel] 0x2e > (0xcd12df0c) > Nov 28 04:05:36 node1 kernel: [<c01684ff>] __user_walk_it [kernel] 0x6f > (0xcd12df20) > Nov 28 04:05:36 node1 kernel: [<c0163097>] sys_lstat64 [kernel] 0x37 > (0xcd12df44) > Nov 28 04:05:36 node1 kernel: [<c016d49d>] vfs_readdir [kernel] 0xad > (0xcd12df4c) > Nov 28 04:05:37 node1 kernel: [<c0157c3e>] sys_fchdir [kernel] 0x4e > (0xcd12df90)Nov 28 04:05:37 node1 kernel: [<c010be37>] system_call > [kernel] 0x33 (0xcd12dfc0) > Nov 28 04:05:37 node1 kernel: > Nov 28 04:05:38 node1 kernel: Code: 0f 0b 29 01 c1 16 3e c0 eb 0d 90 > 90 90 > 90 90 90 90 90 90 90 > > > > ------------------------------------------------------- > SF email is sponsored by - The IT Product Guide > Read honest & candid reviews on hundreds of IT Products from real > users. > Discover which products truly live up to the hype. Start reading now. > http://productguide.itmanagersjournal.com/ > _______________________________________________ > ssic-linux-devel mailing list > ssi...@li... > https://lists.sourceforge.net/lists/listinfo/ssic-linux-devel > > David B. Zafman mailto:da...@za... Never ask a man what computer he uses. If it's a Mac, he'll tell you. If it's not, why embarrass him? - Tom Clancy |