From: Matt W. <ma...@mo...> - 2006-11-18 11:45:51
|
I've been trying to get UML running under a Xen DomU running Centos 4.4, which I understand is a reasonable thing to do. When I started out I was using a download 2.6.12 kernel and the image booted up fine. I then tried 2.6.18.2, which is hanging after VFS: Mounted root (ext3 filesystem) readonly. I've had someone check my kernel and the image on another machine and it works without problem. I've tried some other kernels, and 2.6.14 seemed ok, 2.6.16 wasn't. I've had some problems with TLS under Xen running Berkely DB, so have rebuilt glibc with native threading disabled, but /lib/tls is still there because Bdb complains if it's removed. Below I've give the process list at the time, backtraces of each of these processes and the output of the ./linux ubd0=root_fs_router command. Let me know if more output is required Any help is very much appreciated. Thanks, Matt The UML processes that are running at this time are: matt 15425 15415 0 11:22 pts/1 00:00:00 ./linux ubd0=/home/uml/router/root_fs_router matt 15430 15415 0 11:22 pts/1 00:00:00 ./linux ubd0=/home/uml/router/root_fs_router matt 15431 15415 43 11:22 pts/1 00:07:04 [linux] matt 15415 17607 50 11:22 pts/1 00:08:09 ./linux ubd0=/home/uml/router/root_fs_router matt 15424 15415 0 11:22 pts/1 00:00:00 ./linux ubd0=/home/uml/router/root_fs_router 15431 and 15415 are taking 100% CPU between then. backtraces of those PIDs result in: PID 15425: #0 0xb7eaddbe in __read_nocancel () from /lib/tls/i686/libc.so.6 #1 0x08070510 in file_io () at include/linux/sched.h:1439 #2 0x0806d5b7 in io_thread (arg=0x0) at arch/um/drivers/ubd_kern.c:1374 #3 0xb7ebbcbe in clone () from /lib/tls/i686/libc.so.6 PID 15430: #0 0xb7eb2e3a in poll () from /lib/tls/i686/libc.so.6 #1 0x08072b55 in write_sigio_thread () at include/linux/sched.h:1439 #2 0xb7ebbcbe in clone () from /lib/tls/i686/libc.so.6 PID 15431: Operation not permitted at attach PID 15415: #0 0xb7eb5bff in ptrace (request=3086049268) at ../sysdeps/unix/sysv/linux/ptrace.c:104 #1 0x08077352 in userspace () at include/linux/sched.h:1439 #2 0x08061c79 in new_thread_handler (sig=10) at include/asm/thread_info.h:46 #3 <signal handler called> #4 0xb7e29041 in kill () at ../string/bits/string2.h:1000 #5 0x0828b580 in init_thread_union () #6 0x080735aa in set_signals () at include/linux/sched.h:1439 #7 0x08077b78 in new_thread () at include/linux/sched.h:1439 #8 0x00000000 in ?? () PID 15424: #0 0xb7eaddbe in __read_nocancel () from /lib/tls/i686/libc.so.6 #1 0x08070510 in file_io () at include/linux/sched.h:1439 #2 0x0806f95e in not_aio_thread () at include/linux/sched.h:1439 #3 0xb7ebbcbe in clone () from /lib/tls/i686/libc.so.6 Output from ./linux ubd0=./root_fs_router: [42949372.960000] Linux version 2.6.18.2 (root@localhost) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-3)) #4 Sat Nov 18 11:13:34 GMT 2006 [42949372.960000] Built 1 zonelists. Total pages: 8192 [42949372.960000] Kernel command line: ubd0=/home/uml/router/root_fs_router root=98:0 [42949372.960000] PID hash table entries: 256 (order: 8, 1024 bytes) [42949372.960000] ------------------------ [42949372.960000] | Locking API testsuite: [42949372.960000] ---------------------------------------------------------------------------- [42949372.960000] | spin |wlock |rlock |mutex | wsem | rsem | [42949372.960000] -------------------------------------------------------------------------- [42949372.960000] A-A deadlock:failed|failed| ok |failed|failed|failed| [42949372.960000] A-B-B-A deadlock:failed|failed| ok |failed|failed|failed| [42949372.960000] A-B-B-C-C-A deadlock:failed|failed| ok |failed|failed|failed| [42949372.960000] A-B-C-A-B-C deadlock:failed|failed| ok |failed|failed|failed| [42949372.960000] A-B-B-C-C-D-D-A deadlock:failed|failed| ok |failed|failed|failed| [42949372.960000] A-B-C-D-B-D-D-A deadlock:failed|failed| ok |failed|failed|failed| [42949372.960000] A-B-C-D-B-C-D-A deadlock:failed|failed| ok |failed|failed|failed| [42949372.960000] double unlock: ok | ok |failed| ok |failed|failed| [42949372.960000] initialize held:failed|failed|failed|failed|failed|failed| [42949372.960000] bad unlock order: ok | ok | ok | ok | ok | ok | [42949372.960000] -------------------------------------------------------------------------- [42949372.960000] recursive read-lock: | ok | |failed| [42949372.960000] recursive read-lock #2: | ok | |failed| [42949372.960000] mixed read-write-lock: |failed| |failed| [42949372.960000] mixed write-read-lock: |failed| |failed| [42949372.960000] -------------------------------------------------------------------------- [42949372.960000] hard-irqs-on + irq-safe-A/12:failed|failed| ok | [42949372.960000] soft-irqs-on + irq-safe-A/12:failed|failed| ok | [42949372.960000] hard-irqs-on + irq-safe-A/21:failed|failed| ok | [42949372.960000] soft-irqs-on + irq-safe-A/21:failed|failed| ok | [42949372.960000] sirq-safe-A => hirqs-on/12:failed|failed| ok | [42949372.960000] sirq-safe-A => hirqs-on/21:failed|failed| ok | [42949372.960000] hard-safe-A + irqs-on/12:failed|failed| ok | [42949372.960000] soft-safe-A + irqs-on/12:failed|failed| ok | [42949372.960000] hard-safe-A + irqs-on/21:failed|failed| ok | [42949372.960000] soft-safe-A + irqs-on/21:failed|failed| ok | [42949372.960000] hard-safe-A + unsafe-B #1/123:failed|failed| ok | [42949372.960000] soft-safe-A + unsafe-B #1/123:failed|failed| ok | [42949372.960000] hard-safe-A + unsafe-B #1/132:failed|failed| ok | [42949372.960000] soft-safe-A + unsafe-B #1/132:failed|failed| ok | [42949372.960000] hard-safe-A + unsafe-B #1/213:failed|failed| ok | [42949372.960000] soft-safe-A + unsafe-B #1/213:failed|failed| ok | [42949372.960000] hard-safe-A + unsafe-B #1/231:failed|failed| ok | [42949372.960000] soft-safe-A + unsafe-B #1/231:failed|failed| ok | [42949372.960000] hard-safe-A + unsafe-B #1/312:failed|failed| ok | [42949372.960000] soft-safe-A + unsafe-B #1/312:failed|failed| ok | [42949372.960000] hard-safe-A + unsafe-B #1/321:failed|failed| ok | [42949372.960000] soft-safe-A + unsafe-B #1/321:failed|failed| ok | [42949372.960000] hard-safe-A + unsafe-B #2/123:failed|failed| ok | [42949372.960000] soft-safe-A + unsafe-B #2/123:failed|failed| ok | [42949372.960000] hard-safe-A + unsafe-B #2/132:failed|failed| ok | [42949372.960000] soft-safe-A + unsafe-B #2/132:failed|failed| ok | [42949372.960000] hard-safe-A + unsafe-B #2/213:failed|failed| ok | [42949372.960000] soft-safe-A + unsafe-B #2/213:failed|failed| ok | [42949372.960000] hard-safe-A + unsafe-B #2/231:failed|failed| ok | [42949372.960000] soft-safe-A + unsafe-B #2/231:failed|failed| ok | [42949372.960000] hard-safe-A + unsafe-B #2/312:failed|failed| ok | [42949372.960000] soft-safe-A + unsafe-B #2/312:failed|failed| ok | [42949372.960000] hard-safe-A + unsafe-B #2/321:failed|failed| ok | [42949372.960000] soft-safe-A + unsafe-B #2/321:failed|failed| ok | [42949372.960000] hard-irq lock-inversion/123:failed|failed| ok | [42949372.960000] soft-irq lock-inversion/123:failed|failed| ok | [42949372.960000] hard-irq lock-inversion/132:failed|failed| ok | [42949372.960000] soft-irq lock-inversion/132:failed|failed| ok | [42949372.960000] hard-irq lock-inversion/213:failed|failed| ok | [42949372.960000] soft-irq lock-inversion/213:failed|failed| ok | [42949372.960000] hard-irq lock-inversion/231:failed|failed| ok | [42949372.960000] soft-irq lock-inversion/231:failed|failed| ok | [42949372.960000] hard-irq lock-inversion/312:failed|failed| ok | [42949372.960000] soft-irq lock-inversion/312:failed|failed| ok | [42949372.960000] hard-irq lock-inversion/321:failed|failed| ok | [42949372.960000] soft-irq lock-inversion/321:failed|failed| ok | [42949372.960000] hard-irq read-recursion/123: ok | [42949372.960000] soft-irq read-recursion/123: ok | [42949372.960000] hard-irq read-recursion/132: ok | [42949372.960000] soft-irq read-recursion/132: ok | [42949372.960000] hard-irq read-recursion/213: ok | [42949372.960000] soft-irq read-recursion/213: ok | [42949372.960000] hard-irq read-recursion/231: ok | [42949372.960000] soft-irq read-recursion/231: ok | [42949372.960000] hard-irq read-recursion/312: ok | [42949372.960000] soft-irq read-recursion/312: ok | [42949372.960000] hard-irq read-recursion/321: ok | [42949372.960000] soft-irq read-recursion/321: ok | [42949372.960000] -------------------------------------------------------- [42949372.960000] 142 out of 218 testcases failed, as expected. | [42949372.960000] ---------------------------------------------------- [42949372.960000] Dentry cache hash table entries: 4096 (order: 2, 16384 bytes) [42949372.960000] Inode-cache hash table entries: 2048 (order: 1, 8192 bytes) [42949372.960000] Memory: 29408k available [42949373.480000] Mount-cache hash table entries: 512 [42949373.480000] Checking for host processor cmov support...Yes [42949373.480000] Checking for host processor xmm support...No [42949373.480000] Checking that host ptys support output SIGIO...Yes [42949373.480000] Checking that host ptys support SIGIO on close...No, enabling workaround [42949373.480000] /usr/include/linux/aio_abi.h not present during build [42949373.480000] 2.6 host AIO support not used - falling back to I/O thread [42949373.480000] NET: Registered protocol family 16 [42949373.480000] NET: Registered protocol family 2 [42949373.660000] IP route cache hash table entries: 256 (order: -2, 1024 bytes) [42949373.660000] TCP established hash table entries: 1024 (order: 2, 16384 bytes) [42949373.660000] TCP bind hash table entries: 512 (order: 1, 10240 bytes) [42949373.660000] TCP: Hash tables configured (established 1024 bind 512) [42949373.660000] TCP reno registered [42949373.660000] Checking host MADV_REMOVE support...OK [42949373.660000] mconsole (version 2) initialized on /home/matt/.uml/i5b2KQ/mconsole [42949373.660000] Host TLS support detected [42949373.660000] Detected host type: i386 [42949373.660000] VFS: Disk quotas dquot_6.5.1 [42949373.660000] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) [42949373.660000] Initializing Cryptographic API [42949373.660000] io scheduler noop registered [42949373.660000] io scheduler anticipatory registered (default) [42949373.660000] io scheduler deadline registered [42949373.660000] io scheduler cfq registered [42949373.660000] loop: loaded (max 8 devices) [42949373.660000] nbd: registered device at major 43 [42949373.660000] PPP generic driver version 2.4.2 [42949373.660000] SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256). [42949373.660000] tun: Universal TUN/TAP device driver, 1.6 [42949373.660000] tun: (C) 1999-2004 Max Krasnyansky <ma...@qu...> [42949373.660000] ip_conntrack version 2.4 (229 buckets, 1832 max) - 228 bytes per conntrack [42949374.160000] NET: Registered protocol family 1 [42949374.160000] NET: Registered protocol family 17 [42949375.000000] Bridge firewalling registered [42949375.000000] 802.1Q VLAN Support v1.8 Ben Greear <gr...@ca...> [42949375.000000] All bugs added by David S. Miller <da...@re...> [42949375.000000] Initialized stdio console driver [42949375.000000] Console initialized on /dev/tty0 [42949375.000000] Initializing software serial port version 1 [42949375.000000] ubda: unknown partition table [42949375.000000] kjournald starting. Commit interval 5 seconds [42949375.000000] EXT3-fs: mounted filesystem with ordered data mode. [42949375.000000] VFS: Mounted root (ext3 filesystem) readonly. |
From: Blaisorblade <bla...@ya...> - 2006-11-21 07:13:31
|
On Saturday 18 November 2006 12:45, Matt Willsher wrote: > I've been trying to get UML running under a Xen DomU running Centos > 4.4, which I understand is a reasonable thing to do. > > When I started out I was using a download 2.6.12 kernel and the image > booted up fine. I then tried 2.6.18.2, which is hanging after VFS: > Mounted root (ext3 filesystem) readonly. I've had someone check my > kernel and the image on another machine and it works without problem. > I've tried some other kernels, and 2.6.14 seemed ok, 2.6.16 wasn't. > > I've had some problems with TLS under Xen running Berkely DB, so have > rebuilt glibc with native threading disabled, but /lib/tls is still > there because Bdb complains if it's removed. > > Below I've give the process list at the time, backtraces of each of > these processes and the output of the ./linux ubd0=root_fs_router > command. Let me know if more output is required > > Any help is very much appreciated. I have two suggestions for you to try: * reduce CONFIG_HOST_VM_SPLIT for UML guest kernels (Xen consumes part of the available memory for UML, which causes crashes) * try if booting with mode=tt helps - around 2.6.14 or 2.6.15 skas0 support was introduced (a very big change!) and become the new default, so that's the 1st likely culprit. -- Inform me of my mistakes, so I can add them to my list! Paolo Giarrusso, aka Blaisorblade http://www.user-mode-linux.org/~blaisorblade Chiacchiera con i tuoi amici in tempo reale! http://it.yahoo.com/mail_it/foot/*http://it.messenger.yahoo.com |
From: Matt W. <ma...@mo...> - 2006-11-21 19:23:14
|
On 11/21/06, Blaisorblade <bla...@ya...> wrote: > On Saturday 18 November 2006 12:45, Matt Willsher wrote: > > I've been trying to get UML running under a Xen DomU running Centos > > 4.4, which I understand is a reasonable thing to do. > > > > When I started out I was using a download 2.6.12 kernel and the image > > booted up fine. I then tried 2.6.18.2, which is hanging after VFS: > > Mounted root (ext3 filesystem) readonly. I've had someone check my > > kernel and the image on another machine and it works without problem. > > I've tried some other kernels, and 2.6.14 seemed ok, 2.6.16 wasn't. > I have two suggestions for you to try: > * reduce CONFIG_HOST_VM_SPLIT for UML guest kernels (Xen consumes part of the > available memory for UML, which causes crashes) > > * try if booting with mode=tt helps - around 2.6.14 or 2.6.15 skas0 support > was introduced (a very big change!) and become the new default, so that's the > 1st likely culprit. Thank you for your suggestions. I must correct myself. 2.6.14 does indeed hang in the same way as later kernels. I think I tried 2.6.13 resulting in the same hang but I can't be sure (I've tried rather a lot of kernels this last week!) I've applied your latest patch bundle to the 2.6.18.3 tree, and set the VM_HOST split to 1/3. Booting with this, the guest kernel hangs at the same point. With mode=tt it crashes outright. I've posted the output from the run below. I've also got an strace output, which I can post if you want, but thought I'd check first as it's 380K. The same command line used on the below works with a 2.6.12.2-bb11 kernel. I've also tried init=/bin/sh, again with the same results as below. Checking that ptrace can change system call numbers...OK Checking syscall emulation patch for ptrace...OK Checking advanced syscall emulation patch for ptrace...OK Checking for tmpfs mount on /dev/shm...OK Checking PROT_EXEC mmap in /dev/shm/...OK UML running in TT mode tracing thread pid = 1350 Checking that ptrace can change system call numbers...OK Checking syscall emulation patch for ptrace...OK Checking advanced syscall emulation patch for ptrace...OK Linux version 2.6.18.3-bb2 (root@localhost) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-3)) #3 Tue Nov 21 19:06:28 GMT 2006 Built 1 zonelists. Total pages: 16384 Kernel command line: ubd0=/home/uml/images/ano_router_root_fs ubd1=/home/uml/images/ano_router_swap_fs mode=tt mem=64M con=null con0=fd:0,fd:1 con1=port:9000 eth0=tuntap,umltun0 eth1=tuntap,umltun1 root=98:0 PID hash table entries: 512 (order: 9, 2048 bytes) Dentry cache hash table entries: 8192 (order: 3, 32768 bytes) Inode-cache hash table entries: 4096 (order: 2, 16384 bytes) Memory: 61164k available Mount-cache hash table entries: 512 Checking for host processor cmov support...Yes Checking for host processor xmm support...No Checking that host ptys support output SIGIO...Yes Checking that host ptys support SIGIO on close...No, enabling workaround /usr/include/linux/aio_abi.h not present during build 2.6 host AIO support not used - falling back to I/O thread Failed to initialize umid "anoroute", trying with a random umid make_umid - mkstemp(/home/uml/.uml/MyLtrF) failed: Permission denied NET: Registered protocol family 16 NET: Registered protocol family 2 IP route cache hash table entries: 512 (order: -1, 2048 bytes) TCP established hash table entries: 2048 (order: 1, 8192 bytes) TCP bind hash table entries: 1024 (order: 0, 4096 bytes) TCP: Hash tables configured (established 2048 bind 1024) TCP reno registered Checking host MADV_REMOVE support...OK make_umid - mkstemp(/home/uml/.uml/i5cVhQ) failed: Permission denied ubd: Synchronous mode Netdevice 0 : TUN/TAP backend - Netdevice 1 : TUN/TAP backend - Host TLS support detected Detected host type: i386 Initializing Cryptographic API io scheduler noop registered io scheduler anticipatory registered io scheduler deadline registered io scheduler cfq registered (default) NET: Registered protocol family 1 NET: Registered protocol family 17 Initialized stdio console driver Console initialized on /dev/tty0 Initializing software serial port version 1 ubda: unknown partition table ubdb: unknown partition table kjournald starting. Commit interval 5 seconds EXT3-fs: mounted filesystem with ordered data mode. VFS: Mounted root (ext3 filesystem) readonly. Kernel panic - not syncing: No init found. Try passing init= option to kernel. EIP: 0073:[<2017abc1>] CPU: 0 Not tainted ESP: 007b:203357d4 EFLAGS: 00000216 Not tainted EAX: 00000000 EBX: 00000566 ECX: 0000000a EDX: 20014df2 ESI: 00000000 EDI: 00000000 EBP: 00000000 DS: 007b ES: 007b 20333bdc: [<20048c2c>] notifier_call_chain+0x28/0x3e 20333bf0: [<2000e193>] init+0x0/0x175 20333bf8: [<2003643b>] panic+0x70/0xff 20333c10: [<2000e2a0>] init+0x10d/0x175 20333c2c: [<20026c66>] run_kernel_thread+0x52/0x5c 20333c3c: [<2000e193>] init+0x0/0x175 20333c4c: [<20026c39>] run_kernel_thread+0x25/0x5c 20333cd4: [<2000e193>] init+0x0/0x175 20333cdc: [<20014dc1>] new_thread_handler+0x13f/0x170 20333ce0: [<2000e193>] init+0x0/0x175 20333d48: [<20014df2>] new_thread_proc+0x0/0x43 20333d5c: [<2017abc1>] __kill+0x11/0x20 remove_umid_dir - remove_files_and_dir failed with err = -13 |
From: Blaisorblade <bla...@ya...> - 2006-11-22 00:22:36
|
On Tuesday 21 November 2006 20:23, Matt Willsher wrote: > On 11/21/06, Blaisorblade <bla...@ya...> wrote: > > On Saturday 18 November 2006 12:45, Matt Willsher wrote: > > > I've been trying to get UML running under a Xen DomU running Centos > > > 4.4, which I understand is a reasonable thing to do. > > > > > > When I started out I was using a download 2.6.12 kernel and the image > > > booted up fine. I then tried 2.6.18.2, which is hanging after VFS: > > > Mounted root (ext3 filesystem) readonly. I've had someone check my > > > kernel and the image on another machine and it works without problem. > > > I've tried some other kernels, and 2.6.14 seemed ok, 2.6.16 wasn't. > > > > I have two suggestions for you to try: > > * reduce CONFIG_HOST_VM_SPLIT for UML guest kernels (Xen consumes part of > > the available memory for UML, which causes crashes) > > > > * try if booting with mode=tt helps - around 2.6.14 or 2.6.15 skas0 > > support was introduced (a very big change!) and become the new default, > > so that's the 1st likely culprit. > > Thank you for your suggestions. > > I must correct myself. 2.6.14 does indeed hang in the same way as > later kernels. I think I tried 2.6.13 resulting in the same hang but I > can't be sure (I've tried rather a lot of kernels this last week!) > > I've applied your latest patch bundle to the 2.6.18.3 tree, and set > the VM_HOST split to 1/3. Booting with this, the guest kernel hangs at > the same point. With mode=tt it crashes outright. mode=tt can possibly work only on _old_ kernel, say the first one where skas0 was introduced i.e. (if my guess is right) the first one breaking under Xen. > I've posted the > output from the run below. I've also got an strace output, which I can > post if you want, but thought I'd check first as it's 380K. The output makes me suspicious of some misconfiguration - I've never seen a bug giving a misleading "no init found". And it happens that even smart users use different command lines, say, because they use a script for the default kernel and do not use it for the other one. /home/uml/.uml ($HOME/.uml) seems not writable according to messages below, which is _very_ strange. If the error message is wrong it should be checked in strace output, looking for EPERM or EACCESS on access to files mentioned below I guess (but I'll not have the time to even give a look to it, maybe someone else will be able or you can give a look yourself). > The same command line used on the below works with a 2.6.12.2-bb11 > kernel. I've also tried init=/bin/sh, again with the same results as > below. > > Checking that ptrace can change system call numbers...OK > Checking syscall emulation patch for ptrace...OK > Checking advanced syscall emulation patch for ptrace...OK > Checking for tmpfs mount on /dev/shm...OK > Checking PROT_EXEC mmap in /dev/shm/...OK > UML running in TT mode > tracing thread pid = 1350 > Checking that ptrace can change system call numbers...OK > Checking syscall emulation patch for ptrace...OK > Checking advanced syscall emulation patch for ptrace...OK > Linux version 2.6.18.3-bb2 (root@localhost) (gcc version 3.4.6 > 20060404 (Red Hat 3.4.6-3)) #3 Tue Nov 21 19:06:28 GMT 2006 > Built 1 zonelists. Total pages: 16384 > Kernel command line: ubd0=/home/uml/images/ano_router_root_fs > ubd1=/home/uml/images/ano_router_swap_fs mode=tt mem=64M con=null > con0=fd:0,fd:1 con1=port:9000 eth0=tuntap,umltun0 eth1=tuntap,umltun1 > root=98:0 > PID hash table entries: 512 (order: 9, 2048 bytes) > Dentry cache hash table entries: 8192 (order: 3, 32768 bytes) > Inode-cache hash table entries: 4096 (order: 2, 16384 bytes) > Memory: 61164k available > Mount-cache hash table entries: 512 > Checking for host processor cmov support...Yes > Checking for host processor xmm support...No > Checking that host ptys support output SIGIO...Yes > Checking that host ptys support SIGIO on close...No, enabling workaround > /usr/include/linux/aio_abi.h not present during build > 2.6 host AIO support not used - falling back to I/O thread > Failed to initialize umid "anoroute", trying with a random umid > make_umid - mkstemp(/home/uml/.uml/MyLtrF) failed: Permission denied > NET: Registered protocol family 16 > NET: Registered protocol family 2 > IP route cache hash table entries: 512 (order: -1, 2048 bytes) > TCP established hash table entries: 2048 (order: 1, 8192 bytes) > TCP bind hash table entries: 1024 (order: 0, 4096 bytes) > TCP: Hash tables configured (established 2048 bind 1024) > TCP reno registered > Checking host MADV_REMOVE support...OK > make_umid - mkstemp(/home/uml/.uml/i5cVhQ) failed: Permission denied > ubd: Synchronous mode > Netdevice 0 : TUN/TAP backend - > Netdevice 1 : TUN/TAP backend - > Host TLS support detected > Detected host type: i386 > Initializing Cryptographic API > io scheduler noop registered > io scheduler anticipatory registered > io scheduler deadline registered > io scheduler cfq registered (default) > NET: Registered protocol family 1 > NET: Registered protocol family 17 > Initialized stdio console driver > Console initialized on /dev/tty0 > Initializing software serial port version 1 > ubda: unknown partition table > ubdb: unknown partition table > kjournald starting. Commit interval 5 seconds > EXT3-fs: mounted filesystem with ordered data mode. > VFS: Mounted root (ext3 filesystem) readonly. > Kernel panic - not syncing: No init found. Try passing init= option to > kernel. -- Inform me of my mistakes, so I can add them to my list! Paolo Giarrusso, aka Blaisorblade http://www.user-mode-linux.org/~blaisorblade Chiacchiera con i tuoi amici in tempo reale! http://it.yahoo.com/mail_it/foot/*http://it.messenger.yahoo.com |
From: Matt W. <ma...@mo...> - 2006-11-22 08:05:02
|
On 11/22/06, Blaisorblade <bla...@ya...> wrote: > On Tuesday 21 November 2006 20:23, Matt Willsher wrote: > > On 11/21/06, Blaisorblade <bla...@ya...> wrote: > > > > I've applied your latest patch bundle to the 2.6.18.3 tree, and set > > the VM_HOST split to 1/3. Booting with this, the guest kernel hangs at > > the same point. With mode=tt it crashes outright. > > mode=tt can possibly work only on _old_ kernel, say the first one where skas0 > was introduced i.e. (if my guess is right) the first one breaking under Xen. >From the output below is the line: UML running in TT mode Does that not indicate that it is running in TT mode? The kernel option for TT mode is set. I tried to disable SKAS support but the kernel won't compile with out that set (check_host_supports_tls not found, as reported on the user mailing list last month). I will however try with 2.6.13 and 2.6.14. > The output makes me suspicious of some misconfiguration - I've never seen a > bug giving a misleading "no init found". And it happens that even smart users > use different command lines, say, because they use a script for the default > kernel and do not use it for the other one. The same script was used for both. I copied my start script and simply changed the kernel file being called and added the mode=tt option. I was particular about this. I also used the same base .config file for the 2.6.12 kernel and the 2.6.18 kernel, running menuconfig to sanitise it, and only adjusted the options you've recommended. > /home/uml/.uml ($HOME/.uml) seems not writable according to messages below, > which is _very_ strange. That was my fault. I accidentally ran the binary as root before running it as the unpriviledged user. I fixed the permissions and the error went away but I still get the same crash. As far as further debugging from me, I'm at the limits of my abilities in that area. I'm happy do what every anyone requests of me if I'm given a few pointers in the right direction. I'm keen to see this issue resolved. Thanks again for you time on this, Matt > > The same command line used on the below works with a 2.6.12.2-bb11 > > kernel. I've also tried init=/bin/sh, again with the same results as > > below. > > > > Checking that ptrace can change system call numbers...OK > > Checking syscall emulation patch for ptrace...OK > > Checking advanced syscall emulation patch for ptrace...OK > > Checking for tmpfs mount on /dev/shm...OK > > Checking PROT_EXEC mmap in /dev/shm/...OK > > UML running in TT mode > > tracing thread pid = 1350 > > Checking that ptrace can change system call numbers...OK > > Checking syscall emulation patch for ptrace...OK > > Checking advanced syscall emulation patch for ptrace...OK > > Linux version 2.6.18.3-bb2 (root@localhost) (gcc version 3.4.6 > > 20060404 (Red Hat 3.4.6-3)) #3 Tue Nov 21 19:06:28 GMT 2006 > > Built 1 zonelists. Total pages: 16384 > > Kernel command line: ubd0=/home/uml/images/ano_router_root_fs > > ubd1=/home/uml/images/ano_router_swap_fs mode=tt mem=64M con=null > > con0=fd:0,fd:1 con1=port:9000 eth0=tuntap,umltun0 eth1=tuntap,umltun1 > > root=98:0 > > PID hash table entries: 512 (order: 9, 2048 bytes) > > Dentry cache hash table entries: 8192 (order: 3, 32768 bytes) > > Inode-cache hash table entries: 4096 (order: 2, 16384 bytes) > > Memory: 61164k available > > Mount-cache hash table entries: 512 > > Checking for host processor cmov support...Yes > > Checking for host processor xmm support...No > > Checking that host ptys support output SIGIO...Yes > > Checking that host ptys support SIGIO on close...No, enabling workaround > > /usr/include/linux/aio_abi.h not present during build > > 2.6 host AIO support not used - falling back to I/O thread > > Failed to initialize umid "anoroute", trying with a random umid > > make_umid - mkstemp(/home/uml/.uml/MyLtrF) failed: Permission denied > > NET: Registered protocol family 16 > > NET: Registered protocol family 2 > > IP route cache hash table entries: 512 (order: -1, 2048 bytes) > > TCP established hash table entries: 2048 (order: 1, 8192 bytes) > > TCP bind hash table entries: 1024 (order: 0, 4096 bytes) > > TCP: Hash tables configured (established 2048 bind 1024) > > TCP reno registered > > Checking host MADV_REMOVE support...OK > > make_umid - mkstemp(/home/uml/.uml/i5cVhQ) failed: Permission denied > > ubd: Synchronous mode > > Netdevice 0 : TUN/TAP backend - > > Netdevice 1 : TUN/TAP backend - > > Host TLS support detected > > Detected host type: i386 > > Initializing Cryptographic API > > io scheduler noop registered > > io scheduler anticipatory registered > > io scheduler deadline registered > > io scheduler cfq registered (default) > > NET: Registered protocol family 1 > > NET: Registered protocol family 17 > > Initialized stdio console driver > > Console initialized on /dev/tty0 > > Initializing software serial port version 1 > > ubda: unknown partition table > > ubdb: unknown partition table > > kjournald starting. Commit interval 5 seconds > > EXT3-fs: mounted filesystem with ordered data mode. > > VFS: Mounted root (ext3 filesystem) readonly. > > Kernel panic - not syncing: No init found. Try passing init= option to > > kernel. |
From: Jeff D. <jd...@ad...> - 2006-11-22 16:44:45
|
On Wed, Nov 22, 2006 at 08:04:59AM +0000, Matt Willsher wrote: > UML running in TT mode > > Does that not indicate that it is running in TT mode? The kernel > option for TT mode is set. I tried to disable SKAS support but the > kernel won't compile with out that set (check_host_supports_tls not > found, as reported on the user mailing list last month). I will > however try with 2.6.13 and 2.6.14. Disable CONFIG_MODE_TT. Its presence in the binary, even if it's not running, has caused boot failures. Jeff -- Work email - jdike at linux dot intel dot com |
From: Jeff D. <jd...@ad...> - 2006-11-22 16:48:10
|
On Wed, Nov 22, 2006 at 08:04:59AM +0000, Matt Willsher wrote: > UML running in TT mode Sorry about that last suggestion - I missed that you were trying an old UML. I have to repeat what BB said about the "init not found" error. I never seen that not be a filesystem problem. Can you try booting that filesystem in an environment which is known to work, to make sure that it is sane? Jeff -- Work email - jdike at linux dot intel dot com |
From: Matt W. <ma...@mo...> - 2006-11-24 07:30:05
|
On 11/22/06, Jeff Dike <jd...@ad...> wrote: > On Wed, Nov 22, 2006 at 08:04:59AM +0000, Matt Willsher wrote: > > UML running in TT mode > > Sorry about that last suggestion - I missed that you were trying an > old UML. The TT mode on my last mail was on the latest kernel (2.6.18.3) with the BB patch sets applied. I recompiled with TT disabled and let it run in SKAS0 mode and it hung as per my initial mail. > I have to repeat what BB said about the "init not found" error. I > never seen that not be a filesystem problem. Can you try booting that > filesystem in an environment which is known to work, to make sure that > it is sane? This was only in TT mode on the most recent kernel. The image I was using was one that I am successfully using under 2.6.12.6-bs11. I've not tried 2.6.13 or 2.6.14 again in TT mode, but I'm not sure what that would prove? If it would aid in troubleshooting the problem of running under the latest kernel in SKAS0 mode, give TT mode on the new kernels is obsolete, I am of course happy to try that. My initial searching into this problem returned a reference to a VDSO issue, which is also documented on the new site on the problems page. I'm not sure if this could be the cause of the problem as the UML I'm running is under a Xen DomU, and I have no control over this kernel and /proc/config.gz is not available. |
From: Jeff D. <jd...@ad...> - 2006-11-24 23:18:37
|
On Fri, Nov 24, 2006 at 09:10:50PM +0000, Matt Willsher wrote: > The very same image, uncopied. I have a directory tree set up and > scripts I use and all I did was to change the script to call the newer > kernel. OK, a couple experiments to try: add "init=/bin/bash" to the command line and see what that does find a statically linked shell in the filesystem and use that as the argument to "init=". Jeff -- Work email - jdike at linux dot intel dot com |
From: Jeff D. <jd...@ad...> - 2006-12-01 00:26:48
|
On Fri, Nov 24, 2006 at 09:10:50PM +0000, Matt Willsher wrote: > The very same image, uncopied. I have a directory tree set up and > scripts I use and all I did was to change the script to call the newer > kernel. And the exact same UML binary? I'm grabbing at straws here, in case somehow the broken one is lacking binfmt support for /sbin/init. Jeff -- Work email - jdike at linux dot intel dot com |
From: Matt W. <ma...@mo...> - 2006-12-01 07:16:28
|
The problem I'm having On 12/1/06, Jeff Dike <jd...@ad...> wrote: > On Fri, Nov 24, 2006 at 09:10:50PM +0000, Matt Willsher wrote: > > The very same image, uncopied. I have a directory tree set up and > > scripts I use and all I did was to change the script to call the newer > > kernel. > > And the exact same UML binary? I'm grabbing at straws here, in case > somehow the broken one is lacking binfmt support for /sbin/init. > Just to recap: TT mode support under 2.6.12 works fine and I can boot a custom build kernel with it. A downloaded, known good 2.6.12 also boots a know good linux disk image. 2.6.18, either compiled by myself, or a downloaded known good doesn't boot either my known good disk image as used with 2.6.12 or one downloaded from the web. The 2.6.18 that doesn't work on my machine has been used successfully on someone elses environment boot my known good image. My guess is that this issue is related to SKAS0 support in the newer kernels. I've been meaning to find the earliest kernel that ships with SKAS0 and try both TT mode and SKAS0 mode to see what happens there. http://user-mode-linux.sourceforge.net/new/problems.html#vdso seems to be the problem I'm having but I'm running UML under Xen, and there is not /proc/config.gz to check the kernel config, nor do I access to Dom0. so I have no way to check CONFIG_COMPAT_VDSO or have it changed on my host. I'm willing to have a go at debugging using gdb etc. but I really don't know what I'm looking for there. |