From: Raymond W. <ra...@li...> - 2004-05-20 01:41:23
|
I encountered problem in migrating uml filesystem image from one system to another since the size of the image is more than 2G. I used scp and cp but both failed to handle files more than 2G. What would be the best way to migrate such image? --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.687 / Virus Database: 448 - Release Date: 16/5/2004 |
From: Emiliano C. <ec...@fi...> - 2004-05-20 02:10:14
|
El [ Thu 20, May 04 - 09:39 ] , Raymond Wong expreso: > I encountered problem in migrating uml filesystem image from one system to > another since the size of the image is more than 2G. I used scp and cp but > both failed to handle files more than 2G. What would be the best way to > migrate such image? Have you tried dd with ssh ? You could do something like this: dd if=<sourcefile> bs=<BYTES> count=<BLOCKS> | ssh user@server dd of=<destfile> bs=<BYTES> count=<BLOCKS> IMHO, this is a good work around, although, I haven't tried it with files as large a 2Gigs, but it should work. Ah !! BTW, anyone correct me if I'm wrong, but there is no need to use params bs and count, dd will work just fine until it encounters EOF, right ? On the other hand, if this doesn't work as expected, you could use those to params to break the file into pieces, and then concatenate it on the remote host. As an example: dd if=huge_file bs=1024 count=1M | ssh user@server dd of=huge_file_1 bs=1024 count=1M dd if=huge_file bs=1024 count=1M seek=1M | ssh user@server dd of=huge_file_2 bs=1024 count=1M This would split up the file in to _not_so_huge_ files (still big enough }8-D), of 1Gig each. Hope this will help Cheers !! -- Emiliano Castagnari # Debian Sarge - GNU/Linux - Athos 2.6.6 # # JID: pre...@ja... # ICQ: 107462374 - Nick: mem -------------------------------------------------- - } [ Libera tu mente - Libera tu Codigo ] { - -------------------------------------------------- |
From: roland <for...@gm...> - 2004-05-20 02:35:03
|
you could temporary compress that image (compress, gzip....) to cut down size below 2G. for not-so-good-but-very-fast compression, i would recommend using lzop: http://www.lzop.org/ http://www.lzop.de unfortunately i cannot tell you, if the downloadable binary linux-versions are large-file (>2G) aware. i mainly use lzop on solaris and have compiled it myself with largefile support. regards roland ----- Original Message ----- From: "Emiliano Castagnari" <ec...@fi...> To: <use...@li...> Sent: Thursday, May 20, 2004 4:25 AM Subject: Re: [uml-user] Migrating uml image > El [ Thu 20, May 04 - 09:39 ] , Raymond Wong expreso: > > I encountered problem in migrating uml filesystem image from one system to > > another since the size of the image is more than 2G. I used scp and cp but > > both failed to handle files more than 2G. What would be the best way to > > migrate such image? > > Have you tried dd with ssh ? > > You could do something like this: > > dd if=<sourcefile> bs=<BYTES> count=<BLOCKS> | ssh user@server dd of=<destfile> bs=<BYTES> count=<BLOCKS> > > IMHO, this is a good work around, although, I haven't tried it with files as large > a 2Gigs, but it should work. > > Ah !! BTW, anyone correct me if I'm wrong, but there is no need to use params bs and count, > dd will work just fine until it encounters EOF, right ? > > On the other hand, if this doesn't work as expected, you could use those to params > to break the file into pieces, and then concatenate it on the remote host. > > As an example: > > dd if=huge_file bs=1024 count=1M | ssh user@server dd of=huge_file_1 bs=1024 count=1M > dd if=huge_file bs=1024 count=1M seek=1M | ssh user@server dd of=huge_file_2 bs=1024 count=1M > > This would split up the file in to _not_so_huge_ files (still big enough }8-D), of 1Gig each. > > Hope this will help > > Cheers !! > -- > Emiliano Castagnari > > # Debian Sarge - GNU/Linux - Athos 2.6.6 # > # JID: pre...@ja... > # ICQ: 107462374 - Nick: mem > -------------------------------------------------- > - } [ Libera tu mente - Libera tu Codigo ] { - > -------------------------------------------------- > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > User-mode-linux-user mailing list > Use...@li... > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user > |
From: Anthony B. <Ant...@ou...> - 2004-05-20 21:55:58
|
> -----Original Message----- > Ah !! BTW, anyone correct me if I'm wrong, but there is no need > to use params bs and count, > dd will work just fine until it encounters EOF, right ? Across a network connection it may not matter so much. But past experience has strongly recommended setting the 'bs' parameter to as large a value as possible that will still remain within available free memory. For example, a while back I was copying Solaris 9 HD images between physical drives (on the same machine). The first time I launched the command, I failed to specify the 'bs' parameter (default was 512 bytes). It was less than half completed 29 hours later. I was able to reduce the copy time to around 2 hours by changing the 'bs' parameter to 64M. However, I believe you're right concerning the EOF. The primary reason for count is when copying from /dev/zero (which will NEVER give an EOF) or special file manipulations. Tony |
From: Raymond W. <ra...@li...> - 2004-05-21 02:21:06
|
In Linux, the parameter setting of "bs=1 count=1024" takes longer than "bs=1024 count=1". -----Original Message----- From: use...@li... [mailto:use...@li...] On Behalf Of Anthony Brock Sent: Friday, May 21, 2004 5:56 AM To: use...@li... Subject: RE: [uml-user] Migrating uml image > -----Original Message----- > Ah !! BTW, anyone correct me if I'm wrong, but there is no need > to use params bs and count, > dd will work just fine until it encounters EOF, right ? Across a network connection it may not matter so much. But past experience has strongly recommended setting the 'bs' parameter to as large a value as possible that will still remain within available free memory. For example, a while back I was copying Solaris 9 HD images between physical drives (on the same machine). The first time I launched the command, I failed to specify the 'bs' parameter (default was 512 bytes). It was less than half completed 29 hours later. I was able to reduce the copy time to around 2 hours by changing the 'bs' parameter to 64M. However, I believe you're right concerning the EOF. The primary reason for count is when copying from /dev/zero (which will NEVER give an EOF) or special file manipulations. Tony ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click _______________________________________________ User-mode-linux-user mailing list Use...@li... https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user --- Incoming mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.687 / Virus Database: 448 - Release Date: 16/5/2004 --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.687 / Virus Database: 448 - Release Date: 16/5/2004 |
From: uml <um...@do...> - 2004-05-21 04:55:05
|
Is there any particular reason that my UML won't recognize this switch? It states that it does on the CLI, but inside the UML, it always says that I've only got about 28M. Why? |
From: Raymond W. <ra...@li...> - 2004-05-21 05:55:03
|
Can you dump the boot up screen? -----Original Message----- From: use...@li... [mailto:use...@li...] On Behalf Of uml Sent: Friday, May 21, 2004 1:00 PM To: use...@li... Subject: [uml-user] mem=64M, mem=128M, mem=ANYTHING ignored?? Is there any particular reason that my UML won't recognize this switch? It states that it does on the CLI, but inside the UML, it always says that I've only got about 28M. Why? ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click _______________________________________________ User-mode-linux-user mailing list Use...@li... https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user --- Incoming mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.687 / Virus Database: 448 - Release Date: 16/5/2004 --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.687 / Virus Database: 448 - Release Date: 16/5/2004 |
From: uml <um...@do...> - 2004-05-21 13:51:39
|
Should have done that in the first place... here it is: (BTW, with or without COW or the memory switch, it gives me 28M.. /= ---8<--- Checking for the skas3 patch in the host...found Checking for /proc/mm...found Linux version 2.6.4-1um (root@linux) (gcc version 3.3.3 (SuSE Linux)) #1 Fri May 21 01:02:46 EDT 2004 On node 0 totalpages: 8192 DMA zone: 8192 pages, LIFO batch:2 Normal zone: 0 pages, LIFO batch:1 HighMem zone: 0 pages, LIFO batch:1 Built 1 zonelists Kernel command line: ubd0=cow1 mem=128M root=98:0 PID hash table entries: 16 (order 4: 128 bytes) Memory: 29048k available Calibrating delay loop... 1677.72 BogoMIPS Dentry cache hash table entries: 4096 (order: 2, 16384 bytes) Inode-cache hash table entries: 2048 (order: 1, 8192 bytes) Mount-cache hash table entries: 512 (order: 0, 4096 bytes) Checking for host processor cmov support...Yes Checking for host processor xmm support...No Checking that ptrace can change system call numbers...OK Checking that host ptys support output SIGIO...Yes Checking that host ptys support SIGIO on close...No, enabling workaround Checking for /dev/anon on the host...Not available (open failed with errno 2) POSIX conformance testing by UNIFIX NET: Registered protocol family 16 make_umid - mkstemp failed, errno = 2 UML Audio Relay (host dsp = /dev/sound/dsp, host mixer = /dev/sound/mixer) VFS: Disk quotas dquot_6.5.1 devfs: 2004-01-31 Richard Gooch (rg...@at...) devfs: boot_options: 0x1 JFFS version 1.0, (C) 1999, 2000 Axis Communications AB RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize loop: loaded (max 8 devices) Using anticipatory io scheduler nbd: registered device at major 43 PPP generic driver version 2.4.2 SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256). Universal TUN/TAP device driver 1.5 (C)1999-2002 Maxim Krasnyansky blkmtd: version $Revision: 1.5 $ blkmtd: error: missing `device' name NET: Registered protocol family 2 IP: routing cache hash table of 512 buckets, 4Kbytes TCP: Hash tables configured (established 2048 bind 4096) NET: Registered protocol family 1 NET: Registered protocol family 17 Initializing software serial port version 1 /dev/ubd/disc0: unknown partition table Initializing stdio console driver EXT2-fs warning (device ubda): ext2_fill_super: mounting ext3 filesystem as ext2 VFS: Mounted root (ext2 filesystem) readonly. Mounted devfs on /dev INIT: version 2.84 booting Starting devfs daemon: /sbin/devfsd /dev Started device management daemon v1.3.25 for /dev /etc/rc.d/rc.S: line 23: Testing filesystem status: Read-only file system Checking root filesystem: fsck 1.34 (25-Jul-2003) /dev/ubd/0: clean, 17536/53784 files, 101826/215040 blocks Remounting root device with read-write enabled. /dev/ubd/disc0/disc on / type ext2 (rw) Checking non-root filesystems: fsck 1.34 (25-Jul-2003) none on /dev/pts type devpts (rw,gid=5,mode=620) none on /proc type proc (rw) Using /etc/random-seed to initialize /dev/urandom. INIT: Entering runlevel: 3 Going multiuser... Starting sysklogd daemons: /usr/sbin/syslogd /usr/sbin/klogd -c 3 -x Starting Internet super-server daemon: /usr/sbin/inetd Starting OpenSSH SSH daemon: /usr/sbin/sshd Updating shared library links: /sbin/ldconfig Welcome to Linux 2.6.4-1um (tty0) darkstar login: ---8<--- This is on a nearly vanilla 'minimal' SuSE 9.1 installation. I had to use a .config from another machine in order to compile it, make menuconfig ARC=um on this one produced a .config, but it continually errored on some mtd/sharp.c upon 'make linux modules ARCH=um'. Thanks for the assistance! ----- Original Message ----- From: "Raymond Wong" <ra...@li...> To: "'uml'" <um...@do...>; <use...@li...> Sent: Friday, May 21, 2004 1.52 AM Subject: RE: [uml-user] mem=64M, mem=128M, mem=ANYTHING ignored?? > Can you dump the boot up screen? > > > -----Original Message----- > From: use...@li... > [mailto:use...@li...] On Behalf Of uml > Sent: Friday, May 21, 2004 1:00 PM > To: use...@li... > Subject: [uml-user] mem=64M, mem=128M, mem=ANYTHING ignored?? > > Is there any particular reason that my UML won't recognize this switch? > > It states that it does on the CLI, but inside the UML, it always says that > I've only got about 28M. > > Why? > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > User-mode-linux-user mailing list > Use...@li... > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user > > --- > Incoming mail is certified Virus Free. > Checked by AVG anti-virus system (http://www.grisoft.com). > Version: 6.0.687 / Virus Database: 448 - Release Date: 16/5/2004 > > > --- > Outgoing mail is certified Virus Free. > Checked by AVG anti-virus system (http://www.grisoft.com). > Version: 6.0.687 / Virus Database: 448 - Release Date: 16/5/2004 > > > |
From: Jeff D. <jd...@ad...> - 2004-05-21 17:00:54
|
On Fri, May 21, 2004 at 09:55:34AM -0400, uml wrote: > On node 0 totalpages: 8192 > DMA zone: 8192 pages, LIFO batch:2 > Kernel command line: ubd0=cow1 mem=128M root=98:0 Ummmm, strange. It certainly works here: On node 0 totalpages: 32768 DMA zone: 32768 pages, LIFO batch:8 Kernel command line: ubd0=cow,/home/jdike/roots/debian_22 con0=fd:0,fd:1 con1=none con=pts ssl=pts umid=debian mem=128M root=98:0 That was with 2.6.6. There are no obvious typos in your command line, but it's giving you the default 32M. Can you show us the command line at the shell prompt? Jeff |
From: uml <um...@do...> - 2004-05-21 21:15:39
|
SUCCESS. I wish I had kept the original .config file that was giving me problems because I'd have run a diff against it to see what was throwing the errors. I changed several things at once, however, so I am unsure as to what did it. I'm going to retrace through my steps to see if I can reproduce the error on this system and post what I discover here. I'm going to try to reincorporate -O2 instead of using -O1 as well. Additionally, the UML now shuts down cleanly without any extra 'linux' threads running. Memory is recognized correctly as well and it's running a 2.6.6-1 uml kernel. Thanks for the help and ideas, hopefully I'll be able to contribue some diagnostics for someone else in my position who's willing to search this group. =) ----- Original Message ----- From: "Jeff Dike" <jd...@ad...> To: "uml" <um...@do...> Cc: <use...@li...> Sent: Friday, May 21, 2004 1.47 PM Subject: Re: [uml-user] mem=64M, mem=128M, mem=ANYTHING ignored?? > On Fri, May 21, 2004 at 09:55:34AM -0400, uml wrote: > > On node 0 totalpages: 8192 > > DMA zone: 8192 pages, LIFO batch:2 > > > Kernel command line: ubd0=cow1 mem=128M root=98:0 > > Ummmm, strange. It certainly works here: > > On node 0 totalpages: 32768 > DMA zone: 32768 pages, LIFO batch:8 > > Kernel command line: ubd0=cow,/home/jdike/roots/debian_22 con0=fd:0,fd:1 con1=none con=pts ssl=pts umid=debian mem=128M root=98:0 > > That was with 2.6.6. There are no obvious typos in your command line, but > it's giving you the default 32M. > > Can you show us the command line at the shell prompt? > > Jeff > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > User-mode-linux-user mailing list > Use...@li... > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user > > |
From: roland <for...@gm...> - 2004-05-22 12:54:41
|
success, too. the error definitely seems to be a compile issue. if i compile with -O2, uml just uses the 32M memory default if i compile with -O1, uml uses the param "mem=..." the right way furthermore, with O2 - there are left 5 "linux" processes running, after shutting down uml - if i do a "killall linux" 2 of them being terminated, 3 still left running i need to kill them with "kill -9 pid" - the terminal becomes unusable after shutdown - need to issue an "reset" on that terminal to have it working right again with O1 - there is only 1 "linux" process being left (still bad because i need to kill that to boot again) - the terminal behaves correctly after shutdown mhhh - don`t know how much "improvement" does "O2" in favour of "O1" - but who should we ask now, to help hunting this "bug" ? gcc folks ? regards roland ----- Original Message ----- From: "uml" <um...@do...> To: <use...@li...> Sent: Friday, May 21, 2004 11:22 PM Subject: Re: [uml-user] mem=64M, mem=128M, mem=ANYTHING ignored?? > SUCCESS. > > I wish I had kept the original .config file that was giving me problems > because I'd have run a diff against it to see what was throwing the errors. > > I changed several things at once, however, so I am unsure as to what did it. > I'm going to retrace through my steps to see if I can reproduce the error on > this system and post what I discover here. I'm going to try to > reincorporate -O2 instead of using -O1 as well. > > Additionally, the UML now shuts down cleanly without any extra 'linux' > threads running. Memory is recognized correctly as well and it's running a > 2.6.6-1 uml kernel. > > Thanks for the help and ideas, hopefully I'll be able to contribue some > diagnostics for someone else in my position who's willing to search this > group. =) > > > ----- Original Message ----- > From: "Jeff Dike" <jd...@ad...> > To: "uml" <um...@do...> > Cc: <use...@li...> > Sent: Friday, May 21, 2004 1.47 PM > Subject: Re: [uml-user] mem=64M, mem=128M, mem=ANYTHING ignored?? > > > > On Fri, May 21, 2004 at 09:55:34AM -0400, uml wrote: > > > On node 0 totalpages: 8192 > > > DMA zone: 8192 pages, LIFO batch:2 > > > > > Kernel command line: ubd0=cow1 mem=128M root=98:0 > > > > Ummmm, strange. It certainly works here: > > > > On node 0 totalpages: 32768 > > DMA zone: 32768 pages, LIFO batch:8 > > > > Kernel command line: ubd0=cow,/home/jdike/roots/debian_22 con0=fd:0,fd:1 > con1=none con=pts ssl=pts umid=debian mem=128M root=98:0 > > > > That was with 2.6.6. There are no obvious typos in your command line, but > > it's giving you the default 32M. > > > > Can you show us the command line at the shell prompt? > > > > Jeff > > > > > > ------------------------------------------------------- > > This SF.Net email is sponsored by: Oracle 10g > > Get certified on the hottest thing ever to hit the market... Oracle 10g. > > Take an Oracle 10g class now, and we'll give you the exam FREE. > > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > > _______________________________________________ > > User-mode-linux-user mailing list > > Use...@li... > > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user > > > > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > User-mode-linux-user mailing list > Use...@li... > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user > |
From: uml <um...@do...> - 2004-05-22 14:32:46
|
Wow... I'm going to step through your response -- almost identical to my results! > success, too. Congrats. (= > the error definitely seems to be a compile issue. > if i compile with -O2, uml just uses the 32M memory default > if i compile with -O1, uml uses the param "mem=..." the right way Confirmed. > furthermore, with O2 > - there are left 5 "linux" processes running, after shutting down uml Confirmed. > - if i do a "killall linux" 2 of them being terminated, 3 still left running Confirmed. > i need to kill them with "kill -9 pid" Confirmed. > - the terminal becomes unusable after shutdown - need to issue an "reset" on > that terminal to have it working right again Annoying, but Confirmed. > > with O1 > - there is only 1 "linux" process being left (still bad because i need to kill > that to boot again) > - the terminal behaves correctly after shutdown Ah, now this is where we differ! Mine shuts down cleanly now with no remaining processes. Did you hack the arch/um/Makefile as well? I tried it all four ways: No Makefiles hacked <------ ugh. only root Makefile hacked <------ Partially.. leaves ALL threads only arch/um/Makefile hacked <----- Partially.. leaves fewer threads Both Makefiles hacked <------ Works > mhhh - don`t know how much "improvement" does "O2" in favour of "O1" - but > who should we ask now, to help hunting this "bug" ? > gcc folks ? I'm not sure. I'm at the point now where I can move forward -- here's some quick documentation I wrote on it. https://p0lar.dot1q.net/SuSE-UML-2.6.6-instructions.html I just tried to compile a SMP kernel -- no dice. I'm fortunate enough that I've got a serious amount of hardware here and can do testing pretty quickly. I've set up a script that basically blows away the kernel, re-untars it, repatches it, opens up Menuconfig, hacks the makefiles and then compiles -- total time is a shade under 4 minutes per test now that I know what I'm looking for. |
From: roland <for...@gm...> - 2004-05-23 01:45:44
|
> > that to boot again) > > - the terminal behaves correctly after shutdown > Ah, now this is where we differ! Mine shuts down cleanly now with no > remaining processes. Did you hack the arch/um/Makefile as well? I tried it > all four ways: > No Makefiles hacked <------ ugh. > only root Makefile hacked <------ Partially.. leaves ALL threads > only arch/um/Makefile hacked <----- Partially.. leaves fewer threads > Both Makefiles hacked <------ Works ack! forgot to hack the 2nd makefile - did that: success! no remaining linux process or xterm-window/uml-helper after shutodown ! > I just tried to compile a SMP kernel -- no dice. I'm fortunate enough that > I've got a serious amount of hardware here and can do testing pretty > quickly. I've set up a script that basically blows away the kernel, > re-untars it, repatches it, opens up Menuconfig, hacks the makefiles and > then compiles -- total time is a shade under 4 minutes per test now that I > know what I'm looking for. cool! so you could probably try all that following -O2 options, to get to know, which one causing the problem ? -fforce-mem -foptimize-sibling-calls -fstrength-reduce -fcse-follow-jumps -fcse-skip-blocks -frerun-cse-after-loop -frerun-loop-opt -fgcse -fgcse-lm -fgcse-sm -fdelete-null-pointer-checks -fexpensive-optimizations -fregmov -fschedule-insns -fschedule-insns2 -fsched-interblock -fsched-spec -fcaller-saves -fpeephole2 -freorder-blocks -freorder-functions -fstrict-aliasing -falign-functions -falign-jumps -falign-loops -falign-labels if it`s only one of the compile options causing the problem, you should need < 30 minutes runs to find :) regards roland ----- Original Message ----- From: "uml" <um...@do...> To: <use...@li...> Sent: Saturday, May 22, 2004 4:38 PM Subject: Re: [uml-user] mem=64M, mem=128M, mem=ANYTHING ignored?? > Wow... I'm going to step through your response -- almost identical to my > results! > > > success, too. > Congrats. (= > > > the error definitely seems to be a compile issue. > > if i compile with -O2, uml just uses the 32M memory default > > if i compile with -O1, uml uses the param "mem=..." the right way > Confirmed. > > > furthermore, with O2 > > - there are left 5 "linux" processes running, after shutting down uml > Confirmed. > > > - if i do a "killall linux" 2 of them being terminated, 3 still left > running > Confirmed. > > > i need to kill them with "kill -9 pid" > Confirmed. > > > - the terminal becomes unusable after shutdown - need to issue an "reset" > on > > that terminal to have it working right again > Annoying, but Confirmed. > > > > > with O1 > > - there is only 1 "linux" process being left (still bad because i need to > kill > > that to boot again) > > - the terminal behaves correctly after shutdown > Ah, now this is where we differ! Mine shuts down cleanly now with no > remaining processes. Did you hack the arch/um/Makefile as well? I tried it > all four ways: > No Makefiles hacked <------ ugh. > only root Makefile hacked <------ Partially.. leaves ALL threads > only arch/um/Makefile hacked <----- Partially.. leaves fewer threads > Both Makefiles hacked <------ Works > > > mhhh - don`t know how much "improvement" does "O2" in favour of "O1" - but > > who should we ask now, to help hunting this "bug" ? > > gcc folks ? > I'm not sure. I'm at the point now where I can move forward -- here's some > quick documentation I wrote on it. > https://p0lar.dot1q.net/SuSE-UML-2.6.6-instructions.html > > I just tried to compile a SMP kernel -- no dice. I'm fortunate enough that > I've got a serious amount of hardware here and can do testing pretty > quickly. I've set up a script that basically blows away the kernel, > re-untars it, repatches it, opens up Menuconfig, hacks the makefiles and > then compiles -- total time is a shade under 4 minutes per test now that I > know what I'm looking for. > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > User-mode-linux-user mailing list > Use...@li... > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user > |
From: uml <um...@do...> - 2004-05-23 03:37:45
|
> ack! > > forgot to hack the 2nd makefile - did that: > success! > no remaining linux process or xterm-window/uml-helper after shutodown ! BTW, I finally got my uml_utilities compiled -- forgot readline-devel... $%#$%!!! As usual, it took some digging, but it was certainly listed previously in the mailing list. > cool! so you could probably try all that following -O2 options, to get to know, which one > causing the problem ? I *knew* that was coming.. fortunately, I've set up a script to do just that, but what if it's more than just one? D'oh! > -fforce-mem -foptimize-sibling-calls -fstrength-reduce -fcse-follow-jump s > -fcse-skip-blocks -frerun-cse-after-loop -frerun-loop-opt -fgcse -fgcse- lm > -fgcse-sm -fdelete-null-pointer-checks -fexpensive-optimizations -fregmo v > -fschedule-insns -fschedule-insns2 -fsched-interblock -fsched-spec -fca ller-saves > -fpeephole2 -freorder-blocks -freorder-functions -fstrict-aliasing -fal ign-functions > -falign-jumps -falign-loops -falign-labels 24 x ~5 minutes per test = 120 minutes if I'm efficient at testing -- technically, I can keep compiling one while I'm testing another I suppose -- it would be easier if I could find a way to push the actual RAM the uml SEES back to the host operating the test(s), it would certainly be easy to automate. > if it`s only one of the compile options causing the problem, you should need < 30 minutes > runs to find :) Ah well, if it's for the cause -- the procs are just sitting idle anyway. (= |
From: roland <for...@gm...> - 2004-05-23 10:32:36
|
> 24 x ~5 minutes per test = 120 minutes if I'm efficient at testing -- > technically, I can keep compiling one while I'm testing another I suppose -- > it would be easier if I could find a way to push the actual RAM the uml SEES > back to the host operating the test(s), it would certainly be easy to > automate. i think you don`t need 24 tests - just do it like the "binary search" (http://www.bitesizeinc.net/power.programming.binary.search.html) you try the 1st half of the params - if that works, the second half must have the error. then you bisect until you have left 1 param. hopefully, it`s just one param causing the error :) regards roland |
From: Mark O. <mp...@th...> - 2004-05-24 08:21:11
|
Hi, When you work out the answer can you please let me know or post a patch as I have the same problem here, although mine also ignores umid=anything . I am compiling with gcc 3.4.0 and kernel 2.6.6, with all current uml and skas patches applied :) Thanks Mark roland wrote: >>24 x ~5 minutes per test = 120 minutes if I'm efficient at testing -- >>technically, I can keep compiling one while I'm testing another I suppose -- >>it would be easier if I could find a way to push the actual RAM the uml SEES >>back to the host operating the test(s), it would certainly be easy to >>automate. >> >> >i think you don`t need 24 tests - just do it like the "binary search" >(http://www.bitesizeinc.net/power.programming.binary.search.html) >you try the 1st half of the params - if that works, the second half must have >the error. then you bisect until you have left 1 param. hopefully, it`s just one >param causing the error :) > >regards >roland > > > > >------------------------------------------------------- >This SF.Net email is sponsored by: Oracle 10g >Get certified on the hottest thing ever to hit the market... Oracle 10g. >Take an Oracle 10g class now, and we'll give you the exam FREE. >http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click >_______________________________________________ >User-mode-linux-user mailing list >Use...@li... >https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user > > -- Mark Olliver Thermeon Europe Ltd. e-Card: http://www.thermeoneurope.com/e-Card/mpo Email mp...@th... Web www.thermeoneurope.com Support 0906 515 0908 Int. Support +44 1293 864 341 Support Email su...@th... Sales +44 1293 864 334 Sales Email sa...@th... |
From: roland <for...@gm...> - 2004-05-24 21:10:13
|
mark, there is already a solution: replace -O2 with -O1 in the Makefiles. the todo is just, to find out , which of the -O2 params make uml behave that strange and to find out, why this happens - so we know if it`s a gcc or a uml problem. with -O1 uml seems to work ok. can anyone give a guess, how moch "worse" a "-O1" compiled uml is in comparison to a "-O2" compiled one ? regards roland ----- Original Message ----- From: "Mark Olliver" <mp...@th...> To: <use...@li...> Sent: Monday, May 24, 2004 10:20 AM Subject: Re: [uml-user] mem=64M, mem=128M, mem=ANYTHING ignored?? > Hi, > > When you work out the answer can you please let me know or post a patch > as I have the same problem here, although mine also ignores umid=anything > . > > > I am compiling with gcc 3.4.0 and kernel 2.6.6, with all current uml and > skas patches applied :) > > Thanks > > Mark > > roland wrote: > > >>24 x ~5 minutes per test = 120 minutes if I'm efficient at testing -- > >>technically, I can keep compiling one while I'm testing another I suppose -- > >>it would be easier if I could find a way to push the actual RAM the uml SEES > >>back to the host operating the test(s), it would certainly be easy to > >>automate. > >> > >> > >i think you don`t need 24 tests - just do it like the "binary search" > >(http://www.bitesizeinc.net/power.programming.binary.search.html) > >you try the 1st half of the params - if that works, the second half must have > >the error. then you bisect until you have left 1 param. hopefully, it`s just one > >param causing the error :) > > > >regards > >roland > > > > > > > > > >------------------------------------------------------- > >This SF.Net email is sponsored by: Oracle 10g > >Get certified on the hottest thing ever to hit the market... Oracle 10g. > >Take an Oracle 10g class now, and we'll give you the exam FREE. > >http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > >_______________________________________________ > >User-mode-linux-user mailing list > >Use...@li... > >https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user > > > > > > -- > Mark Olliver > > Thermeon Europe Ltd. > > e-Card: http://www.thermeoneurope.com/e-Card/mpo > > Email mp...@th... > Web www.thermeoneurope.com > > Support 0906 515 0908 > Int. Support +44 1293 864 341 > Support Email su...@th... > > Sales +44 1293 864 334 > Sales Email sa...@th... > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > User-mode-linux-user mailing list > Use...@li... > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user > |
From: Michael R. <mr...@ne...> - 2004-05-22 19:17:09
|
roland wrote: > success, too. > > the error definitely seems to be a compile issue. > if i compile with -O2, uml just uses the 32M memory default > if i compile with -O1, uml uses the param "mem=..." the right way I'm not sure and I'm to lazy to check this out, but IIRC I had the same problems using gcc-3.4.0. > mhhh - don`t know how much "improvement" does "O2" in favour of "O1" - but > who should we ask now, to help hunting this "bug" ? > gcc folks ? Maybe the gcc manual? RTFM? ;-) http://gcc.gnu.org/onlinedocs/gcc-3.3.3/gcc/Optimize-Options.html O1 Options: -fdefer-pop -fmerge-constants -fthread-jumps -floop-optimize -fcrossjumping -fif-conversion -fif-conversion2 -fdelayed-branch -fguess-branch-probability -fcprop-registers O2 Options (O1 options plus the following): -fforce-mem -foptimize-sibling-calls -fstrength-reduce -fcse-follow-jumps -fcse-skip-blocks -frerun-cse-after-loop -frerun-loop-opt -fgcse -fgcse-lm -fgcse-sm -fdelete-null-pointer-checks -fexpensive-optimizations -fregmov -fschedule-insns -fschedule-insns2 -fsched-interblock -fsched-spec -fcaller-saves -fpeephole2 -freorder-blocks -freorder-functions -fstrict-aliasing -falign-functions -falign-jumps -falign-loops -falign-labels Note: First, gcc-3.4.0 is slighlty different. Second, if the problem states only with gcc-3.x (x>=1) and not with gcc-3.0.0 or gcc-2.x, then maybe the bug in uml is triggered by the opcode scheduler in the newer gcc. IIRC correctly, the scheduler changed somewhere in the gcc-3.x line. But as always, I'm not sure. ;-) Michael Roth |
From: uml <um...@do...> - 2004-05-21 18:39:01
|
You bet.. --- 8<--- From the host --- 8<--- linux:/tmp/uml # ./uml1 ubd0=cow1 mem=256M ........ linux:/tmp/uml # ls -lhs 0 drwxr-xr-x 3 root root 264 May 21 09:40 . 1.5K drwxrwxrwt 11 root root 1.7K May 21 14:25 .. 806K -rw-r--r-- 1 root root 211M May 21 14:26 cow1 211M -rw-r--r-- 1 root root 210M May 20 23:41 root_fs_slack9.1 4.0K -rwxr-xr-x 1 root root 436 May 21 01:40 startuml 512 drwxr-xr-x 17 root root 536 Apr 6 00:43 tools 2.1M -rwxr-xr-x 1 root root 2.1M May 21 01:31 uml1 --- 8<--- 'startuml' is a script I was using to fire it up originally (but was causing problems with the tuntap config) so I abandoned it until I can make the other fundamentals work correctly. uml1 = 'linux' renamed so that I can track the processes more easily (i know there should be a PID stored somewhere if I use umid='whatever' but I was eliminating variables). Even if I call the original 'linux' command, I get the same behaviour. --- 8<--- From within the host ---8<--- linux:/tmp/uml # free total used free shared buffers cached Mem: 775472 144276 631196 0 20328 65964 -/+ buffers/cache: 57984 717488 Swap: 1048552 0 1048552 --- 8<--- --- 8<--- from within the UML ---8<--- root@darkstar:~# free total used free shared buffers cached Mem: 29096 18760 10336 0 1028 13084 -/+ buffers/cache: 4648 24448 Swap: 0 0 0 root@darkstar:~# ---8<--- Also, I see this after I do a 'shutdown -h now' from within the UML, might it be related?? From the host: linux:/tmp/uml # ps -A | grep -i "uml1" 2952 pts/26 00:00:00 uml1 2953 pts/26 00:00:00 uml1 linux:/tmp/uml # ueh? Hardware: 2 x AMD Athlon MP 1800, 768MB ECC RAM, 3 x 36GB 15k on RAID0 -- I'm baffled. And I didn't turn on SMP support in the uml kernel either (ok, so I tried originally, but it bombed on compile for one reason or another). (= Perhaps I should recompile? Here are the steps I used: (all files in /tmp to start with) cd /tmp tar -xjvf linux-2.6.4.tar.bz2 cd /tmp/linux-2.6.4 bzcat ../uml-patch-2.6.4-1.bz2 | patch -p1 make menuconfig ARDH=um I verify that under 'Loadable Module Support' it is 'Enabled' but none of the other options such as Module Unloading, Module Versioning Support, or Automatic Kernel module loading are checked. I verifiy that /proc/mm is turned on. I verify that skas support is turned on. I save the .config file. make linux modules ARCH=um <wait wait wait> *poof* - out comes my uml kernel (well at least sometimes -- others, it fails miserably on some sharp.c.../=) Should I be doing this differently? I hope I didn't overwhelm with [useless] information here... ----- Original Message ----- From: "Jeff Dike" <jd...@ad...> To: "uml" <um...@do...> Cc: <use...@li...> Sent: Friday, May 21, 2004 1.47 PM Subject: Re: [uml-user] mem=64M, mem=128M, mem=ANYTHING ignored?? > On Fri, May 21, 2004 at 09:55:34AM -0400, uml wrote: > > On node 0 totalpages: 8192 > > DMA zone: 8192 pages, LIFO batch:2 > > > Kernel command line: ubd0=cow1 mem=128M root=98:0 > > Ummmm, strange. It certainly works here: > > On node 0 totalpages: 32768 > DMA zone: 32768 pages, LIFO batch:8 > > Kernel command line: ubd0=cow,/home/jdike/roots/debian_22 con0=fd:0,fd:1 con1=none con=pts ssl=pts umid=debian mem=128M root=98:0 > > That was with 2.6.6. There are no obvious typos in your command line, but > it's giving you the default 32M. > > Can you show us the command line at the shell prompt? > > Jeff > |
From: uml <um...@do...> - 2004-05-21 21:07:05
|
Ok, I've now got 2.6.6 compiling after I renamed /boot/config-2.6.4-54.5-smp to /boot/config-2.6.4-54.5-smp.bak. That was causing all sorts of issues with the default .config -- any other way around that than my little kludge? At any rate, with or without -O1 or -O2, it still registers 28M. As well, if I enable highmem, it won't compile, so I can't test that. Perhaps I should post my .config? I hate to spam the group with it unnecessarily. ----- Original Message ----- From: "Jeff Dike" <jd...@ad...> To: "uml" <um...@do...> Cc: <use...@li...> Sent: Friday, May 21, 2004 1.47 PM Subject: Re: [uml-user] mem=64M, mem=128M, mem=ANYTHING ignored?? > On Fri, May 21, 2004 at 09:55:34AM -0400, uml wrote: > > On node 0 totalpages: 8192 > > DMA zone: 8192 pages, LIFO batch:2 > > > Kernel command line: ubd0=cow1 mem=128M root=98:0 > > Ummmm, strange. It certainly works here: > > On node 0 totalpages: 32768 > DMA zone: 32768 pages, LIFO batch:8 > > Kernel command line: ubd0=cow,/home/jdike/roots/debian_22 con0=fd:0,fd:1 con1=none con=pts ssl=pts umid=debian mem=128M root=98:0 > > That was with 2.6.6. There are no obvious typos in your command line, but > it's giving you the default 32M. > > Can you show us the command line at the shell prompt? > > Jeff > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > User-mode-linux-user mailing list > Use...@li... > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user > > |
From: uml <um...@do...> - 2004-05-23 05:14:04
|
> -fforce-mem -foptimize-sibling-calls -fstrength-reduce -fcse-follow-jump s > -fcse-skip-blocks -frerun-cse-after-loop -frerun-loop-opt -fgcse -fgcse- lm > -fgcse-sm -fdelete-null-pointer-checks -fexpensive-optimizations -fregmo ve > -fschedule-insns -fschedule-insns2 -fsched-interblock -fsched-spec -fca ller-saves > -fpeephole2 -freorder-blocks -freorder-functions -fstrict-aliasing -fal ign-functions > -falign-jumps -falign-loops -falign-labels > > if it`s only one of the compile options causing the problem, you should need < 30 minutes > runs to find :) here's the modification I made to both the root Makefile and arch/um/Makefile that caused it not to compile, the others may have triggered a few complaints, but still worked -- I've no idea why -O2 didn't generate this error upon compilation, nor why -fschedule-insns2 doesn't generate it as I'm no kernel or C expert. :%s/-O2/-O1 -fschedule-insns/g (vi substitution regex) ---8<--- HOSTCC scripts/sumversion.o scripts/sumversion.c: In function `parse_source_files': scripts/sumversion.c:405: error: unable to find a register to spill in class `AREG' scripts/sumversion.c:405: error: this is the insn: (insn:HI 280 279 294 16 0x4049e0b0 (parallel [ (set (reg:SI 2 ecx [118]) (unspec:SI [ (mem:BLK (reg/v/f:SI 64 [ dir ]) [0 A8]) (reg:QI 103) (const_int 1 [0x1]) (reg:SI 102) ] 20)) (use (reg:SI 19 dirflag)) (clobber (reg/f:SI 5 edi [orig:120 dir ] [120])) (clobber (reg:CC 17 flags)) ]) 473 {strlenqi_1} (insn_list 279 (insn_list:REG_DEP_ANTI 271 (nil))) (expr_list:REG_DEAD (reg:SI 19 dirflag) (expr_list:REG_UNUSED (reg:CC 17 flags) (expr_list:REG_UNUSED (reg/f:SI 5 edi [orig:120 dir ] [120]) (expr_list:REG_EQUAL (unspec:SI [ (mem:BLK (reg/v/f:SI 64 [ dir ]) [0 A8]) (reg:QI 103) (const_int 1 [0x1]) (reg:SI 102) ] 20) (nil)))))) scripts/sumversion.c:405: confused by earlier errors, bailing out make[1]: *** [scripts/sumversion.o] Error 1 make: *** [scripts] Error 2 ---8<--- Here's the line I used elsewhere that worked: (vi substitution regex) :%s/-O2/-O1 -fforce-mem -foptimize-sibling-calls -fstrength-reduce -fcse-fol low-jumps -fcse-skip-blocks -frerun-cse-after-loop -frerun-loop-opt -fgcse - fgcse-lm -fgcse-sm -fdelete-null-pointer-checks -fexpensive-optimizations -f regmove -fschedule-insns2 -fsched-interblock -fsched-spec -fcaller-saves -fp eephole2 -freorder-blocks -freorder-functions -fstrict-aliasing -falign-func tions -falign-jumps -falign-loops -falign-labels/g I don't know what to make of this, but if you want to compile with nearly all of the O2 options, you can just leave that one out I believe, YMMV. |
From: uml <um...@do...> - 2004-05-24 16:52:51
|
What's your exact command line? Have you tried compiling your kernel with -O1 instead of -O2? The one that caused me issues using gcc 3.3.x (not 3.4.x (untested)) was -fschedule-insns. -fschedule-insns2 worked fine, however, for what that's worth. ----- Original Message ----- From: "Mark Olliver" <mp...@th...> To: <use...@li...> Sent: Monday, May 24, 2004 4.20 AM Subject: Re: [uml-user] mem=64M, mem=128M, mem=ANYTHING ignored?? > Hi, > > When you work out the answer can you please let me know or post a patch > as I have the same problem here, although mine also ignores umid=anything > . > > > I am compiling with gcc 3.4.0 and kernel 2.6.6, with all current uml and > skas patches applied :) > > Thanks > > Mark > > roland wrote: > > >>24 x ~5 minutes per test = 120 minutes if I'm efficient at testing -- > >>technically, I can keep compiling one while I'm testing another I suppose -- > >>it would be easier if I could find a way to push the actual RAM the uml SEES > >>back to the host operating the test(s), it would certainly be easy to > >>automate. > >> > >> > >i think you don`t need 24 tests - just do it like the "binary search" > >(http://www.bitesizeinc.net/power.programming.binary.search.html) > >you try the 1st half of the params - if that works, the second half must have > >the error. then you bisect until you have left 1 param. hopefully, it`s just one > >param causing the error :) > > > >regards > >roland > > > > > > > > > >------------------------------------------------------- > >This SF.Net email is sponsored by: Oracle 10g > >Get certified on the hottest thing ever to hit the market... Oracle 10g. > >Take an Oracle 10g class now, and we'll give you the exam FREE. > >http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > >_______________________________________________ > >User-mode-linux-user mailing list > >Use...@li... > >https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user > > > > > > -- > Mark Olliver > > Thermeon Europe Ltd. > > e-Card: http://www.thermeoneurope.com/e-Card/mpo > > Email mp...@th... > Web www.thermeoneurope.com > > Support 0906 515 0908 > Int. Support +44 1293 864 341 > Support Email su...@th... > > Sales +44 1293 864 334 > Sales Email sa...@th... > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > User-mode-linux-user mailing list > Use...@li... > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user > |
From: Michael R. <mr...@ne...> - 2004-05-21 12:42:58
|
uml wrote: > Is there any particular reason that my UML won't recognize this switch? > > It states that it does on the CLI, but inside the UML, it always says that > I've only got about 28M. > > Why? If you compiled your uml using gcc-3.4.0, maybe using gcc-3.x will help. |
From: uml <um...@do...> - 2004-05-21 13:41:11
|
linux:/tmp/uml # gcc -v Reading specs from /usr/lib/gcc-lib/i586-suse-linux/3.3.3/specs Configured with: ../configure --enable-threads=posix --prefix=/usr --with-local- prefix=/usr/local --infodir=/usr/share/info --mandir=/usr/share/man --enable -lan guages=c,c++,f77,objc,java,ada --disable-checking --libdir=/usr/lib --enable -lib gcj --with-gxx-include-dir=/usr/include/g++ --with-slibdir=/lib --with-syste m-zl ib --enable-shared --enable-__cxa_atexit i586-suse-linux Thread model: posix gcc version 3.3.3 (SuSE Linux) Guess that's probably not it then? (= ----- Original Message ----- From: "Michael Roth" <mr...@ne...> To: "uml" <um...@do...> Cc: <use...@li...> Sent: Friday, May 21, 2004 8.41 AM Subject: Re: [uml-user] mem=64M, mem=128M, mem=ANYTHING ignored?? > uml wrote: > > Is there any particular reason that my UML won't recognize this switch? > > > > It states that it does on the CLI, but inside the UML, it always says that > > I've only got about 28M. > > > > Why? > > If you compiled your uml using gcc-3.4.0, maybe using gcc-3.x will help. > > > |
From: Michael R. <mr...@ne...> - 2004-05-20 14:00:13
|
Raymond Wong wrote: > I encountered problem in migrating uml filesystem image from one system to > another since the size of the image is more than 2G. I used scp and cp but > both failed to handle files more than 2G. What would be the best way to > migrate such image? "man split" |