From: Pipkins, J. <Jef...@po...> - 2007-03-20 15:40:41
|
Is there a way to have an embedded system running just one kernel (i.e., = not UML), but still using a cow file as the root file system? The reason I ask, is that I'm thinking I want to use UML during = development, but not ship it as part of an embedded system. But the use = of the cow file is very attractive, especially since the system can be = restored to factory defaults simply by deleting a file. So, does the use of a cow file necessarily require a UML guest kernel? = Or can I use the same technique with an ordinary linux kernel? Thanks, --Jeff |
From: Jeff D. <jd...@ad...> - 2007-03-20 17:32:59
|
On Tue, Mar 20, 2007 at 10:40:26AM -0500, Pipkins, Jeff wrote: > So, does the use of a cow file necessarily require a UML guest kernel? Short answer - yes. Longer answer - the UML COW stuff can be split out from the ubd driver and used in a native kernel, but no one has done so. Jeff -- Work email - jdike at linux dot intel dot com |
From: Anselm L. <ans...@li...> - 2007-03-20 23:18:21
|
Pipkins, Jeff wrote: > So, does the use of a cow file necessarily require a UML guest kernel? Or > can I use the same technique with an ordinary linux kernel? In theory, you can use the device mapper for COW functionality with any Linux file system. Whether this is something you want to get into on an embedded system is a different question altogether. Anselm -- Anselm Lingnau ... Linup Front GmbH ... Linux-, Open-Source- & Netz-Schulungen Linup Front GmbH, Postfach 100121, 64201 Darmstadt, Germany ans...@li..., +49(0)6151-9067-103, Fax -299, www.linupfront.de |
From: Christopher M. <chr...@ya...> - 2007-03-21 14:17:32
|
Jeff: I don't know if you can use the same code with a host kernel (that manages cow files in guest kernels), but I can think of at least one way to achieve the same effect. The unionfs filesystem allows you to stack filesystems on top of each other. Say you have two directories A, and B, and you mount them on top of each other at directory C. C's contents are then the union of the files in A and B. When you declare, during the mount, that A is read-only and B is read-write, the effect is that any modifications to the contents of C are stored in B without molesting the contents of A. If you delete a file f1 in C (that really exists in A/f1), the unionfs driver handles that by creating a "whiteout file" in B (B/.wh.f1) that indicates that f1 should no longer exist in the mount at C. If you rewrite a portion of a file in C (that really exists in A), the modified file is written to B. This has lots of applications. I somtimes use it to see exactly what files a program is attempting to modify by mounting the root directory as part of a read-only stack at /usr/root, using /usr/root-rw as the read-write part of the stack. I then chroot into /usr/root, run the program, and examine the contents of /usr/root-rw to see what files were modified / deleted. In your case, you can alter the boot process by writing a linuxrc file inside an initrd.gz to setup a union mount of your root filesystem using a ramdisk as a read-write stack and the real root filesystem as a read-only stack. After every reboot, any changes to the root filesystem will disappear, and new changes will be stored in the ramdisk. This very technique is how some live-CD or live-DVD distributions of linux work. If you are interested, I have a bash script I use to create the necessary files for doing this with slackware. There were some nasty details I had to work out that took me a while to figure out. It may save you some time if you decide to go the same route. Chris Marshall --- "Pipkins, Jeff" <Jef...@po...> wrote: > Is there a way to have an embedded system running just one kernel (i.e., not UML), but still > using a cow file as the root file system? > > The reason I ask, is that I'm thinking I want to use UML during development, but not ship it as > part of an embedded system. But the use of the cow file is very attractive, especially since > the system can be restored to factory defaults simply by deleting a file. > > So, does the use of a cow file necessarily require a UML guest kernel? Or can I use the same > technique with an ordinary linux kernel? > > Thanks, > --Jeff > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV> _______________________________________________ > User-mode-linux-user mailing list > Use...@li... > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user > |
From: Pipkins, J. <Jef...@po...> - 2007-03-21 15:04:16
|
Fascinating. Yes, I'd love to get a copy of your shell script. It = would be interesting to explore this some more, regardless of whether I = end up using it for this particular application. Thanks, --Jeff -----Original Message----- From: Christopher Marshall [mailto:chr...@ya...] Sent: Wed 2007-03-21 9:17 AM To: Pipkins, Jeff; use...@li... Subject: Re: [uml-user] UML for embedded system development =20 Jeff: I don't know if you can use the same code with a host kernel (that = manages cow files in guest kernels), but I can think of at least one way to achieve the same = effect. The unionfs filesystem allows you to stack filesystems on top of each = other. Say you have two directories A, and B, and you mount them on top of each other at = directory C. C's contents are then the union of the files in A and B. When you declare, during the = mount, that A is read-only and B is read-write, the effect is that any modifications to the = contents of C are stored in B without molesting the contents of A. If you delete a file f1 in C (that = really exists in A/f1), the unionfs driver handles that by creating a "whiteout file" in B = (B/.wh.f1) that indicates that f1 should no longer exist in the mount at C. If you rewrite a portion = of a file in C (that really exists in A), the modified file is written to B. This has lots of applications. I somtimes use it to see exactly what = files a program is attempting to modify by mounting the root directory as part of a = read-only stack at /usr/root, using /usr/root-rw as the read-write part of the stack. I then chroot = into /usr/root, run the program, and examine the contents of /usr/root-rw to see what files were = modified / deleted. In your case, you can alter the boot process by writing a linuxrc file = inside an initrd.gz to setup a union mount of your root filesystem using a ramdisk as a = read-write stack and the real root filesystem as a read-only stack. After every reboot, any changes = to the root filesystem will disappear, and new changes will be stored in the ramdisk. This very technique is how some live-CD or live-DVD distributions of = linux work. If you are interested, I have a bash script I use to create the = necessary files for doing this with slackware. There were some nasty details I had to work out that = took me a while to figure out. It may save you some time if you decide to go the same route. Chris Marshall --- "Pipkins, Jeff" <Jef...@po...> wrote: > Is there a way to have an embedded system running just one kernel = (i.e., not UML), but still > using a cow file as the root file system? >=20 > The reason I ask, is that I'm thinking I want to use UML during = development, but not ship it as > part of an embedded system. But the use of the cow file is very = attractive, especially since > the system can be restored to factory defaults simply by deleting a = file. >=20 > So, does the use of a cow file necessarily require a UML guest kernel? = Or can I use the same > technique with an ordinary linux kernel? >=20 > Thanks, > --Jeff > > = -------------------------------------------------------------------------= > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to = share your > opinions on IT & business topics through brief surveys-and earn cash > = http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D= DEVDEV> _______________________________________________ > User-mode-linux-user mailing list > Use...@li... > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user >=20 |
From: Christopher M. <chr...@ya...> - 2007-03-22 16:33:46
|
--- "Pipkins, Jeff" <Jef...@po...> wrote: > Fascinating. Yes, I'd love to get a copy of your shell script. It would be interesting to > explore this some more, regardless of whether I end up using it for this particular application. > Thanks, > --Jeff > > -----Original Message----- > From: Christopher Marshall [mailto:chr...@ya...] > Sent: Wed 2007-03-21 9:17 AM > To: Pipkins, Jeff; use...@li... > Subject: Re: [uml-user] UML for embedded system development > > Jeff: > > I don't know if you can use the same code with a host kernel (that manages cow files in guest > kernels), but I can think of at least one way to achieve the same effect. > > The unionfs filesystem allows you to stack filesystems on top of each other. Say you have two > directories A, and B, and you mount them on top of each other at directory C. C's contents are > then the union of the files in A and B. When you declare, during the mount, that A is read-only > and B is read-write, the effect is that any modifications to the contents of C are stored in B > without molesting the contents of A. If you delete a file f1 in C (that really exists in A/f1), > the unionfs driver handles that by creating a "whiteout file" in B (B/.wh.f1) that indicates > that > f1 should no longer exist in the mount at C. If you rewrite a portion of a file in C (that > really > exists in A), the modified file is written to B. > > This has lots of applications. I somtimes use it to see exactly what files a program is > attempting to modify by mounting the root directory as part of a read-only stack at /usr/root, > using /usr/root-rw as the read-write part of the stack. I then chroot into /usr/root, run the > program, and examine the contents of /usr/root-rw to see what files were modified / deleted. > > In your case, you can alter the boot process by writing a linuxrc file inside an initrd.gz to > setup a union mount of your root filesystem using a ramdisk as a read-write stack and the real > root filesystem as a read-only stack. After every reboot, any changes to the root filesystem > will > disappear, and new changes will be stored in the ramdisk. > > This very technique is how some live-CD or live-DVD distributions of linux work. > > If you are interested, I have a bash script I use to create the necessary files for doing this > with slackware. There were some nasty details I had to work out that took me a while to figure > out. It may save you some time if you decide to go the same route. > > Chris Marshall > > --- "Pipkins, Jeff" <Jef...@po...> wrote: > > > Is there a way to have an embedded system running just one kernel (i.e., not UML), but still > > using a cow file as the root file system? > > > > The reason I ask, is that I'm thinking I want to use UML during development, but not ship it > as > > part of an embedded system. But the use of the cow file is very attractive, especially since > > the system can be restored to factory defaults simply by deleting a file. > > > > So, does the use of a cow file necessarily require a UML guest kernel? Or can I use the same > > technique with an ordinary linux kernel? > > > > Thanks, > > --Jeff > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > opinions on IT & business topics through brief surveys-and earn cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV> > _______________________________________________ > > User-mode-linux-user mailing list > > Use...@li... > > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user > > > > > Jeff: Here is the script I used to work out the details of setting up a unionfs mount of the root filesystem, where the root filesystem is mounted read-only with a ramdisk acting as the read-write area. This is the basic technique several live-CD distributions of linux use. Let's call is make_union_initrd.sh. Here's the text: #!/bin/bash dev=$1 if [ -z "$1" ] ; then echo "usage: <hd device>" exit 1 fi tree=/boot/initrd-tree mkinitrd -c -k 2.6.10 -m reiserfs:unionfs mkdir ${tree}/memory mkdir ${tree}/union mkdir ${tree}/union/root mkdir ${tree}/union/live mkdir ${tree}/union/live/initrd cp rc.S ${tree} ( echo "#!/bin/ash" echo echo "mount -n proc /proc -t proc" echo ". ./load_kernel_modules" echo echo "mkdir /memory" echo "mount -n -t tmpfs none /memory" echo "mkdir /memory/changes" echo echo "mkdir /union" echo "mkdir /union/root" echo "mkdir /union/live" echo echo "mount -n ${dev} /union/root" echo "mount -n -t unionfs none /union/live -o dirs=/memory/changes=rw:/union/root=ro" echo "mkdir /union/live/initrd" echo "cat /rc.S > /union/live/etc/rc.d/rc.S" echo "cd /union/live" echo echo "echo \"unmounting proc\"" echo "/bin/umount /proc" echo "/bin/ash" echo echo "pivot_root . initrd" echo "exec /usr/sbin/chroot . sbin/init -i <dev/console >dev/console 2>&1" ) > /boot/initrd-tree/linuxrc chmod a+rx /boot/initrd-tree/linuxrc mkinitrd mv /boot/initrd.gz /boot/initrd-2.6.10-union.gz This version of the script assumes you are booting from a root partition on a hard drive and you want to use unionfs to mount the root partition as read-only. My reason for setting it up that way was to make it easy to change something and see the effects. If I were working with creating bootable CDs, I would have to burn another CD every time I did another experiment, instead of simply rebooting from a hard drive. Assuming you are running it from on a slackware workstation the normally boots from /dev/hda6, you would invoke the script like this: make_union_initrd /dev/hda6 Then you would have to add a stanza to your lilo.conf using the resulting initrz.gz to test it. In slackware, there is a script for making initrd files called mkinitrd. Close to the top of this script, I call it with arguments specifying that I want the resierfs and unionfs modules pulled into the initrd.gz for kernel version 2.6.10, which is what I was running back when I wrote it. mkinitrd creates the directory tree it uses to create the initrd at /boot/initrd-tree. If you call mkinitrd without arguments, it takes whatever is in the tree and sticks it into /boot/initrd.gz. My script, then, calls mkinitrd toward the top to create the directory at /boot/initrd-tree, makes some mount points inside /boot/initrd-tree, writes the linuxrc script, and calls mkinitrd toward the end to capture the contents of /boot/initrd-tree in /boot/initrd.gz. Toward the middle of the script you'll notice it writes a line into linuxrc "/bin/ash". That causes the booting kernel to give you an ash prompt in the middle of the process of executing linuxrc so you can check on things. I used this quite a bit during debugging. Once you are booting and hit that point and are done checking things, you need to hit control-d to exit cleanly from it. If you type "exit", you'll derail the boot process. The other tricky parts are the exact mount command that sets up the unionfs stack, and the pivot_root + chroot sequence at the end. Notice the "-i" argument to sbin/init. That forces init to act as if it were booting the OS and not trying to communicate the an already running init. I had to look through the source code for init to discover the -i argument. You won't find it in the man page for init. For some reason I never determined, sbin/init in my linuxrc thinks it is not booting the OS but trying to communicate with another init process so I have to force it this way. Another script detail. The script assumes you have the file rc.S sitting in the directory you invoke it from. make_union_initrd.sh copies rc.S into the initrd.gz, where linuxrc copies it into the root mount it is about to boot (after the union mount is set up). rc.S is the slackware boot script responsible for running fsck on the root filesystem from time to time. fsck will always fail when run against a filesystem mounted read-only (which it will be mounted as, after the union mount is setup by linuxrc). So I had to modify it by removing the fsck attempt. You'll have to do something similar for whatever distribution of linux you are working with. I hope that is enough to get you started. Chris Marshall |
From: Jeff D. <jd...@ad...> - 2007-03-21 16:21:44
|
On Wed, Mar 21, 2007 at 07:17:16AM -0700, Christopher Marshall wrote: > The unionfs filesystem allows you to stack filesystems on top of each other. Yup, this is file-by-file COW-ing rather than block-level COW-ing. Same idea, different level. Also, it's in -mm, I think, but not mainline, and there are enough architectural questions about it that I think it's not going to mainline soon. Jeff -- Work email - jdike at linux dot intel dot com |