Menu
▾
▴
devil-linux-discuss
|
From: Dick M. <di...@li...> - 2008-04-09 16:29:32
|
Hi, I just thought I'd throw this in just as you're all working away with the detail of the first 1.4 ;-) Have just done a few system updates I find one thing irritates me: that is the process of copying files from the old to new config. It's the feeling of uncertainty about what's happening - what's being copied that shouldn't be and, more important, what isn't being copied but should be. To be fair the only problem I've had are with sym-links in /etc/rcN.d ... but it still troubles me. I thought there might be a better way. A while ago I was mucking about with unionfs (http://www.am-utils.org/project-unionfs.html) and it seems to me this might give a more straightforward way of doing the job. Unionfs can be thought of as a change-on-write layer over a read-only filesystem. It provides the capability to appear to change the contents of, say, a DVD; the changes actually being stored elsewhere. For DL this would allow the standard /etc to be put on release cdrom image and admin configurations to be layered over the top at will. The changes only can then easily be copied to the config media with save-config and restored on reboot. For system upgrades there is no need for any additional action. All the admin's files remain unchanged but the new default files are there underneath if needed. There is another possible benefit too. It would allow those who wish to add addition programs or libraries to the system directories (such as dovecot's cmusieve plugin) to do just that. I'm not sure what the implications are for security or how that can be controlled but the principle of 'reboot to restore' would not be broken. What do you think? Dick |
|
From: Serge L. <fi...@in...> - 2008-04-09 19:33:27
|
Hi, Dick Middleton wrote: > Hi, > > I just thought I'd throw this in just as you're all working away with the > detail of the first 1.4 ;-) > ... > I thought there might be a better way. A while ago I was mucking about with > unionfs (http://www.am-utils.org/project-unionfs.html) and it seems to me this > might give a more straightforward way of doing the job. ... > What do you think? unionfs is a part of DL 1.3 and we try to keep the package up-to-date. There are no scripts for DL with unonfs usage examples of course but you can use it anyway. I use it often due to my laziness to rebuild ISO :-) -- Sincerely, Serge Leschinsky |
|
From: Dick M. <di...@li...> - 2008-04-09 19:45:37
|
Serge, > unionfs is a part of DL 1.3 and we try to keep the package up-to-date. There are > no scripts for DL with unonfs usage examples of course but you can use it > anyway. I use it often due to my laziness to rebuild ISO :-) Doesn't seem to be part of my system. There's no kernel module and no tools. No sign of aufs either. Is it called something else? I'm using: v1.3.4-2008-04-02-i586 Dick |
|
From: Bruce S. <bw...@ar...> - 2008-04-09 20:03:13
|
> > unionfs is a part of DL 1.3 and we try to keep the package up-to-date. There are > > no scripts for DL with unonfs usage examples of course but you can use it > > anyway. I use it often due to my laziness to rebuild ISO :-) > > Doesn't seem to be part of my system. There's no kernel module and no tools. No > sign of aufs either. Is it called something else? > > I'm using: v1.3.4-2008-04-02-i586 It's not selected for inclusion in the standard build by default, but it's definitely part of 1.3. Does anyone know if it compiles cleanly? As soon as Heiko checks in the latest openldap update, I plan on compiling a fresh 1.3, and will select unionfs. I can upload the ISO to the ftp server if there is any interest. - BS |
|
From: Heiko Z. <he...@zu...> - 2008-04-09 20:08:30
|
Quoting Bruce Smith <bw...@ar...>: >> > unionfs is a part of DL 1.3 and we try to keep the package >> up-to-date. There are >> > no scripts for DL with unonfs usage examples of course but you can use it >> > anyway. I use it often due to my laziness to rebuild ISO :-) >> >> Doesn't seem to be part of my system. There's no kernel module and >> no tools. No >> sign of aufs either. Is it called something else? >> >> I'm using: v1.3.4-2008-04-02-i586 > > It's not selected for inclusion in the standard build by default, but > it's definitely part of 1.3. Does anyone know if it compiles cleanly? > > As soon as Heiko checks in the latest openldap update, I plan on > compiling a fresh 1.3, and will select unionfs. I can upload the ISO to > the ftp server if there is any interest. Guess what, openldap broke nagios. We'll see what else. I hope I can finish it today. -- Regards Heiko Zuerker http://www.devil-linux.org ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. |
|
From: Serge L. <fi...@in...> - 2008-04-09 20:22:00
|
Bruce Smith wrote: >>> unionfs is a part of DL 1.3 and we try to keep the package up-to-date. There are >>> no scripts for DL with unonfs usage examples of course but you can use it >>> anyway. I use it often due to my laziness to rebuild ISO :-) >> Doesn't seem to be part of my system. There's no kernel module and no tools. No >> sign of aufs either. Is it called something else? >> >> I'm using: v1.3.4-2008-04-02-i586 > > It's not selected for inclusion in the standard build by default, but > it's definitely part of 1.3. Does anyone know if it compiles cleanly? I know (I use it). It compiles cleanly. -- Serge |
|
From: Dick M. <di...@li...> - 2008-04-09 20:31:33
|
Bruce, >>> unionfs is a part of DL 1.3 > As soon as Heiko checks in the latest openldap update, I plan on > compiling a fresh 1.3, and will select unionfs. I can upload the ISO to > the ftp server if there is any interest. I'm interested but I'm not sure what I'd do with it - my DL is a live system. So it's qualified interest. I'd much rather some enthusiastic developer leapt on the idea and changed the scripts for a later release. Dick |
|
From: Bruce S. <bw...@ar...> - 2008-04-09 20:55:00
|
> >>> unionfs is a part of DL 1.3 > > > As soon as Heiko checks in the latest openldap update, I plan on > > compiling a fresh 1.3, and will select unionfs. I can upload the ISO to > > the ftp server if there is any interest. > > I'm interested but I'm not sure what I'd do with it - my DL is a live system. So > it's qualified interest. I'd much rather some enthusiastic developer leapt on > the idea and changed the scripts for a later release. I like the idea, but I've never used unionfs. The docs for unionfs seem to be kind of scarce. Can you post some examples of how you use it? If I understand your suggestion before, the /etc directory would be un-tar'ed on the CD. Then a unionfs in memory would be a branch above it and all the additions/changes are made there? How do we modify save-config to only save the changes in the upper modified branch? Can we just save that directory? (I need to play with unionfs, except it's not included in my 1.3 either :) Another advantage of doing this is the size of the config file saved to the floppy/usb would be a LOT smaller! - BS |
|
From: Serge L. <fi...@in...> - 2008-04-10 06:04:59
|
Bruce, Bruce Smith wrote: >>>>> unionfs is a part of DL 1.3 >>> As soon as Heiko checks in the latest openldap update, I plan on >>> compiling a fresh 1.3, and will select unionfs. I can upload the ISO to >>> the ftp server if there is any interest. >> I'm interested but I'm not sure what I'd do with it - my DL is a live system. So >> it's qualified interest. I'd much rather some enthusiastic developer leapt on >> the idea and changed the scripts for a later release. > > I like the idea, but I've never used unionfs. The docs for unionfs seem > to be kind of scarce. Can you post some examples of how you use it? The most detail documentation is the kernel docs, i.e. linux-2.6.xx/Documentation/filesystems/unionfs > > How do we modify save-config to only save the changes in the upper > modified branch? Can we just save that directory? (I need to play with > unionfs, except it's not included in my 1.3 either :) Yes. we can. United dirs may be saved an restored. > > Another advantage of doing this is the size of the config file saved to > the floppy/usb would be a LOT smaller! Disadvantage of that exists also. Our boot logic based on the config file searching and parsing config files. If it was union/au fs based solution I'm not sure we will able to keep the logic unchanged. -- Serge |
|
From: Bruce S. <br...@ar...> - 2008-04-10 12:45:40
|
> > Another advantage of doing this is the size of the config file saved to > > the floppy/usb would be a LOT smaller! > Disadvantage of that exists also. Our boot logic based on the config file > searching and parsing config files. If it was union/au fs based solution I'm not > sure we will able to keep the logic unchanged. Right, we would have to change the boot process to mount /etc from CDROM and then mount the unionfs on top of that and untar the changes there. We'd also have to change save-config to only save the unionfs. There would also have to be changes to the "upgrade to a new release" logic. This may also make 1.2 and 1.4 not compatible with each other's config. So, I guess this is a good time to do it, if we're going to. I guess it comes down to asking Heiko if this is the direction we want to go? I think it's a nice solution to be managing changes in /etc and making the saved config much smaller. Much better than tar'ring up some of the large, rarely used, packages in /etc and putting them on the CD (as we talked about doing some time ago). If Heiko okay's this change, I'll volunteer to do the work. Heiko?? :-) - BS |
|
From: Heiko Z. <he...@zu...> - 2008-04-10 13:03:54
|
-- Regards Heiko Zuerker http://www.devil-linux.org Quoting Bruce Smith <br...@ar...>: >> > Another advantage of doing this is the size of the config file saved to >> > the floppy/usb would be a LOT smaller! >> Disadvantage of that exists also. Our boot logic based on the config file >> searching and parsing config files. If it was union/au fs based >> solution I'm not >> sure we will able to keep the logic unchanged. > > Right, we would have to change the boot process to mount /etc from CDROM > and then mount the unionfs on top of that and untar the changes there. > > We'd also have to change save-config to only save the unionfs. There > would also have to be changes to the "upgrade to a new release" logic. > > This may also make 1.2 and 1.4 not compatible with each other's config. > So, I guess this is a good time to do it, if we're going to. > > I guess it comes down to asking Heiko if this is the direction we want > to go? I think it's a nice solution to be managing changes in /etc and > making the saved config much smaller. Much better than tar'ring up some > of the large, rarely used, packages in /etc and putting them on the CD > (as we talked about doing some time ago). > > If Heiko okay's this change, I'll volunteer to do the work. Heiko?? :-) Yes lets give this a shot. The size of the /etc.tar.bz2 keeps bothering me and this would really be a help. Then we can also get rid of all the stupid symlinks in /etc. Heiko ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. |
|
From: Serge L. <fi...@in...> - 2008-04-10 13:28:48
|
Bruce Smith wrote: >>> Another advantage of doing this is the size of the config file saved to >>> the floppy/usb would be a LOT smaller! >> Disadvantage of that exists also. Our boot logic based on the config file >> searching and parsing config files. If it was union/au fs based solution I'm not >> sure we will able to keep the logic unchanged. > > Right, we would have to change the boot process to mount /etc from CDROM > and then mount the unionfs on top of that and untar the changes there. :-) Which comes first the chicken or the egg? To be able to mount CDROM we have to find out the config file and load modules from /etc/sysconfi/config:INITRD_MODULES ... Another way is to do something like I did in install-on-hdd, i.e. load all modules, but include into initrd/ramfs only really necessary. Bruce, don't take me wrong, I like the idea. I only try to define the points we should discuss and modify. -- Serge |
|
From: Bruce S. <bw...@ar...> - 2008-04-10 13:33:39
|
> > Right, we would have to change the boot process to mount /etc from CDROM > > and then mount the unionfs on top of that and untar the changes there. > :-) Which comes first the chicken or the egg? > To be able to mount CDROM we have to find out the config file and load modules > from /etc/sysconfi/config:INITRD_MODULES ... Right, that will be a problem with anything other than a standard IDE CDROM drive. > Another way is to do something like I did in install-on-hdd, i.e. load all > modules, but include into initrd/ramfs only really necessary. > > Bruce, don't take me wrong, I like the idea. I only try to define the points we > should discuss and modify. It's a good point. Can we load the config directory changes before the underlying /etc directory on the CD? That would let us read sysconfig/config ... - BS |
|
From: Heiko Z. <he...@zu...> - 2008-04-10 13:41:35
|
Quoting Serge Leschinsky <fi...@in...>: > Bruce Smith wrote: >>>> Another advantage of doing this is the size of the config file saved to >>>> the floppy/usb would be a LOT smaller! >>> Disadvantage of that exists also. Our boot logic based on the config file >>> searching and parsing config files. If it was union/au fs based >>> solution I'm not >>> sure we will able to keep the logic unchanged. >> >> Right, we would have to change the boot process to mount /etc from CDROM >> and then mount the unionfs on top of that and untar the changes there. > :-) Which comes first the chicken or the egg? > To be able to mount CDROM we have to find out the config file and > load modules > from /etc/sysconfi/config:INITRD_MODULES ... > > Another way is to do something like I did in install-on-hdd, i.e. load all > modules, but include into initrd/ramfs only really necessary. > > Bruce, don't take me wrong, I like the idea. I only try to define > the points we > should discuss and modify. > Actually the 2.6 kernel is pretty good in detecting which modules to load, so we may not need to worry about this anymore. -- Regards Heiko Zuerker http://www.devil-linux.org ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. |
|
From: Serge L. <fi...@in...> - 2008-04-10 14:15:06
|
Heiko Zuerker wrote: > Quoting Serge Leschinsky <fi...@in...>: > >> Bruce Smith wrote: >>>>> Another advantage of doing this is the size of the config file saved to >>>>> the floppy/usb would be a LOT smaller! >>>> Disadvantage of that exists also. Our boot logic based on the config file >>>> searching and parsing config files. If it was union/au fs based >>>> solution I'm not >>>> sure we will able to keep the logic unchanged. >>> Right, we would have to change the boot process to mount /etc from CDROM >>> and then mount the unionfs on top of that and untar the changes there. >> :-) Which comes first the chicken or the egg? >> To be able to mount CDROM we have to find out the config file and >> load modules >> from /etc/sysconfi/config:INITRD_MODULES ... >> >> Another way is to do something like I did in install-on-hdd, i.e. load all >> modules, but include into initrd/ramfs only really necessary. >> >> Bruce, don't take me wrong, I like the idea. I only try to define >> the points we >> should discuss and modify. >> > > Actually the 2.6 kernel is pretty good in detecting which modules to > load, so we may not need to worry about this anymore. > Heiko, I'm afraid it's not so good as we expect. To check it you may want to install DL on SATA HDD - most probably it will not boot (conf media not found, DL cdrom not found). The problem is in busybox's mdev (I think so) that is why Ubuntu Live-CD has original udev binaries linked against uclibc, not busybox. But I agree - it will be fixed sooner or later. The question is "when". -- Serge |
|
From: Bruce S. <bw...@ar...> - 2008-04-10 14:23:59
|
> > Actually the 2.6 kernel is pretty good in detecting which modules to > > load, so we may not need to worry about this anymore. > > > Heiko, I'm afraid it's not so good as we expect. To check it you may want to > install DL on SATA HDD - most probably it will not boot (conf media not found, > DL cdrom not found). The problem is in busybox's mdev (I think so) that is why > Ubuntu Live-CD has original udev binaries linked against uclibc, not busybox. What about a SATA CDROM? Would that be a good test? I have one system with a SATA CD drive, that I could try to boot DL on. - BS |
|
From: Serge L. <fi...@in...> - 2008-04-10 14:32:02
|
Bruce Smith wrote: >>> Actually the 2.6 kernel is pretty good in detecting which modules to >>> load, so we may not need to worry about this anymore. >>> >> Heiko, I'm afraid it's not so good as we expect. To check it you may want to >> install DL on SATA HDD - most probably it will not boot (conf media not found, >> DL cdrom not found). The problem is in busybox's mdev (I think so) that is why >> Ubuntu Live-CD has original udev binaries linked against uclibc, not busybox. > > What about a SATA CDROM? Would that be a good test? I suppose yes. My expectation is that you will get "DL cd not found" but I would be happy to be wrong. -- Serge |
|
From: Serge L. <fi...@in...> - 2008-04-16 13:45:41
|
Bruce, Bruce Smith wrote: >>> Actually the 2.6 kernel is pretty good in detecting which modules to >>> load, so we may not need to worry about this anymore. >>> >> Heiko, I'm afraid it's not so good as we expect. To check it you may want to >> install DL on SATA HDD - most probably it will not boot (conf media not found, >> DL cdrom not found). The problem is in busybox's mdev (I think so) that is why >> Ubuntu Live-CD has original udev binaries linked against uclibc, not busybox. > > What about a SATA CDROM? Would that be a good test? > > I have one system with a SATA CD drive, that I could try to boot DL on. I checked busybox web page and found out that something was done for mdev. Maybe the problem with hardware detection is really away. I'm going to try version 1.9.2 - current stable. 12 February 2008 -- BusyBox 1.9.1 (stable) BusyBox 1.9.1. (svn, patches, how to add a patch) This is a bugfix-only release, with fixes to fsck, iproute, mdev, mkswap, msh, nameif, stty, test, zcip. -- Sincerely, Serge |
|
From: Bruce S. <bw...@ar...> - 2008-04-16 14:30:07
|
> >>> Actually the 2.6 kernel is pretty good in detecting which modules to > >>> load, so we may not need to worry about this anymore. > >>> > >> Heiko, I'm afraid it's not so good as we expect. To check it you may want to > >> install DL on SATA HDD - most probably it will not boot (conf media not found, > >> DL cdrom not found). The problem is in busybox's mdev (I think so) that is why > >> Ubuntu Live-CD has original udev binaries linked against uclibc, not busybox. > > > > What about a SATA CDROM? Would that be a good test? > > > > I have one system with a SATA CD drive, that I could try to boot DL on. > > I checked busybox web page and found out that something was done for mdev. Maybe > the problem with hardware detection is really away. I'm going to try version > 1.9.2 - current stable. I haven't had a chance to try booting a DL CD on my SATA CD machine yet. Hopefully I'll remember tonight, when I get home from work. I'll let you know ... > 12 February 2008 -- BusyBox 1.9.1 (stable) > BusyBox 1.9.1. (svn, patches, how to add a patch) > > This is a bugfix-only release, with fixes to fsck, iproute, mdev, mkswap, msh, > nameif, stty, test, zcip. - BS |
|
From: Serge L. <fi...@in...> - 2008-04-16 15:53:00
|
Bruce Smith wrote: > I haven't had a chance to try booting a DL CD on my SATA CD machine yet. > Hopefully I'll remember tonight, when I get home from work. > > I'll let you know ... Thank you, Bruce. -- Sincerely, Serge |
|
From: Bruce S. <bw...@ar...> - 2008-04-17 02:29:19
|
> > I haven't had a chance to try booting a DL CD on my SATA CD machine yet. > > Hopefully I'll remember tonight, when I get home from work. > > > > I'll let you know ... > > Thank you, Bruce. It looks for the saved config, can't find any, asks if I want to continue without a config, I answer "y", and then it says: !!! Devil-Linux CD-ROM not found !!! Please check your Hardware! Booting will NOT continue, you have to reset to try again - BS |
|
From: Serge L. <fi...@in...> - 2008-04-09 20:11:20
|
Dick Middleton wrote: > Serge, > >> unionfs is a part of DL 1.3 and we try to keep the package up-to-date. There are >> no scripts for DL with unonfs usage examples of course but you can use it >> anyway. I use it often due to my laziness to rebuild ISO :-) > > Doesn't seem to be part of my system. There's no kernel module and no tools. No > sign of aufs either. Is it called something else? > > I'm using: v1.3.4-2008-04-02-i586 Hm.. Quick research and the answer was found out: [root@z-dev build]# grep -r CONFIG_UNIONF scripts/* scripts/configuration/unionfs.config:menu_add "System|Filesystems" bool scripts/configuration/profiles/default:CONFIG_UNIONFS=n It's disabled by default. -- Serge Leschinsky |
|
From: Heiko Z. <he...@zu...> - 2008-04-10 13:31:08
|
Quoting Bruce Smith <bw...@ar...>: >> both unionfs and aufs are std in Debian if you've got a deb desktop. > > What's the difference between unionfs & aufs? Does DL have aufs? > Which one should we use for this? I don't know what the difference is. Aufs is not in DL right now. If aufs is really more stable and maybe better maintained, then we should go for that one. We also need to choose wisely, so we don't have to wait for a patch for a new kernel for a couple of months. -- Regards Heiko Zuerker http://www.devil-linux.org ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. |
|
From: Bruce S. <bw...@ar...> - 2008-04-10 13:35:49
|
> >> both unionfs and aufs are std in Debian if you've got a deb desktop. > > > > What's the difference between unionfs & aufs? Does DL have aufs? > > Which one should we use for this? > > I don't know what the difference is. Aufs is not in DL right now. > If aufs is really more stable and maybe better maintained, then we > should go for that one. > We also need to choose wisely, so we don't have to wait for a patch > for a new kernel for a couple of months. I looked at both web pages, and unionfs impressed me as being better maintained than aufs. I don't know the difference in functionality. (yet :) I also noticed another alternative that is not part of the kernel, but uses FUSE. (for the name now) - BS |
|
From: Dick M. <di...@li...> - 2008-04-10 15:51:06
|
Heiko Zuerker wrote:
> Quoting Bruce Smith <bw...@ar...>:
>
>>> both unionfs and aufs are std in Debian if you've got a deb desktop.
>> What's the difference between unionfs & aufs? Does DL have aufs?
>> Which one should we use for this?
>
> I don't know what the difference is. Aufs is not in DL right now.
> If aufs is really more stable and maybe better maintained, then we
> should go for that one.
I don't know either. The aufs people understandably advocate their offering
enthusiastically. See below.
However I use Debian sid amd64 and I've not had unionfs working for a long while
for one reason or another whereas aufs worked first time out of the box so I'm
favourably disposed towards it.
Aufs claim their offering is smaller lighter, more reliable and better featured.
Since we're only going to use a basic part of the functionality I doubt it
will make much difference.
> We also need to choose wisely, so we don't have to wait for a patch
> for a new kernel for a couple of months.
unionfs says it is in Andrew Mortons patches but I think that's as near either
are to being included in the kernel.
Dick
Why to use AuFS instead of unionfs:
-----------------------------------
I am an AuFS user for a long time and what I really appreciate
(from the user's point of view) is the following:
# AuFS supports writable branch balancing. That means, you can setup several
partitions for writing and AuFS will split all new/modified files between them,
based on free disk space, existence of parent directory, randomly, or combinations.
# AuFS supports huge amount of branches. I'm currently using hundreds of
branches without just a small slowdown (which is obvious).
# AuFS provides a list of branches through /sys, which doesn't have the
limitation like /proc/mounts. For that reason, it works correctly even with
thousand of branches (while so much branches would break /proc/mounts at all).
# AuFS implements 'rr' branch mode, it means 'really-readonly'. This is really
useful, particularly for ISO images or SquashFS filesystems as a brach, as AuFS
doesn't need to re-lookup those filesystems. (You know, a readonly branch 'ro'
can be modified from another place, eg. network, so there can occur a 'direct
branch access' even for read-only directories and AuFS handles it correctly.)
# last, but not the least, AuFS is really stable in real world situations. I
used unionfs in the past, but my second name for it was 'NULL POINTER
DEREFERENCE'. I can see those errors still happening in latest unionfs as well,
last one I've found is from 27th of May 2008 ... BUG: unable to handle kernel
NULL pointer dereference. ... I have absolutely no idea what that means, but the
same errors keep appearing in unionfs for years. You won't see anything like
that in AuFS. Guess why knoppix and other projects switched to it :)
Tomas M
slax.org
|
