Menu
▾
▴
open-sharedroot-devel
|
From: Marc G. <gr...@at...> - 2009-02-10 19:53:39
|
Hello *, finally we got the first stage of the new version of the osr cluster suite in the preview channel. This will be the first release candidate for the next productive channel. NOTE: Those packages are NOT YET production ready. And they might include bugs as they did not undergo the Q&A. So use them at your own risc. 2nd NOTE: Those packages were only tested with OSR<RHEL5> clusters based on GFS or NFS. OSR<SLES10/ocfs2>,OSR<Fedora/gfs2|nfs> and OSR<RHEL5/ocfs2> will be tested next. This means you shouldn't use those packages with ocfs2 as rootfilesystem. As this is only a preview update I will sum up the most important changes not automated: - Rewritten hardwaredetection: https://bugzilla.atix.de/show_bug.cgi?id=325 - Usability review: https://bugzilla.atix.de/show_bug.cgi?id=323 - Support for breakpoints in bootprocess - Break shell will always continue the bootprocess when exited - Bootparameters may be overwritten in Breakshell - Rootfs Check with bootparameter rootfsck (use with care and up to now only with GFS) - NFS OSR will not start scsi detection - Preview support for OSR on fedora (NFS) - Preview support for OSR<NFS4> without authentication - Preview support for OSR<RHEL5 with localfs> - Many small bugfixes Patches from Gordan Bobic (all preview status): - glusterfs support - Software RAID with md support - diet patch (new option to mkinitrd <-l> to make initrd smaller) Please let me know what you think Thanks Marc P.S. May the root always be shared! ;-) -- Gruss / Regards, Marc Grimme http://www.atix.de/ http://www.open-sharedroot.org/ |
|
From: Gordan B. <go...@bo...> - 2009-02-12 02:51:58
|
Marc Grimme wrote: > - glusterfs support One minor issue here, in the glusterfs rpm list there should be fuse-lib package included. Sorry if I sent a broken patch. :( Gordan |
|
From: Marc G. <gr...@at...> - 2009-02-12 09:13:53
|
On Thursday 12 February 2009 03:51:50 Gordan Bobic wrote: > Marc Grimme wrote: > > - glusterfs support > > One minor issue here, in the glusterfs rpm list there should be fuse-lib > package included. Sorry if I sent a broken patch. :( No problem. I'll update it. But how about your hw-detection does it work now stable? Cause my cluster that had the problems is running just fine. Marc. -- Gruss / Regards, Marc Grimme http://www.atix.de/ http://www.open-sharedroot.org/ |
|
From: Gordan B. <go...@bo...> - 2009-02-12 10:30:03
|
I'll try to get around to trying it out over the weekend. That particular cluster is a production one so I couldn't just bounce it. I have, however, tried the new updates (including the extras packages for glusterfs and md) on the cluster I'm building at the moment, and it all seems to work just fine. :) Gordan On Thu, 12 Feb 2009 10:13:38 +0100, Marc Grimme <gr...@at...> wrote: > On Thursday 12 February 2009 03:51:50 Gordan Bobic wrote: >> Marc Grimme wrote: >> > - glusterfs support >> >> One minor issue here, in the glusterfs rpm list there should be fuse-lib >> package included. Sorry if I sent a broken patch. :( > No problem. I'll update it. > But how about your hw-detection does it work now stable? Cause my cluster > that > had the problems is running just fine. > > Marc. |
|
From: Gordan B. <go...@bo...> - 2009-02-13 13:31:56
|
On Thu, 12 Feb 2009 10:13:38 +0100, Marc Grimme <gr...@at...> wrote: > But how about your hw-detection does it work now stable? Cause my cluster > that had the problems is running just fine. I've not tested the new versions on the cluster that was having problems, but I have tested it on another cluster, and although it does seem to work fine in general, the new probing approach managed to crash the machine once. I'll test it some more since I've only seen that happen once (after a reboot it came up OK). A few observations - the first probing pass doesn't clean up and unload modules properly. The dependencies among the drivers don't get resolved, so, for example, scsi disk drivers don't get unloaded properly because they have dependencies, which causes errors to get reported back. The other immediate observation is that the probing is very slow compared to the old image. It takes 2-3 times longer to get to the final init. The fact that I have a lot of SATA ports (10) in that system probably doesn't help. So, what would be really nice is a way to only do probing if configuration information isn't available (e.g. modprobe.conf^H^H^H^H^H^H^H^H^H^H^H^H^H^H driver= parameter in cluster.conf ;-)) - and by this I am referring to disk controller drivers in addition to the NIC drivers. Multi-pass probing is all well and good for a default approach, but it would be really nice to be able to skip it for performance reasons with the configuration explicitly specified. Gordan |
|
From: Marc G. <gr...@at...> - 2009-02-13 14:08:11
|
On Friday 13 February 2009 12:33:06 Gordan Bobic wrote: > On Thu, 12 Feb 2009 10:13:38 +0100, Marc Grimme <gr...@at...> wrote: > > But how about your hw-detection does it work now stable? Cause my cluster > > that had the problems is running just fine. > > I've not tested the new versions on the cluster that was having problems, > but I have tested it on another cluster, and although it does seem to work > fine in general, the new probing approach managed to crash the machine > once. I'll test it some more since I've only seen that happen once (after a > reboot it came up OK). ;-) > > A few observations - the first probing pass doesn't clean up and unload > modules properly. The dependencies among the drivers don't get resolved, > so, for example, scsi disk drivers don't get unloaded properly because they > have dependencies, which causes errors to get reported back. > > The other immediate observation is that the probing is very slow compared > to the old image. It takes 2-3 times longer to get to the final init. The > fact that I have a lot of SATA ports (10) in that system probably doesn't > help. > > So, what would be really nice is a way to only do probing if configuration > information isn't available (e.g. modprobe.conf^H^H^H^H^H^H^H^H^H^H^H^H^H^H > driver= parameter in cluster.conf ;-)) - and by this I am referring to disk > controller drivers in addition to the NIC drivers. Multi-pass probing is > all well and good for a default approach, but it would be really nice to be > able to skip it for performance reasons with the configuration explicitly > specified. Hm. I see your point. What about this if and only if the @driver is specified for every node in the cluster.conf we are using these drivers to detect the MAC Addresses and skip everything else? The about no loading scsidrivers and the like is you cannot selectively trigger udev. Or I didn't find a way. -- Gruss / Regards, Marc Grimme http://www.atix.de/ http://www.open-sharedroot.org/ |
|
From: Gordan B. <go...@bo...> - 2009-02-13 14:23:39
|
On Fri, 13 Feb 2009 15:08:04 +0100, Marc Grimme <gr...@at...> wrote: >> > But how about your hw-detection does it work now stable? Cause my >> > cluster >> > that had the problems is running just fine. >> >> I've not tested the new versions on the cluster that was having problems, >> but I have tested it on another cluster, and although it does seem to >> work >> fine in general, the new probing approach managed to crash the machine >> once. I'll test it some more since I've only seen that happen once (after >> a reboot it came up OK). > > ;-) Yeah, it's a bit deja-vu after the reason for the improved probing was that booting would randomly fail. ;-) >> A few observations - the first probing pass doesn't clean up and unload >> modules properly. The dependencies among the drivers don't get resolved, >> so, for example, scsi disk drivers don't get unloaded properly because >> they have dependencies, which causes errors to get reported back. >> >> The other immediate observation is that the probing is very slow compared >> to the old image. It takes 2-3 times longer to get to the final init. The >> fact that I have a lot of SATA ports (10) in that system probably doesn't >> help. >> >> So, what would be really nice is a way to only do probing if >> configuration information isn't available (e.g. >> modprobe.conf^H^H^H^H^H^H^H^H^H^H^H^H^H^H >> driver= parameter in cluster.conf ;-)) - and by this I am referring to >> disk controller drivers in addition to the NIC drivers. Multi-pass probing is >> all well and good for a default approach, but it would be really nice to >> be able to skip it for performance reasons with the configuration explicitly >> specified. > > Hm. I see your point. > > What about this if and only if the @driver is specified for every node in > the cluster.conf we are using these drivers to detect the MAC Addresses and > skip everything else? Sounds good. :) > The about no loading scsidrivers and the like is you cannot selectively > trigger udev. Or I didn't find a way. Fair enough. But if the probing isn't triggered iif the NIC @driver is specified, then that should avoid the probing anyway. :) Gordan |
|
From: Marc G. <gr...@at...> - 2009-03-27 16:18:14
|
Hello *, I finished the last implementations and bugfixes of the (nearly) final release candidate of the new comoonics tools (Version 4.5 including comoonics-bootimage). Here are the last relevant changes: * Some bugfixes in hardware detection specially with udev renaming nics and therefore "deadlocking" udevd (timeout gets called) * Bugfixes in powering up nics. They might be called multiple times (i.e. when using bridging and bonding and vlans at once ;-) ) * Verified ocfs2 with RHEL5 and ocfs2 with SLES10 * Updated the lite option in mkinitrd <gordans approch> to be more general and not necessarily need "lsmod". * Implemented filters to globally remove files from initrd (Gordan's idea) * Implemented an update feature in the initrd (see https://bugzilla.atix.de/show_bug.cgi?id=340). That's a cool one. Updating an initrd within seconds. * Fixed bugs (338, 331, ..) Some thoughts to sum up: 1. It is now possible to use the lite version of mkinitrd. It should reduce the size of the initrd by 50%. Use -L if you want to include all loaded modules or if you need more modules use either -M <modulename> or specify in /etc/comoonics/comoonics-bootimage.cfg 2. The mkinitrd -U updates an existing initrd an therefore also speeds up the initrd building. With -A <kernelversion> and -D <kernelversion> even kernels can be exchanged in the same initrd. So that's it. Open issues before having a proper RC is: Retest with SLES10(ocfs2/nfs) and test with FC10(nfs/gfs2). That's it, let me know what you think and have fun. NOTE: Those packages are NOT YET production ready. And they might include bugs as they did not undergo the Q&A. So use them at your own risc. P.S. And may the root always be shared! ;-) On Tuesday 10 February 2009 20:53:18 Marc Grimme wrote: > Hello *, > finally we got the first stage of the new version of the osr cluster suite > in the preview channel. This will be the first release candidate for the > next productive channel. > > NOTE: Those packages are NOT YET production ready. And they might include > bugs as they did not undergo the Q&A. So use them at your own risc. > > 2nd NOTE: Those packages were only tested with OSR<RHEL5> clusters based on > GFS or NFS. OSR<SLES10/ocfs2>,OSR<Fedora/gfs2|nfs> and OSR<RHEL5/ocfs2> > will be tested next. This means you shouldn't use those packages with ocfs2 > as rootfilesystem. > > As this is only a preview update I will sum up the most important changes > not automated: > > - Rewritten hardwaredetection: https://bugzilla.atix.de/show_bug.cgi?id=325 > - Usability review: https://bugzilla.atix.de/show_bug.cgi?id=323 > - Support for breakpoints in bootprocess > - Break shell will always continue the bootprocess when exited > - Bootparameters may be overwritten in Breakshell > - Rootfs Check with bootparameter rootfsck (use with care and up to now > only with GFS) > - NFS OSR will not start scsi detection > - Preview support for OSR on fedora (NFS) > - Preview support for OSR<NFS4> without authentication > - Preview support for OSR<RHEL5 with localfs> > - Many small bugfixes > Patches from Gordan Bobic (all preview status): > - glusterfs support > - Software RAID with md support > - diet patch (new option to mkinitrd <-l> to make initrd smaller) > > Please let me know what you think > > Thanks > Marc > > P.S. May the root always be shared! ;-) > > -- > Gruss / Regards, > > Marc Grimme > http://www.atix.de/ http://www.open-sharedroot.org/ > > > --------------------------------------------------------------------------- >--- Create and Deploy Rich Internet Apps outside the browser with > Adobe(R)AIR(TM) software. With Adobe AIR, Ajax developers can use existing > skills and code to build responsive, highly engaging applications that > combine the power of local resources and data with the reach of the web. > Download the Adobe AIR SDK and Ajax docs to start building applications > today-http://p.sf.net/sfu/adobe-com > _______________________________________________ > Open-sharedroot-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/open-sharedroot-users -- Gruss / Regards, Marc Grimme http://www.atix.de/ http://www.open-sharedroot.org/ |
|
From: Gordan B. <go...@bo...> - 2009-03-27 20:42:11
|
Marc Grimme wrote: > * Updated the lite option in mkinitrd <gordans approch> to be more general and > not necessarily need "lsmod". I'm curious - how do you get away from using lsmod to see what's needed? Or are you going to make me chance "yum update" and hope it works just to see the code? ;) > * Implemented filters to globally remove files from initrd (Gordan's idea) Is this due to (or coupled with) the approach of dumping all of the required files from the rpm database in one pass, rather than querying each packake in a separate rpm call? > * Implemented an update feature in the initrd (see > https://bugzilla.atix.de/show_bug.cgi?id=340). That's a cool one. Updating an > initrd within seconds. Why is this faster than rebuilding from scratch? What gets skipped? Compression stage has to remain, so the other big win can only be the rpm extraction query stage. If this was implemented in the single pass, then is it really that big a win? > Some thoughts to sum up: > 1. It is now possible to use the lite version of mkinitrd. It should reduce > the size of the initrd by 50%. > Use -L if you want to include all loaded modules or if you need more modules > use either -M <modulename> or specify > in /etc/comoonics/comoonics-bootimage.cfg How do you determine what modules are necessary vs what modules are currently loaded? Some of them are likely to be ambiguos, depending on the use case and rootfs type. Relying on what's loaded in the running system seems safer (except for the initial migration from standalone to shared root, but that might as well be a full fat initrd to get the system bootstrapped). Gordan |
|
From: Marc G. <gr...@at...> - 2009-03-30 07:49:09
|
On Friday 27 March 2009 21:41:37 Gordan Bobic wrote:
> Marc Grimme wrote:
> > * Updated the lite option in mkinitrd <gordans approch> to be more
> > general and not necessarily need "lsmod".
>
> I'm curious - how do you get away from using lsmod to see what's needed?
> Or are you going to make me chance "yum update" and hope it works just
> to see the code? ;)
It's (at least in my view) very easy. I'm just querying all involved parts to
give a list of the drivers they require. This is triggered by the function
get_min_modules(chroot-lib.sh). This function triggers more functions from
the dependent modules like i.e. storage_get_drivers, ${clutype}_get_drivers,
${rootfs}_get_drivers .. (but I didn't integrated drbd, glusterfs and iscsi
yet, but I'll do it today). Then all drivers found in cluster.conf are
included. These are all drivers found as a driver attribute in any path in
the initrd. Finally you can define more drivers either by commandline (-M) or
in /etc/comoonics/bootimage.cfg.
To validate which modules get loaded into initrd you can invoke mkinitrd
with -V.
>
> > * Implemented filters to globally remove files from initrd (Gordan's
> > idea)
>
> Is this due to (or coupled with) the approach of dumping all of the
> required files from the rpm database in one pass, rather than querying
> each packake in a separate rpm call?
It were my action points taken from the discussion. One was the single pass
rpm and the other was having global filters.
I tried to implement it with single pass rpm but didn't succeed with gaining
any performance from it. So I revoked the changed for the single pass rpm
(they were too ugly) and stayed with the "additional" global filters. So we
now have rpm filters that are given per rpm and filters applied globally
(like *.pyc).
>
> > * Implemented an update feature in the initrd (see
> > https://bugzilla.atix.de/show_bug.cgi?id=340). That's a cool one.
> > Updating an initrd within seconds.
>
> Why is this faster than rebuilding from scratch? What gets skipped?
> Compression stage has to remain, so the other big win can only be the
> rpm extraction query stage. If this was implemented in the single pass,
> then is it really that big a win?
First yes. The rpm single pass is no big win. Both setups take 1.11min
(aprox.). The problem is bash but I recognized it later.
I broke the single pass rpm because rpm single pass is nearly exactly the same
speed then the multi pass.
But when playing with the update functionality I implemented a copy based on a
filelist for all files in the initrd with modification timestamps saved
(see .index.list in initrd). The copy ran throught the pregenerated list and
only copied the files that have changed. At this time I had ca. 2000 files in
the initrd. The virtual machine took 40 secs to run throught this process
(with 2000 files). Then I implemented the same functions with python and it
took 4 seconds (see std-lib.sh vs. stdlib.py). So my finding is that bash is
the bottleneck.
So right now the python implementation is only used in the update process and
that takes the time it needs to unpack the initrd copy the files and kernel
if need be (-A, -D) and then pack it again. That's like up to 10 times faster
then building the initrd.
On the long run I have to think about implementing hot spots in the mkinitrd
and initrd process in high level languages (python) as they seem to be much
faster.
>
> > Some thoughts to sum up:
> > 1. It is now possible to use the lite version of mkinitrd. It should
> > reduce the size of the initrd by 50%.
> > Use -L if you want to include all loaded modules or if you need more
> > modules use either -M <modulename> or specify
> > in /etc/comoonics/comoonics-bootimage.cfg
>
> How do you determine what modules are necessary vs what modules are
> currently loaded? Some of them are likely to be ambiguos, depending on
> the use case and rootfs type. Relying on what's loaded in the running
> system seems safer (except for the initial migration from standalone to
> shared root, but that might as well be a full fat initrd to get the
> system bootstrapped).
See above.
All the same when using lite initrd I reduced the size (one kernel in the
initrd) from 45MB to 24MB (including the global filters). That's at least
something ;-) . And that can still be optimized cause right now only *.py[co]
and /usr/share/doc /usr/share/man are excluded.
Marc.
--
Gruss / Regards,
Marc Grimme
http://www.atix.de/ http://www.open-sharedroot.org/
|
|
From: Gordan B. <go...@bo...> - 2009-03-30 12:13:59
|
On Mon, 30 Mar 2009 08:48:54 +0100, Marc Grimme <gr...@at...> wrote: >> > * Implemented filters to globally remove files from initrd (Gordan's >> > idea) >> >> Is this due to (or coupled with) the approach of dumping all of the >> required files from the rpm database in one pass, rather than querying >> each packake in a separate rpm call? > It were my action points taken from the discussion. One was the single pass > rpm and the other was having global filters. > I tried to implement it with single pass rpm but didn't succeed with > gaining > any performance from it. So I revoked the changed for the single pass rpm > (they were too ugly) and stayed with the "additional" global filters. So we > now have rpm filters that are given per rpm and filters applied globally > (like *.pyc). Can you send me the single-pass implementation you tried? In my preliminary tests the single pass was MASSIVELY faster. Here are my figures: # rpm -qa > packages.list Current equivalent: # time for package in `cat packages.list`; do rpm -q --list $package >> files.list; done real 1m4.470s user 0m13.644s sys 0m4.498s Single pass: # time rpm -q --list `cat packages.list` > files.list real 0m2.501s user 0m1.853s sys 0m0.582s Theoretical best case: # time rpm -qa --list > files.list real 0m2.220s user 0m1.588s sys 0m0.568s The above list included about 700 packages. While this is way more than the initrd will ever contain, it does illustrate that there are potential savings to be made here. Whatever other bottlenecks may affect this, at least rpm time will still get cut by about 25-fold according to these figures. Gordan |
Thanks for helping keep SourceForge clean.
X