open-sharedroot-devel

[OSR-devel] GlusterFS and mdadm mods

[Gordan B. <go...@bo...>]

Hi,

Attached are:

My latest clusterfs-lib.sh that includes a temporary bodge for a problem 
with glusterfs that makes operations on the FS fail for a few seconds 
after mounting because fuse takes a little while to settle. So the bodge 
is to include "sleep 5; ls -la <mount-point>" to make sure that the real 
  root is ready by the time we mount cdsl.local. It's a patched version 
of the latest package that was preview (I haven't seen any updates to 
preview in at least a few weeks).

Also attached is hardware-lib.sh that has a different approach to mdadm 
stuff (this is, AFAIK, only really useful for gluster at the moment). 
Instead of:

if [ -f /etc/mdadm.conf ]

it checks for

if [ -x /sbin/mdadm ]

because I've been having problems with mdadm.conf contents - it looks 
like despite them being cdsl-ed out to cdsl.local, the node that builds 
the initrd hard-puts it's own mdadm.conf into the initrd.
Anyway, instead of looking for a pre-existing mdadm.conf, the patch 
makes a new one:

mdadm --examine --scan > /etc/mdadm.conf

This file should now be removed from the package:
/etc/comoonics/bootimage/files.initrd.d/mdadm.list
(or at least be empty), since it only contained /etc/mdadm.conf

Thanks.

Gordan

Re: [OSR-devel] GlusterFS and mdadm mods

[Gordan B. <go...@bo...>]

Ooops, forgot to actually attach the added files. *headdesk*
Here they are.

Gordan Bobic wrote:
> Hi,
> 
> Attached are:
> 
> My latest clusterfs-lib.sh that includes a temporary bodge for a problem 
> with glusterfs that makes operations on the FS fail for a few seconds 
> after mounting because fuse takes a little while to settle. So the bodge 
> is to include "sleep 5; ls -la <mount-point>" to make sure that the real 
>  root is ready by the time we mount cdsl.local. It's a patched version 
> of the latest package that was preview (I haven't seen any updates to 
> preview in at least a few weeks).
> 
> Also attached is hardware-lib.sh that has a different approach to mdadm 
> stuff (this is, AFAIK, only really useful for gluster at the moment). 
> Instead of:
> 
> if [ -f /etc/mdadm.conf ]
> 
> it checks for
> 
> if [ -x /sbin/mdadm ]
> 
> because I've been having problems with mdadm.conf contents - it looks 
> like despite them being cdsl-ed out to cdsl.local, the node that builds 
> the initrd hard-puts it's own mdadm.conf into the initrd.
> Anyway, instead of looking for a pre-existing mdadm.conf, the patch 
> makes a new one:
> 
> mdadm --examine --scan > /etc/mdadm.conf
> 
> This file should now be removed from the package:
> /etc/comoonics/bootimage/files.initrd.d/mdadm.list
> (or at least be empty), since it only contained /etc/mdadm.conf
> 
> Thanks.
> 
> Gordan
> 

Re: [OSR-devel] GlusterFS and mdadm mods

[Marc G. <gr...@at...>]

Gordan,
hardware-lib.sh -> upstream
clusterfs-lib.sh ;-) 
+
+  
#############################################################################
+  # GlusterFS settle time bug bodge!
+  sleep 5
+  ls -la ${mountpoint} > /dev/null
+  ls -la ${mountpoint}/${prefix} > /dev/null
+  
#############################################################################
+

I simply cannot do this.
I could add a sleep with a timeout defaults 0 but which can be overwritten as 
bootparameter.
But do you really need those ls?
Marc.

On Wednesday 08 July 2009 23:17:24 Gordan Bobic wrote:
> Ooops, forgot to actually attach the added files. *headdesk*
> Here they are.
>
> Gordan Bobic wrote:
> > Hi,
> >
> > Attached are:
> >
> > My latest clusterfs-lib.sh that includes a temporary bodge for a problem
> > with glusterfs that makes operations on the FS fail for a few seconds
> > after mounting because fuse takes a little while to settle. So the bodge
> > is to include "sleep 5; ls -la <mount-point>" to make sure that the real
> >  root is ready by the time we mount cdsl.local. It's a patched version
> > of the latest package that was preview (I haven't seen any updates to
> > preview in at least a few weeks).
> >
> > Also attached is hardware-lib.sh that has a different approach to mdadm
> > stuff (this is, AFAIK, only really useful for gluster at the moment).
> > Instead of:
> >
> > if [ -f /etc/mdadm.conf ]
> >
> > it checks for
> >
> > if [ -x /sbin/mdadm ]
> >
> > because I've been having problems with mdadm.conf contents - it looks
> > like despite them being cdsl-ed out to cdsl.local, the node that builds
> > the initrd hard-puts it's own mdadm.conf into the initrd.
> > Anyway, instead of looking for a pre-existing mdadm.conf, the patch
> > makes a new one:
> >
> > mdadm --examine --scan > /etc/mdadm.conf
> >
> > This file should now be removed from the package:
> > /etc/comoonics/bootimage/files.initrd.d/mdadm.list
> > (or at least be empty), since it only contained /etc/mdadm.conf
> >
> > Thanks.
> >
> > Gordan



-- 
Gruss / Regards,

Marc Grimme
http://www.atix.de/               http://www.open-sharedroot.org/

Re: [OSR-devel] GlusterFS and mdadm mods

[Gordan B. <go...@bo...>]

On Thu, 23 Jul 2009 15:27:22 +0200, Marc Grimme <gr...@at...> wrote:
> Gordan,
> hardware-lib.sh -> upstream
> clusterfs-lib.sh ;-) 
> +
> +  
>
#############################################################################
> +  # GlusterFS settle time bug bodge!
> +  sleep 5
> +  ls -la ${mountpoint} > /dev/null
> +  ls -la ${mountpoint}/${prefix} > /dev/null
> +  
>
#############################################################################
> +
> 
> I simply cannot do this.

I know, it's horrible.

> I could add a sleep with a timeout defaults 0 but which can be
overwritten
> as bootparameter.

Yes, I guess that would be fine.

> But do you really need those ls?

I suspect that those aren't necessary any more with the latest version of
GlusterFS. There used to be a bug that caused first post-mounting access to
always fail, but I think this has been fixed now.

The simple fact is that it is a GlusterFS issue, not an OSR issue, so I
fully understand the justification for not including this horrible
workaround at all. I only bothered posting it because there was no fix time
given.

Gordan

Re: [OSR-devel] GlusterFS and mdadm mods

[Gordan B. <go...@bo...>]

On 23/07/2009 14:53, Gordan Bobic wrote:
> On Thu, 23 Jul 2009 15:27:22 +0200, Marc Grimme<gr...@at...>  wrote:
>> Gordan,
>> hardware-lib.sh ->  upstream
>> clusterfs-lib.sh ;-)
>> +
>> +
>>
> #############################################################################
>> +  # GlusterFS settle time bug bodge!
>> +  sleep 5
>> +  ls -la ${mountpoint}>  /dev/null
>> +  ls -la ${mountpoint}/${prefix}>  /dev/null
>> +
>>
> #############################################################################
>> +
>>
>> I simply cannot do this.
>
> I know, it's horrible.
>
>> I could add a sleep with a timeout defaults 0 but which can be
> overwritten
>> as bootparameter.
>
> Yes, I guess that would be fine.
>
>> But do you really need those ls?
>
> I suspect that those aren't necessary any more with the latest version of
> GlusterFS. There used to be a bug that caused first post-mounting access to
> always fail, but I think this has been fixed now.
>
> The simple fact is that it is a GlusterFS issue, not an OSR issue, so I
> fully understand the justification for not including this horrible
> workaround at all. I only bothered posting it because there was no fix time
> given.

Just an update/confirmation - it looks like the problem requiring this 
workaround has been fixed in gluster since at least 2.0.8 (current 
version at the moment is 2.0.9). Those horrible "ls" lines aren't needed 
any more. :) But since they were never included in clusterfs-lib.sh, I 
guess it doesn't really matter. ;)

Gordan

Re: [OSR-devel] GlusterFS and mdadm - fsck patch

[Gordan B. <go...@bo...>]

Since GlusterFS is backed by a local rather than a cluster file system, 
it is possible to fsck the file system properly before mounting. Here is 
a patch that implements fscking of the file system based on the same 
criteria as in RHEL5 rc.sysinit (mount counts, unclean shutdowns, etc.).

I haven't tested it with a broken file system yet, but fsck seems to do 
a cursory glance at the fs and if it's clean, things just move on, so 
please treat this is as experimental at the moment.

Most of this is ripped straight out of RHEL5 rc.sysinit, but it doesn't 
do the checking of things like fastboot, etc.

The reason I did this is because I've had a problem in the past where a 
small amount of corruption crept in somehow (possibly through an unclean 
power-down) and caused problems.

Gordan