Re: [OSR-devel] iSCSI Root

[Gordan B. <go...@bo...>]

On Fri, 12 Oct 2007, Marc Grimme wrote:

>>>> Now, in theory, I should be able to bring up another node on the same
>>>> file system. All I would need to do is clone the /boot partition to the
>>>> other box, and it should just come up.
>>>
>>> Why cloning it and not using the same. Isn't that possible. We are always
>>> doing it this way.
>>
>> Because I'm not booting this off DHCP. I'm booting the kernel and the
>> initrd off the local disk. So I need to clone the boot partition with the
>> kernel and the initrd to each of the nodes.
>
> ok.
> How about PXE. IMHO you could use one shared bootimage couldn't you?

Sure I could - but I'd still prefer to reclaim all the initrd memory. No 
point in wasting it when I have an 80GB RAID1 local disk that'll only ever 
get used for swap and /tmp.

>>>> As far as unsharing things under /var, I _think_ only /var/lock actually
>>>> needs to be unshared. Can I do this with the running image with:
>>>>
>>>> com-mkcdsl -r / -a /var/lock
>>>
>>> you can skip the -r/ it is default.
>>> How about /var/run, /var/log, /var/cache, /var/tmp, /var/spool. All of
>>> these normally need to be hostdependent.
>>
>> I'm not sure why /var/cache and /var/spool would need to be host
>> dependent. I can see reasons why I'd want to them to be shared.
>
> I think e.g. /var/spool/mail or just from the name it should be. But it's up
> to you.

I would _definitely_ prefer to have /var/spool/mail shared. More to the 
point, I'm planning to use this cluster for a big mail system with 
maildirs, so it'd better work! :-p

>> I agree that /var/run and /var/lock should be private.
>>
>> It would be _nice_ to have a shared /var/log, but from past experience,
>> the logs will get messed up when multiple syslogs try to write to them.
>> Is there a shared logging solution for this? I know I can pick a master
>> log node and get syslog pointed at this, but this won't work for all the
>> other non-syslog services (e.g. Apache).
>
> Why did I want to say (use a syslog-server)? Right with apache it does not
> work. For e.g. apache we've written a log analysis tool to merge the logs.
> It's in the addons channel and is called mgrep.
> I think I also read a howto integrate apache into syslog somewhere.

Or there is a Spread based Apache logging system. I know there are 
workarounds. Shame logging doesn't work as atomic writes - that would 
have made things much easier for this scenario... :-(

>> I plan to link /var/tmp to /tmp, and have /tmp mounted to a big local
>> partition (local disks are only planned to have /boot, /tmp and swap).
>>
>> Which brings me to the next question - how do I use a local disk partition
>> instead of the initrd? What's the procedure for that? It seems a more
>> efficient solution than relying on a ramdisk that eats memory after
>> booting up when there is plenty of local disk space available. How do I
>> use /etc/sysconfig/comoonics-chroot ?
>
> Yes. So I suppose you don't want to configure your local disk with lvm ;-) .

LOL! I'd prefer not. :-)

> So I'll explain it without.
> It's basically quite easy:
> 1. For every node: spare one partition for the chroot (let's say it
> is /dev/sda4) and let it be at least 500M.
> 2. For every node: mkfs.ext3 /dev/sda4
> 3. Add to the com_info section for every node the following:
> <chrootenv mountpoint="/var/comoonics/chroot" fstype="ext3" device="/dev/sda4"
> chrootdir="/var/comoonics/chroot"/>
> 4. Make a new initrd
> 5. reboot every node
> That's it no everything should be running on your local disk instead of tmpfs.

OK - how does this work, then? Does it copy the initrd to the disk at 
boot time? Or does the mkinitrd build the init root straight on that 
partition? Or does something else happen? What does 
/etc/sysconfig/comoonics-chroot do, then? I thought it had some part to 
play in this.

Gordan