Re: [OSR-devel] Bug reports / fixes and iSCSI

[Gordan B. <go...@bo...>]

On Wed, 10 Oct 2007, Marc Grimme wrote:

>>> no the iscsid.conf should be created from cluster.conf. Because of the ip
>>> of the iscsi-source. It might be one server gets it root from on
>>> iscsi-device and another from another iscsi-device.
>>> Some customers have clusters with different roots for different nodes.
>>
>> Err - I thought this was about _shared_ roots, not unshared-roots.  :-p
>
> Yes it still is. One cluster with root x and another with root y but still in
> the same cluster. ;-)

>> What makes iscsid.conf from cluster.conf, then?
> You. ;-)

That seems contradictory. Do I create the iscsid.conf myself, or does it 
get created from cluster.conf? If I create it myself, then it should 
arguably be copied by the initrd maker from the source machine - at least 
in cases where the machines in the cluster all use the same root.

> That has to be done. I don't remember how it looks like. I implemented it a
> long time ago. But wouldn't it do it to just copy a template /etc/iscsi.conf
> into the initrd and then add the lines for the device from the cluster.conf.
> You should get the value for the rootsource as described on my blog.

That's what I was saying - the initrd maker needs to include 
/etc/iscsi/iscsid.conf in the initrd.

>> And what would configure the /var/lib/iscsi/*?
>
> Hmm. What is in there?

The iscsi device information yielded by the iscsi discovery step. What I 
usually do is set iscsid.conf to "manual" and put the username/password 
for the share there. Then I do iSCSI discovery, and modify the file for 
the share I want to mount to set that share connecting to "automatic". 
Only the automatic shares get automatically attached to /dev/sd? nodes.

Then I can mount the device as a normal block device.

So, if this is to be automated, it would need to copy the pre-prepared 
/var/lib/iscsi/* files into the initrd.

Unless you're about to tell me that there is a better, auto-magical way to 
achieve all this, purely from the cluster.conf file.

>> I've found I have to jump through a few hoops to get this working even
>> without the shared GFS. I have to set up connections to manual, and set
>> the username/password up in iscsid.conf, then do the iscsi discovery, then
>
> Uhh, username/password.
> Ahh rootsource="iscsi://username:password@iscsi-ip:port/" that sound not too
> bad.

You'd need the iSCSI volume ID in there as well somehow. This would 
typically look something like the following:

iqn.2001-05.com.mydomain:x-xxxxxx-xxxxxxxxx-xxxxxxxxxxxxxxxx-nodename

where x is a hex number.

>> just change the volume I want to mount to automatic. Then it correctly
>> mounts on the correct /dev/sd? device.
>
> Ok these commands should be added to iscsiLoad in
> boot-scripts/etc/iscsi-lib.sh.

That sounds a tad complicated for something to build at start-up. I think 
just including the iscsid.conf and /var/log/iscsi/* would be a better 
option.

>>>> 3) Add iscsi and iscsid init scripts to the relevant run-level on OSR
>>>> initrd.
>>>
>>> there are no run-levels inside the initrd ;-( . So the starting has to be
>>> done manually within iscsi-lib and the iscsiLoad Function.
>>
>> Well, run-levels aren't what I meant as such. Are init scripts handled in
>> a similar way to the full setup? Even if it is a startup script invoking:
>> iscsid start
>> iscsi start
> This can easily be done why not. Cannot think of anything against it.

OK. I just wanted to make sure I'm following the intended paradigm, rather 
than start mixing BSD ways with SystemV ways. :-)

>> I have yet to get a single node booting off GFS as root. I'm some way off
>> having a whole cluster running at the moment. Need to get the iSCSI
>> working first.
> Yes.
> This takes some time. But is nice when it is running.

Do entries in:
/etc/comoonics/bootimage/files.initrd.d/*
support wildcards?

i.e. can I just include a line in a file that says:
/var/lib/iscsi/*
?

>>> rootvolume has to be given always. It tells where to mount the rootfs
>>> from. rootsource is optional: I would use it for telling the cluster that
>>> the rootdevice itself is an iscsione and the ipadress is x.y.z.
>>
>> Right, OK. But before this can work, surely the iscsi daemons have to be
>> running first?
>
> If so start them before.

Where is the template init script file that ends up on the initrd? 
Presumably that is where I would need to add this.

>>> com-expert: imediately starts a shell after initrd is loaded.
>>
>> Well, at the moment it fails with the message that it can't find root and
>> drops me to the initrd shell. What I need at that point is functioning
>> iscsi components. I think that is logically the next thing I need to add
>> to the initrd.
>
> exactly ;-)

If/when I can get this working, do you want me to post the changes/patch 
here? I would have thought that most people who use SANs/GFS would be 
needing iSCSI support.

Gordan