Thread: RE: [SSI-devel] openSSI running over SF IPoIB
Brought to you by:
brucewalker,
rogertsang
From: Smith, S. <sta...@in...> - 2004-08-20 23:41:37
|
>=20 > Are there additional boot process agents which examine boottab other > than linuxrc? I'd like to rename ibtab back to what it should be > 'boottab'. >>No. I'm not attached to the 'boottab' name, so if Infiniband has already >>claimed the name, OpenSSI can use something else. >> >>Regards, >> >>Brian Infiniband has not claimed the name 'boottab'. My thoughts were to keep the initrd filename 'boottab' the same regardless of ICS hardware; no need to change. The name 'ibtab' came about thru my ignorance of how/who accessed boottab. From additional experiments, I now understand that a file, formatted exactly like boottab, which contains IPoIB bound IP addresses and IB hardware addresses works fine; 'ibtab' has gone away. I'm currently working on mkinitrd to auto-generate an initrd*.img file which matches the hand-crafted IB initrd*.img file. Stan. |
From: Smith, S. <sta...@in...> - 2004-08-26 23:52:24
|
=20 >> Sorry for the delay. I've been very focused on the OpenSSI/2.6 port lately. No problem... 2.6 is the 'future'. As a 2.6 side-note, all this Infiniband work will be transitioned over to the openib.org stack once OpenSSI/2.6 is released and the openib.org stack stabilizes. The openib.org stack is targeted for eventual kernel.org inclusion. >> How are you planning to change the boottab format to support IPoIB? No format change required, to date. The only difference is using the Infiniband HCA port guid as the 'hardware address' instead of an Enet MacAddr. The pain comes in determining the port guid. Without an IB fabric viewer, one is required to run IB subnet management from a system which is IB connected but not part of the cluster to configure the IB fabric. Boot the initnode, PXE boot (over Ethernet) the cluster nodes. A clusternode's 1st boot will fail as the IB port guid is not in the ramdisk's boottab. Linuxrc echos the node local port guids. From the initnode, enter the cluster node IB port guid, rebuild the ramdisk and reboot the cluster node. Pretty much a hack at this juncture, but enough to get things going. >> Note that mkdhcpd.conf is the program responsible for generating boottab=20 >> (I know this isn't exactly intuitive; it happened for historical=20 >> reasons). The actual work is done by the write_boottab() routine in=20 >> /cluster/lib/Clustertab.pm. Thanks for the pointers! |
From: Brian J. W. <Bri...@hp...> - 2004-08-31 21:42:00
|
Smith, Stan wrote: > No problem... 2.6 is the 'future'. As a 2.6 side-note, all this > Infiniband work will be transitioned over to the openib.org stack once > OpenSSI/2.6 is released and the openib.org stack stabilizes. The > openib.org stack is targeted for eventual kernel.org inclusion. Sounds good. > No format change required, to date. The only difference is using the > Infiniband HCA port guid as the 'hardware address' instead of an Enet > MacAddr. The pain comes in determining the port guid. Without an IB > fabric viewer, one is required to run IB subnet management from a system > which is IB connected but not part of the cluster to configure the IB > fabric. Boot the initnode, PXE boot (over Ethernet) the cluster nodes. A > clusternode's 1st boot will fail as the IB port guid is not in the > ramdisk's boottab. Linuxrc echos the node local port guids. From the > initnode, enter the cluster node IB port guid, rebuild the ramdisk and > reboot the cluster node. Pretty much a hack at this juncture, but enough > to get things going. As long as it can be documented, it's a good start. Any automation you can do (like enhancing mkinitrd to generate an IB-capable ramdisk) is a bonus. ;) BTW, you'll need to enhance the validate_hwaddr() routine in Clustertab.pm to allow GUIDs, as well as Ethernet MAC addresses. This routine is used to validate the hardware addresses in /etc/clustertab, which the source used to generate the boottab file in ramdisk. Regards, Brian |
From: Brian J. W. <Bri...@hp...> - 2004-08-26 21:48:19
|
Hi Stan, Sorry for the delay. I've been very focused on the OpenSSI/2.6 port lately. > Infiniband has not claimed the name 'boottab'. My thoughts were to keep > the initrd filename 'boottab' the same regardless of ICS hardware; no > need to change. The name 'ibtab' came about thru my ignorance of how/who > accessed boottab. From additional experiments, I now understand that a > file, formatted exactly like boottab, which contains IPoIB bound IP > addresses and IB hardware addresses works fine; 'ibtab' has gone away. How are you planning to change the boottab format to support IPoIB? > I'm currently working on mkinitrd to auto-generate an initrd*.img file > which matches the hand-crafted IB initrd*.img file. Note that mkdhcpd.conf is the program responsible for generating boottab (I know this isn't exactly intuitive; it happened for historical reasons). The actual work is done by the write_boottab() routine in /cluster/lib/Clustertab.pm. Thanks, Brian |