Recent changes to Home

Home modified by Geert Geurts

Geert Geurts — Mon, 13 May 2013 12:50:19 -0000

--- v15
+++ v16
@@ -1,84 +0,0 @@
-TUDA-rsync_install
-=======================
-introduction
--------------
-The installation of compute nodes, as well as keeping them software wise identical, can be difficult to accomplish. Because of this we created a initrd image similar to the initrd image used by [Bright Cluster Manager](http://www.brightcomputing.com/Linux-Cluster-Node-Provisioning.php).
-
-Usage of the TUDA-rsync_install initrd image makes the boot process roughly look like the following steps:
-**1.**) The machine boots over PXE, and receives a kernel and the TUDA-rsync_install initrd.
-**2.**) The TUDA-rsync_install initrd image checks the partition layout and decides if it's ok or if it needs to be repartitioned.
-**3.**) The TUDA-rsync_install initrd image partitions the disk if needed, and/or mounts the rootfs partition.
-**4.**) The TUDA-rsync_install initrd image initiates an rsyc to update the root filesystem to the root filesystem template on the provisioning node.
-**5.**) The TUDA-rsync_install initrd image unmounts the root filesystem, and continues boot as normal.
-**6.**) After this the normal initrd boot process takes over, mounts the root filesystem again, does some checks and switches root to the root filesystem to finish the booting process.
-
-See how it works using virtualbox images
--------------
-I've created an VirtualBox master and slave image so you can easily see what it does, and how it works.
-**1.**) Firsts download the archive:[VirtualBox-VMs.tar.bz2](http://sourceforge.net/projects/tudarsyncinstal/files/VirtualBox-VMs.tar.bz2/download)
-**2.**) Unpack the archive and open both the master and the slave images.
-*notice the size of the slave harddisk (slave.vdi(36K) it's empty)*
-**3.**) Start first the master image.
-*notice the /node-images/default-image directory*
-**4.**) Start the slave machine.
-**5.**) Enjoy the TUDA magic ;)
-*The root password of both master and slave is "tuda"*
-*The slave has the id_rsa key of the master to allow passwordless root access to the slave.*
-*The slave initrd image is accessibly over ssh. You might need to modify the known_hosts file (tuda-master:/root/.ssh/known_hosts) as the slave switches from ssh_id between initrd and normal root fs sshd.*
-*The slave boot options are modifiable in tuda-master:/srv/tftpboot/pxelinux.cfg/default*
-*Try to add/remove some files in tuda-master:/node-images/default-image to see how they appear on the slave node.* 
-*Remember! the slave root filesystem is UPDATED with the contents of /node-images/default-image. No need to recreate the virtualbox disk.*
-
-Installing it on a real cluster
-=======================
-prerequisites
--------------
- * The rpm provided has been build for SLES11. However, because there are only bash script typical commands used, it should be easily adaptable to any distribution.
- * For the provisioning a pxe server is needed.
- * For the provisioning a template root filesystem is need, this is explained in GGGIMGGENSTEPGGG 
-
-Installation
--------------
-The easiest installation is using a cluster with the compute nodes identical to the provisioning node. This because the initrd image has to be generated on a compute node or when possible on a identical provisioning node. 
-The next step is to install the rpm, or generate an rpm from included spec file.
- * rpm -i tuda-rsync_install-1-latestSVN_trunk.x86_64.rpm
-or
- * rpmbuild -bb SPEC/tuda-rsync_install.spec
-*If you're going to use the specfile to generate your own rpm, you're going to have to modify it as it assumes it's being run on my workstation, so you should modify some paths.*
-
-Initrd generation 
--------------
-The rpm contains a script, tuda-mkinitrd, to generate a tuda-rsync_install initrd image.
-*The script again assumes it is run on my workstation so you might have to modify it
-Important is one or more template root filesystem directories in /node-images/ as explained in GGGIMGGENSTEPGGG.*
-The script makes sure modules are going to be available for the kernel your generating the image for and needs one argument. The interface the nodes should request an ip to contact the provisioning node.
-eg. tuda-mkinitrd eth0 for eth0.
-once the script is finished a new shiny initrd will be placed in /tftpboot/xcat/tuda/.
-*The script assumes xcat integration, if this is not the case simply change the /tftpboot/xcat/tuda/ directory references to some other place. **REMEMBER** mkinitrd needs the vmlinuz of the correct version in the destination directory!*
-
-Kernel options
--------------
-All the options available have to be specified to the tuda-rsync_install initrd trough kernel options.
-Here follow the list of available options and their description:
-** * provnode=172.20.0.254**
-The provnode option is used to specify the ip address of the provisioning node. *eg. the node to initiate the rsync with*
-** * root=/dev/sda2**
-The root option is used to specify which partition is to be used as root filesystem for the node.
-** * image=/node-images/default-image**
-The image option is used to specify the full path on the provisioning node where the template root filesystem can be found which you want to be installed on the node.
-** * partitionsinfo=/dev/sda1:linux-swap:1000:100,/dev/sda2:ext3:4700:470:/**
-The partitionsinfo option is used to specify how you want the disks on the node to be partition. It's a comma separated list of partition definitions. A partition definition is a doublepoint seperated list of 5 elements. 
-**1**) The blockdevice used for the filesystem.  */dev/sda2*
-**2**) The filesystem type                                        *ext3*
-**3**) The size of the partition in MB                            *4700*
-**4**) The allowed difference when checking the partition layout. *470*
-**5**) The mountpoint the partition should be mounted at.         *(left empty for swap)*
-The number of partitions cannot be more than 4. *I didn't have a need for more partitions so I didn't spend time in adding an extended partition feature. It shouldn't be hard to implement though, as the partitions are mounted in the right order. eg. / is mounted before /usr even when /usr is sda1 and / is sda2 for example. Also more then one disk should work, but again this is un-tested as I didn't have the need for this. Shouldn't be difficult to implement also as the scripts are written for multiple disks.*
-** * instif=eth0**
-The instif option is used for specifying which interface to use for the rsync step. Should always be the same as the option to tuda-mkinitrd.
-** * full=2013-03-01_00:00:00**
-The full option can be used to specify till when the node should be repartition and fetch a fresh root filesystem even when the partition layout seems perfectly fine.
-** * tuda_debug=1**
-The tuda_debug option can be used in two ways.
-1) tuda_debug=1 ssh daemon is started in the beginning of initrd execution, so you can ssh to the node to see how the tuda-rsync_install initrd is progressing.
-2) tuda_debug=2 ssh daemon is started in the beginning of initrd execution and the execution of the initrd scripts is halted till a logout occurs. Can be used to manually initiate the scripts to see why a script hangs. Execution might take up to 20 seconds to continue after logout as this is only checked once every 20 seconds. *Shouldn't ever be needed to be specified ;)*

Home modified by Geert Geurts

Geert Geurts — Mon, 13 May 2013 05:31:35 -0000

--- v14
+++ v15
@@ -15,7 +15,7 @@
 See how it works using virtualbox images
 -------------
 I've created an VirtualBox master and slave image so you can easily see what it does, and how it works.
-**1.**) Firsts download the archive:[VirtualBox-VMs.tar.bz2](http://verweggistan.eu/VirtualBox-VMs.tar.bz2)
+**1.**) Firsts download the archive:[VirtualBox-VMs.tar.bz2](http://sourceforge.net/projects/tudarsyncinstal/files/VirtualBox-VMs.tar.bz2/download)
 **2.**) Unpack the archive and open both the master and the slave images.
 *notice the size of the slave harddisk (slave.vdi(36K) it's empty)*
 **3.**) Start first the master image.

Home modified by Geert Geurts

Geert Geurts — Sun, 12 May 2013 21:07:39 -0000

--- v13
+++ v14
@@ -15,7 +15,7 @@
 See how it works using virtualbox images
 -------------
 I've created an VirtualBox master and slave image so you can easily see what it does, and how it works.
-**1.**) Firsts download the archive from:http://sourceforge.net/projects/tudarsyncinstal/files/VirtualBox-VMs.tar.bz2/download
+**1.**) Firsts download the archive:[VirtualBox-VMs.tar.bz2](http://verweggistan.eu/VirtualBox-VMs.tar.bz2)
 **2.**) Unpack the archive and open both the master and the slave images.
 *notice the size of the slave harddisk (slave.vdi(36K) it's empty)*
 **3.**) Start first the master image.

Home modified by Geert Geurts

Geert Geurts — Sun, 12 May 2013 20:49:22 -0000

--- v12
+++ v13
@@ -15,7 +15,7 @@
 See how it works using virtualbox images
 -------------
 I've created an VirtualBox master and slave image so you can easily see what it does, and how it works.
-**1.**) Firsts download the archive from:GGG
+**1.**) Firsts download the archive from:http://sourceforge.net/projects/tudarsyncinstal/files/VirtualBox-VMs.tar.bz2/download
 **2.**) Unpack the archive and open both the master and the slave images.
 *notice the size of the slave harddisk (slave.vdi(36K) it's empty)*
 **3.**) Start first the master image.

Home modified by Geert Geurts

Geert Geurts — Sun, 12 May 2013 20:45:58 -0000

--- v11
+++ v12
@@ -12,6 +12,25 @@
 **5.**) The TUDA-rsync_install initrd image unmounts the root filesystem, and continues boot as normal.
 **6.**) After this the normal initrd boot process takes over, mounts the root filesystem again, does some checks and switches root to the root filesystem to finish the booting process.

+See how it works using virtualbox images
+-------------
+I've created an VirtualBox master and slave image so you can easily see what it does, and how it works.
+**1.**) Firsts download the archive from:GGG
+**2.**) Unpack the archive and open both the master and the slave images.
+*notice the size of the slave harddisk (slave.vdi(36K) it's empty)*
+**3.**) Start first the master image.
+*notice the /node-images/default-image directory*
+**4.**) Start the slave machine.
+**5.**) Enjoy the TUDA magic ;)
+*The root password of both master and slave is "tuda"*
+*The slave has the id_rsa key of the master to allow passwordless root access to the slave.*
+*The slave initrd image is accessibly over ssh. You might need to modify the known_hosts file (tuda-master:/root/.ssh/known_hosts) as the slave switches from ssh_id between initrd and normal root fs sshd.*
+*The slave boot options are modifiable in tuda-master:/srv/tftpboot/pxelinux.cfg/default*
+*Try to add/remove some files in tuda-master:/node-images/default-image to see how they appear on the slave node.* 
+*Remember! the slave root filesystem is UPDATED with the contents of /node-images/default-image. No need to recreate the virtualbox disk.*
+
+Installing it on a real cluster
+=======================
 prerequisites
 -------------
  * The rpm provided has been build for SLES11. However, because there are only bash script typical commands used, it should be easily adaptable to any distribution.

Home modified by Geert Geurts

Geert Geurts — Tue, 23 Apr 2013 06:30:50 -0000

--- v10
+++ v11
@@ -49,17 +49,17 @@
 The image option is used to specify the full path on the provisioning node where the template root filesystem can be found which you want to be installed on the node.
 ** * partitionsinfo=/dev/sda1:linux-swap:1000:100,/dev/sda2:ext3:4700:470:/**
 The partitionsinfo option is used to specify how you want the disks on the node to be partition. It's a comma separated list of partition definitions. A partition definition is a doublepoint seperated list of 5 elements. 
-**1**) The blockdevice used for the filesystem. */dev/sda2*
-**2**) The filesystem type *ext3*
-**3**) The size of the partition in MB *4700*
+**1**) The blockdevice used for the filesystem.  */dev/sda2*
+**2**) The filesystem type                                        *ext3*
+**3**) The size of the partition in MB                            *4700*
 **4**) The allowed difference when checking the partition layout. *470*
-**5**) The mountpoint the partition should be mounted at. (left empty for swap)*
+**5**) The mountpoint the partition should be mounted at.         *(left empty for swap)*
 The number of partitions cannot be more than 4. *I didn't have a need for more partitions so I didn't spend time in adding an extended partition feature. It shouldn't be hard to implement though, as the partitions are mounted in the right order. eg. / is mounted before /usr even when /usr is sda1 and / is sda2 for example. Also more then one disk should work, but again this is un-tested as I didn't have the need for this. Shouldn't be difficult to implement also as the scripts are written for multiple disks.*
-* **instif=eth0**
+** * instif=eth0**
 The instif option is used for specifying which interface to use for the rsync step. Should always be the same as the option to tuda-mkinitrd.
-* **full=2013-03-01_00:00:00**
+** * full=2013-03-01_00:00:00**
 The full option can be used to specify till when the node should be repartition and fetch a fresh root filesystem even when the partition layout seems perfectly fine.
-* **tuda_debug=1**
+** * tuda_debug=1**
 The tuda_debug option can be used in two ways.
 1) tuda_debug=1 ssh daemon is started in the beginning of initrd execution, so you can ssh to the node to see how the tuda-rsync_install initrd is progressing.
 2) tuda_debug=2 ssh daemon is started in the beginning of initrd execution and the execution of the initrd scripts is halted till a logout occurs. Can be used to manually initiate the scripts to see why a script hangs. Execution might take up to 20 seconds to continue after logout as this is only checked once every 20 seconds. *Shouldn't ever be needed to be specified ;)*

Home modified by Geert Geurts

Geert Geurts — Tue, 23 Apr 2013 06:15:24 -0000

--- v9
+++ v10
@@ -48,11 +48,12 @@
 ** * image=/node-images/default-image**
 The image option is used to specify the full path on the provisioning node where the template root filesystem can be found which you want to be installed on the node.
 ** * partitionsinfo=/dev/sda1:linux-swap:1000:100,/dev/sda2:ext3:4700:470:/**
-The partitionsinfo option is used to specify how you want the disks on the node to be partition. It's a comma separated list of partition definitions. A partition definition is a doublepoint seperated list of 4 elements. 
-**1**) The blockdevice used for the filesystem.
-**2**) The filesystem type 
-**3**) The size of the partition in MB
-**4**) The allowed difference when checking the partition layout.
+The partitionsinfo option is used to specify how you want the disks on the node to be partition. It's a comma separated list of partition definitions. A partition definition is a doublepoint seperated list of 5 elements. 
+**1**) The blockdevice used for the filesystem. */dev/sda2*
+**2**) The filesystem type *ext3*
+**3**) The size of the partition in MB *4700*
+**4**) The allowed difference when checking the partition layout. *470*
+**5**) The mountpoint the partition should be mounted at. (left empty for swap)*
 The number of partitions cannot be more than 4. *I didn't have a need for more partitions so I didn't spend time in adding an extended partition feature. It shouldn't be hard to implement though, as the partitions are mounted in the right order. eg. / is mounted before /usr even when /usr is sda1 and / is sda2 for example. Also more then one disk should work, but again this is un-tested as I didn't have the need for this. Shouldn't be difficult to implement also as the scripts are written for multiple disks.*
 * **instif=eth0**
 The instif option is used for specifying which interface to use for the rsync step. Should always be the same as the option to tuda-mkinitrd.

Home modified by Geert Geurts

Geert Geurts — Tue, 23 Apr 2013 06:12:12 -0000

--- v8
+++ v9
@@ -41,13 +41,13 @@
 -------------
 All the options available have to be specified to the tuda-rsync_install initrd trough kernel options.
 Here follow the list of available options and their description:
- * **provnode=172.20.0.254**
+** * provnode=172.20.0.254**
 The provnode option is used to specify the ip address of the provisioning node. *eg. the node to initiate the rsync with*
- * **root=/dev/sda2**
+** * root=/dev/sda2**
 The root option is used to specify which partition is to be used as root filesystem for the node.
- * **image=/node-images/default-image**
+** * image=/node-images/default-image**
 The image option is used to specify the full path on the provisioning node where the template root filesystem can be found which you want to be installed on the node.
- * **partitionsinfo=/dev/sda1:linux-swap:1000:100,/dev/sda2:ext3:4700:470:/**
+** * partitionsinfo=/dev/sda1:linux-swap:1000:100,/dev/sda2:ext3:4700:470:/**
 The partitionsinfo option is used to specify how you want the disks on the node to be partition. It's a comma separated list of partition definitions. A partition definition is a doublepoint seperated list of 4 elements. 
 **1**) The blockdevice used for the filesystem.
 **2**) The filesystem type

Home modified by Geert Geurts

Geert Geurts — Tue, 23 Apr 2013 06:09:58 -0000

--- v7
+++ v8
@@ -14,9 +14,9 @@

 prerequisites
 -------------
-*The rpm provided has been build for SLES11. However, because there are only bash script typical commands used, it should be easily adaptable to any distribution.
-*For the provisioning a pxe server is needed.
-*For the provisioning a template root filesystem is need, this is explained in GGGIMGGENSTEPGGG 
+ * The rpm provided has been build for SLES11. However, because there are only bash script typical commands used, it should be easily adaptable to any distribution.
+ * For the provisioning a pxe server is needed.
+ * For the provisioning a template root filesystem is need, this is explained in GGGIMGGENSTEPGGG 

 Installation
 -------------

Home modified by Geert Geurts

Geert Geurts — Mon, 22 Apr 2013 21:41:57 -0000

--- v6
+++ v7
@@ -41,24 +41,24 @@
 -------------
 All the options available have to be specified to the tuda-rsync_install initrd trough kernel options.
 Here follow the list of available options and their description:
- * **provnode**=172.20.0.254
+ * **provnode=172.20.0.254**
 The provnode option is used to specify the ip address of the provisioning node. *eg. the node to initiate the rsync with*
- * **root**=/dev/sda2
+ * **root=/dev/sda2**
 The root option is used to specify which partition is to be used as root filesystem for the node.
- * **image**=/node-images/default-image
+ * **image=/node-images/default-image**
 The image option is used to specify the full path on the provisioning node where the template root filesystem can be found which you want to be installed on the node.
- * **partitionsinfo**=/dev/sda1:linux-swap:1000:100,/dev/sda2:ext3:4700:470:/
+ * **partitionsinfo=/dev/sda1:linux-swap:1000:100,/dev/sda2:ext3:4700:470:/**
 The partitionsinfo option is used to specify how you want the disks on the node to be partition. It's a comma separated list of partition definitions. A partition definition is a doublepoint seperated list of 4 elements. 
 **1**) The blockdevice used for the filesystem.
 **2**) The filesystem type 
 **3**) The size of the partition in MB
 **4**) The allowed difference when checking the partition layout.
 The number of partitions cannot be more than 4. *I didn't have a need for more partitions so I didn't spend time in adding an extended partition feature. It shouldn't be hard to implement though, as the partitions are mounted in the right order. eg. / is mounted before /usr even when /usr is sda1 and / is sda2 for example. Also more then one disk should work, but again this is un-tested as I didn't have the need for this. Shouldn't be difficult to implement also as the scripts are written for multiple disks.*
-* **instif**=eth0
+* **instif=eth0**
 The instif option is used for specifying which interface to use for the rsync step. Should always be the same as the option to tuda-mkinitrd.
-* **full**=2013-03-01_00:00:00
+* **full=2013-03-01_00:00:00**
 The full option can be used to specify till when the node should be repartition and fetch a fresh root filesystem even when the partition layout seems perfectly fine.
-* **tuda_debug**=1
+* **tuda_debug=1**
 The tuda_debug option can be used in two ways.
 1) tuda_debug=1 ssh daemon is started in the beginning of initrd execution, so you can ssh to the node to see how the tuda-rsync_install initrd is progressing.
 2) tuda_debug=2 ssh daemon is started in the beginning of initrd execution and the execution of the initrd scripts is halted till a logout occurs. Can be used to manually initiate the scripts to see why a script hangs. Execution might take up to 20 seconds to continue after logout as this is only checked once every 20 seconds. *Shouldn't ever be needed to be specified ;)*