[Kestrelhpc-developers] How KestrelHPC works

[Jon A. H. <jo...@gm...>]

Hi everybody :

KestrelHPC is a set of tools which help setting up the basic services
required for creating a simple cluster :

- Create the node os image.
- Set up the nfs root export.
- Set up the nfs home export.
- Create ssh keys for login into the nodes without passwords.
- Set up a dhcp server, and a tftp server for PXE.
- etc...

We also wanted to be capable of reconfigure everything easily. So the
scripts recreate the configuration for each service reading some
variables from a config file.
Those scripts are called the configuration scripts :

/usr/share/kestrel/frontend/configure.d/
/usr/share/kestrel/node/configure.d/

So when we change the config file, we only need to run
kestrel-reconfigure --all, and everything will be reconfigured.


We also may need to do some actions when we create a new node image or
when we remove it. We also have to do some actions when we install or
remove kestrel. Does actions are done by the installation scripts :

/usr/share/kestrel/frontend/install.d/
/usr/share/kestrel/node/install.d/


Also note, that some scripts need to be run inside a chroot under the
node image. We use the suffix "_chroot" to denote that.


We also have 2 daemons : kestrel-daemon and kestrel-rpc


- kestrel-daemon reads events/commands from fifo file :
/var/lib/kestrel/rpc/fifo. We have connect, disconnect, register,
desregister, wake and shutdown events/commands :

* connect and disconnect events are sent by kestrel-rpc when a client
makes a rpc call to the server indicating that is has started up or is
shuting down.

* register event is like connect. The only diference is that the
client should be registered (we are registering new nodes with
kestrel-control --register).

* desregister is called from kestrel-control. ToDo.

* wake, shutdown are called from kestrel-control. ToDo. We need these
events to do the necesary steps before starting the nodes, for example
for granting access to the nodes to the nfs services through the
firewall. And for the counterpart when shutting down a node.


- kestrel-rpc. Is a simple rpc and easily extendable written with python.


Well this was a quite informal, and quite incomplete explanation,
please ask if you have any doubt. :-)


Regards,

Jon Ander.