Re: [BProc] MATLAB and BProc

[Reza S. <sh...@en...>]

Hello,

    I tried the script you sent below, but now the nodes get stuck with 
a status of boot when Clustermatic is restarted.  I can't bpsh to the 
nodes or anything.  On the screen of node 0, the boot sequence gets 
stuck at bpslave-0: setting node number to 0, and stays that way if not 
restarted.  This does not happen when the regular node_up/nfs.init 
scripts are used, but of course, I am still not able to get the NFS 
mount working in this case either.  Any more ideas?

Thanks,

Reza

Daniel Gruner wrote:

>Reza,
>
>For some reason, in Clustermatic 5, trying to do NFS mounts according
>to the "manual" (which is what you tried, and used to work in 
>Clustermatic 4), doesn't work anymore.  We've had to do some hacks in order
>to make it work.  In short, do NOT try the NFS mounts in 
>/etc/clustermatic/fstab.  What you have to do is run a script from the
>/etc/clustermatic/node_up script, which will do all the necessary stuff on
>the nodes.
>
>I am attaching here my /etc/clustermatic/node_up, and another file called
>nfs.init which is also put in /etc/clustermatic.  This scheme works
>well for us, and it should work for you as well.  You will need to modify
>the nfs.init script to mount your particular filesystem(s).
>
>Regards,
>Daniel
>
>  
>
>------------------------------------------------------------------------
>
>#!/bin/sh
>#
># This shell script is called automatically by BProc to perform any
># steps necessary to bring up the nodes.  This is just a stub script
># pointing to the program that does the real work.
>#
># $Id: node_up.stub,v 1.3 2003/11/12 23:30:59 mkdist Exp $
>
># All changes up to "############" line by
># Michal Jaegermann, mi...@ha...
>
>seterror () {
>    bpctl -S $1 -s error
>    exit 1
>}
>
>if [ -x /usr/lib64/beoboot/bin/node_up ] ; then
>    /usr/lib64/beoboot/bin/node_up $* || seterror $*
>else
>    /usr/lib/beoboot/bin/node_up $* || seterror $*
>fi
># we are "sourcing" these script so variable assignments
># remain like in here; pass a node number as an argument
># if you want to _run_ them from a shell and wrap in a loop
># for multiple nodes
>#
># lm_sensors - 'bpsh 3 sensors' will produce sensors information for node 3
># . /etc/clustermatic/sensors.init
># if we use pathscale libraries we have to make them available on nodes
># . /etc/clustermatic/pathscale.init
># similarly for Intel compiler
># . /etc/clustermatic/intel.init
># Turn the next line on for NFS support on nodes
>. /etc/clustermatic/nfs.init
>
>exit
>
>############
>
># below the original script - now NOT executing due to 'exit' above
>
>if [ -x /usr/lib64/beoboot/bin/node_up ] ; then
>    exec /usr/lib64/beoboot/bin/node_up $*
>else
>    exec /usr/lib/beoboot/bin/node_up $*
>fi
>
># If we reach this point there's an error.
>bpctl -S $* -s error
>exit 1
>
># If you want to put more setup stuff here, make sure do replace the
># "exec" above with the following:
># /usr/lib/beoboot/bin/node_up $* || exit 1
>  
>
>------------------------------------------------------------------------
>
>#!/bin/sh
>#
># A sample how to get NFS modules on a node.
># Make sure that /etc/modules.conf.dist for a node does not
># define any 'install' actions for these
>#
>#  Michal Jaegermann, 2004/Aug/19, mi...@ha...
>#
>
>node=$1
># get the list of modules, and copy them to the node
>mod=nfs
>modules=$( grep $mod.ko /lib/modules/$(uname -r)/modules.dep)
>modules=${modules/:/}
>modules=$(
>for m in $modules ; do
>    echo $m
>done | tac )
>( cd /
>    for m in $modules ; do
>	echo $m
>    done
>) | ( cd / ; cpio -o -c --quiet ) | bpsh $node cpio -imd --quiet
>bpsh $node depmod -a
># fix the permissions after cpio
>bpsh $node chmod -R a+rX /lib
># load the modules
>for m in $modules ; do
>    m=$(basename $m .ko)
>    m=${m/_/-}
>    case $m in
>	sunrpc)
>	    bpsh $node modprobe -i sunrpc
>	    bpsh $node mkdir -p /var/lib/nfs/rpc_pipefs
>	    bpsh $node mount | grep -q rpc_pipefs || \
>		bpsh $node mount -t rpc_pipefs sunrpc /var/lib/nfs/rpc_pipefs
>	    ;;
>	*)  bpsh $node modprobe -i $m
>    esac
>done
># these are for a benfit of rpc.statd
>bpsh $node mkdir -p /var/lib/nfs/statd/
>bpsh $node mkdir -p /var/run
>bpsh $node portmap
>bpsh $node rpc.statd
>bpsh $node mkdir /home
>bpsh $node mount -t nfs -o nfsvers=3,rw,noac  master:/home /home
>bpsh $node mkdir /usr/local
>bpsh $node mount -t nfs -o nfsvers=3,rw,noac  master:/usr/local /usr/local
>  
>