[BProc] How scripts are run in batch spooling

[<ha...@no...>]

Clubmask batch spooling can run scripts via BProc and commercial PBS
also can. BProc 3.1.6+ can exec() on slave getting image from master,
but this most likely is not the PBS way of running scripts. We are
somewhere near to scripts transparently executing on slaves, but not
yet there.

How exactly is script run? What's where during script execution?

I can imagine these scenarios:

1) Slave nodes have NFS mounted not only /home but also /bin, /usr/bin
etc. Execution server is on master. Script is moved to slave somehow like

   bpsh NODE bash script

and whenever bash executes command from script, it

  a) just gets the binary over NFS mount and does normal local exec()

  b) uses NFS just to look around, when it comes to exec(), binary is
     fetched from master via BProc transparently to bash which does
     not know that exec() does something unusual

2) Already execution server is BProc-moved to slave. Everything is
   NFS-mounted as in 1), server runs on slave and gets everything
   via NFS.

3) Execution server is on master, scripts are run on master via
   modified bash which BProc-moves just certain heavy-duty executables
   to slaves. Just /home is NFS-mounted.


I do not quite beleive that /bin and /usr/bin are NFS mounted on all
slaves as in 1) and 2) and I do not beleive that there is modified
bash as in 3). So Clubmask and PBS probably use some method which is
over my imagination. Please tell me what it is.


I also wander about performance comparison of BProc and NFS.  For
dynamically linked executable, overhead of 1)a) and 1)b) should be
comparable:

 1)a) executable is moved via BProc
      libraries are cached on slave

 1)b) executable got via NFS
      libraries got via NFS, next time probably cached in filesystem cache

so the only advantage of BProc would be common PID space. However many
docs imply BProc move is better. So what is wrong with comparison
above? NFS is flaky and does not move data as quickly as BProc?
Something else?

(Sorry for these amateur questions. I prommise to learn quickly... ;-)

The reason for all this is that I'd like to use Grid Engine with
BProc, though I will also consider Clubmask (and Clustermatic? does it
have batch spooling?).

Best Regards

Vaclav Hanzl