TaskManager Files

TASKMANAGER
===========

TaskManager manages calculation jobs in a computer cluster environment

TaskManager is an open source infrastructure software for distributing and managing calculation jobs
in a computer cluster environment. The TaskManager package consists of several servers,
TaskDispatcher, TaskManagerServer and InfoServer, and clients, which communicate with these servers.
The main server, TaskDispatcher, is responsible for receiving jobs from users, storing detailed
information about each job, sending jobs to vacant computer in a cluster, and controlling their
execution. A TaskManagerServer is invoked by each user, respectively, in the background with his
unix permissions. Therefore, it has the rights to access user specific data in the file
system. Finally, the InfoServer is invoked on every computer in the cluster to gather information
about the computer. The servers and clients communicate over secure socket layers (SSL)
authenticated with certificates, which are generated by the task manager admin.

The following computer environment configuation is required:

- Unix system with a python installation
- One or more computers with one or more cores
- Enable login into each computer via ssh without typing a password 
- Access to the home file system over ssh

Additionally to the Servers and Clients, the TaskManager package contains a Zope product for a
webinterface. For that, a Zope webserver is required.



Installation & Configuration
----------------------------

In order to install the TaskManager follow the given configuation steps:

1. Copy entire TaskManager directory to a directory of your choice, e.g., /usr/local/opt

2. Create a frozen version of several python programs with freeze. Modify and execute script

      scripts/mkDist.sh

For each program all necessary python libraries are copied to a single directory and a binary is
created. Therefore a load of python libraries over an intranet is not anymore necessary and it makes
the execution faster.

3. Set permissions of binaries that every user can execute it.

4. Create certificate for user taskdispatcher who represents the TaskDispatcher. Modify and execute
script

      cd scripts; createCertificate.sh taskdispatcher <YOUREMAIL>; cd -

5. Create for each user a certificate

      cd scripts; createCertificate.sh <USER> <USEREMAIL>; cd -

For each user the following files are created in etc/certs

      <USER>.key   private key of user
      <USER>.csr   certificate request file (not necessary)
      <USER>.crt   self signed certificate of user
      ca_certs.<USER>.crt   certificates of taskdispatcher and user himself

6. Each user has to create the directory .taskManager in his home directory and copy the following
files to that directory:

      <USER>.key
      <USER>.crt
      ca_certs.<USER>.crt

7. Include all users who are allowed to use the TaskManager and their certificate files into

      etc/users

8. Assign users to groups

      etc/groups

9. Configure computer cluster by given information about each computer in the cluster in 

      etc/ComputerCluster.config

10. Set host and port of taskdispatcher in

      etc/TaskDispatcher.info

11. Start TaskDispatcher as the TaskManager admin

      cd Server
      python TaskDispatcher -e ../var/TaskDispatcherError.log -p 101010

12. Run as a user

      bin/hRunJob -h



Files and directories
---------------------

|-- Client
|   |-- hRun.py
|   |-- hRunJob.py
|   |-- hSend.py
|-- README
|-- Server
|   |-- InfoServer.py
|   |-- TaskDispatcher.py
|   |-- daemon.py
|-- UserServer
|   |-- TMS.py
|   |-- daemon.py
|-- ZopeProduct
|   `-- ZPTaskManager
|       |-- ZPTaskManagerSite.py
|       |-- ZPTaskManagerSite.pyc
|       |-- __init__.py
|       |-- __init__.pyc
|       |-- css                         ... directory
|       |-- js                          ... directory
|       |-- lib
|       |   |-- hSocket.py
|       |-- pics                        ... directory
|       |-- refresh.txt
|       `-- zpt
|           |-- AddComputer.zpt
|           |-- ChangeComputer.zpt
|           |-- ChangeTaskDispatcher.zpt
|           |-- ChangeTaskDispatcherProperties.zpt
|           |-- ChangeTaskDispatcherProperties.zpt~
|           |-- HostInfoSite.zpt
|           |-- JobInformation.zpt
|           |-- ProcSite.zpt
|           |-- ShowJobs.zpt
|           |-- ShowPendingJobs.zpt
|           |-- TaskDispatcherLog.zpt
|           |-- TaskDispatcherOutput.zpt
|           |-- TopSite.zpt
|           |-- ZPTaskManager.zpt
|           |-- ZPTaskManagerAdmin.zpt
|           |-- ZPTaskManagerFinishedJobs.zpt
|           `-- addZPTaskManagerSiteForm.zpt
|-- bin
|   |-- TMS
|   |-- TMSMenial
|   |-- hRun
|   |-- hRunJob
|   |-- hSend
|-- etc
|   |-- TMS.config
|   |-- TaskDispatcher.info
|   |-- TaskManagerOverview.dia
|   |-- certs                            ... directory
|   |-- groups
|   |-- python2.5.csh
|   |-- python2.5.sh
|   |-- python2.5_64.csh
|   |-- python2.5_64.sh
|   |-- python2.6.4_linux64.csh
|   |-- python2.6.4_linux64.sh
|   |-- python2.6_local64.csh
|   |-- python2.6_local64.sh
|   `-- users
|-- lib
|   |-- SocketServer.py
|   |-- hSocket.py
|-- scripts
|   |-- createAuthorizedCertsFile.sh
|   |-- createCertificate.sh
|   |-- createCertificates.txt
|   |-- createStandAloneClients.sh
|   |-- mkDist.sh
|   |-- startTaskDispatcher.csh
`-- var                                  ... directory