[Pipeline130-developers] DRMAA Java interface

[Nils H. <nil...@uc...>]

I would be happy to work on the DRMAA interface into the cluster (the
Workflow engine); let me know if this is a problem.  I could use one or two
more people to help programming our engine to interact with the DRMAA API.
You should read the DRMAA manual:
http://www.ogf.org/documents/GFD.130.pdf

We need to start assigning programming tasks, so think of what you want to
contribute.

I would propose the following Object Orientated abstraction in the Engine.

1. The engine fundamentally forms a queue of Workflow objects (Workflow
class).  To check the status of a workflow, it only needs to iterate through
this queue.

2.  Each workflow class is composed of jobs (nodes).  The workflow simply
needs to store a list representing the canonical ordering of the jobs
(nodes).

3.  Each node stores the bulk of the necessary information.
    3.a COMMAND: stores the "command", including I/O path, path to script,
command line arguments, etc.
    3.b NODE PROPERTIES: stores references to incoming nodes, outgoing
nodes, and whether the have received confirmations (ok to run) from incoming
nodes, whether it has sent confirmations to child nodes.

4.  The engine maintains a list of jobs IDS that it has submitted (for
tracking).  Once a job completes (fails, succeeds, or aborted), the engine
transfers this status to the workflow.  The workflow class signals the
correct node of the status.  The node then acts accordingly (signals
children, throws an exception, returns a status etc.).

I think this message passing scheme:
DRMAA <-> ENGINE
ENGINE <-> WORKFLOW
WORKFLOW <-> NODE

Works well and should be easy to implement.

What do you guys think?

Nils