Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.
I read through the documentation of the project, and it is a very exciting. I recently worked with a group modifying ptolemy to be a generic grid for executing arbitrary commands, like
nmap somehost | grep something | awk something
This was done all through Java like the jppf project.
My question is, what prevents the use of Runtime.exec in jppf to simulate a generic command grid like the one I described. The only real matters of concern would be the SecurityManager, IO streams, and OS dependencies, ie. most Windows machines don't have preinstalled.
I don't know if I understand the issue correctly, but, if i'm getting it right (concerned about security), you should know the Security Manager is a part of Java Virtual Machine for some time now and nodes have a file that allows what other classes can and cant execute/do on that system (linux/windows/others).
Anyway, the authors should be able to explain that better that I.
Right, but my question was more along the lines of, how easy would it be to convert/use jppf for a generic command execution framework. Like, send it java classes which do a Runtime.exec of some lengthy process like creating rainbow tables.
From what I've read, jppf is more for executing Java, whatever that may be. But, unless I'm missing something, it could be used for what I said above with little to no modification, except maybe modifying security constraints already in place.
As Pedro mentioned, the security mechanism in JPPF lets you adjust the nodes permissions to what you need.
From there, you can certainly submit tasks that invoke Runtime.exec() (actually the preferred way since 1.5 is to use ProcessBuilder).
What I'd be concerned about would be OS-related.
Unless you have full control over what nodes can actually be part of the grid, there would be no guarantee that the OS can execute a command like grep.
Ideally it would be best to perform a Java equivalent of grep or whatever command you want to execute, with probably less efficiency. I know Ant has some built-in capabilities (see "replace" task for instance), but I doubt it's as complete and fast as grep.
Excellent, thank you for the quick response. Another question. Would it be possible to attach meta data to a node. For example, this node accepts this list of commands, or this node is running this OS. I know the OS part is easily accomplished with jakarta commons, however, the discovery of what command line programs the computer runs is not always that easy considering pathing issues.
Hmmm, now I see what you're getting at :-)
Unfortunately, what you're describing definitely doesn't fit in the architecture and design of JPPF.
Let me explain a bit. Obtaining OS medata for each node may not be trivial, but it's still the easy part in this.
Where trouble comes is when we want to use that metadata, like determining which tasks should be sent to which nodes, based on knowledge about the nodes and the tasks.
The JPPF server doesn't know anything about the tasks, nor about the code that's inside. It only sees them as a bunch of bytes. The role of the server is mostly that of a dispatcher/load balancer.
This serves two major purposes: high performance and security. Performance is attained by having the server do a minimum of work, excluding things like deserialization/unmarshalling as well as compression and decompression of the data sent over the network. By excluding those, we also ensure the confidentiality of the code in the tasks, since we refuse to know about it by design.
So in clear, this type of capability is not currently part of the vision for JPPF, I'd rather state this honestly. I just hope you're not too disappointed.
Do you are looking for something like Condor ClassAd?
They have a resource description language, a "query" language to express what resources does the job require to run and the Condor scheduler use to specify what runs where.
As Laurent said, yet it is not in your vision to do. Probably we will need something like this if we dive into data grid issues.
If you have anything in might about how describe and process meta-data about resources let us know.
My team investigated Condor, but we settled on Ptolemy as a starting point.
Thank you for all the information,