using Lime for distributed computing?
Brought to you by:
amy_murphy,
gpietro
Menu
▾
▴
Hi,
I have a scientific algorithm that distributes the computation to various clients on a network. Right now it uses IBM's TSpaces for the messaging, but I would like to replace it with something open-source. Would Lime work for this purpose?
I have been trying to implement it in Lime, but the notion of a fixed client that can communicate with a tuple space on a remote server seems to be something that Lime doesn't address. Am I wrong about this?
Thanks.
Hi there.
"Would Lime work for this?" -> I say yes.
"Am I wrong about this?" -> 90% yes, you are wrong. I don;t have much information from you to work with toward a complete answer but I would say you can do it. You can have an agent (say this is the client agent, demanding some service from a server) that extends the StationaryAgent class (I think that is the correct name) and you can also have another agent extending a StationaryAgent on a different machine to run your server code.
Good luck.
R~
Thanks for your response.
Sorry I didn't include any more details.
You are right about this. It is possible, but...
The main point is that I would like the clients to be "lightweight" in that they only request information from the server and send responses back. They should not need to start a separate thread (and take up a network port) waiting for messages from the server (which is what your idea would result in, correct?)
Ideally I would like to be able to write a simple client program that just connects to the tuple space on my server, sends a message ("I'm here and waiting for data to process"), and waits for a response ("here's your data"), and then starts computing.
It seems like the limitation that all tuple space messaging must be done from an Agent's run() method is what I am trying to get around.
Any other ideas would be much appreciated!
For your scenario, what you could do is to write a proxy agent at the server side for accessing to the tuple space and responding to your light-weight client side code using socket level code.
The question then would be, why tuple space in the first place. I guess it has some advantage for your application. If you are interested in tuplespace alone, then "lights" might be a better place to start than LIME.
Just my 2 cents.
Qingfeng
By "lights" I meant GP's work of "LighTS". You can access it from:
http://lights.sourceforge.net/
Have fun,
Qingfeng
From what you havre told me, Lime is probably not the best tool for you. Lime is more targeted toward peer-to-peer computing rather than strictly client-server, although it can be used for this purpose.
I would instead steer you toward LighTS. LighTS is simply a tuple space (a very simple tuple space) that I believe can be easily extended with an RMI interface. If you don't mind using RMI, then all your clients would need to do is make RMI calls to the server, and query results will be returned by the LighTS methods from afar.
Of course, this is not exactly the mechanism that you described...but the mechanism that you describe for exchanging information (namely make a request and it is pushed toward you) is not exactly what Linda was designed for (namely pull info out of the shared space). Lime provides this with Reactions (similar to events in TSpaces), but this is not available in LighTS.
With Lime, I'm afraid that your clients will be as "heavy-weight" as your server...meaning that each of them will need to have a local tuple space, and all the restrictions placed on them (e.g., that only an agent can actually call methods on the LimeTupleSpace. Also, although you won't necessarily see it from the programmer's perspective, Lime will keep connections open to receive data (events).
If you have any questions about LighTS, check out lights.sf.net and contact Gian Pietro Picco (the author), although he is probably watching this thread as well.
-Amy
Ok, let's take it step by step:
Like any piece of software, your clients will need an entry point. In this particular framework your clients have to be agents and their entry point will be the run method (versus main in C/C++ or Java). I don't think this is a problem for you. Then you will need to launch your agent(s) somehow and, since they don;t have a 'main()' they won;t be run by the JVM. You will load them in Lime and Lime will take care of their execution. Of course an agent will take an extra thread, but I think this is expected, right?
For communication purposes, you will not need to open any network connections by yourself, you don't need to worry about ports, sockets and other stuff. Once you said 'out' to a tuple, it is Lime who takes care of the transportation (and believe me Lime doesn't open a new port for every single client you launch). If you want to simply send a tuple and receive a tuple, I could show you the few lines of code it takes to do this and also explain you how the thing works (I won;t do it in this message but if you ask me to I will).
The bottom line is that "I'm here and waiting for data to process" and "here's your data" will work.
You have a point about the messaging being done from the run() method but you can get around it. Immagina an agent that monitors a queue. All this agent does is read job requests from the queue and execute them. These job requests could be like "write this to the tuple space" or "go fetch that from tuple space", .. etc. In this case the agent can run an "infinite" message processing - like loop and do what you're asking it to do, while you can get aroung the run method thing...
Is it any more clear now?
R~
Thanks for your response.
Sorry I didn't include any more details.
You are right about this. It is possible, but...
The main point is that I would like the clients to be "lightweight" in that they only request information from the server and send responses back. They should not need to start a separate thread (and take up a network port) waiting for messages from the server (which is what your idea would result in, correct?)
Ideally I would like to be able to write a simple client program that just connects to the tuple space on my server, sends a message ("I'm here and waiting for data to process"), and waits for a response ("here's your data"), and then starts computing.
It seems like the limitation that all tuple space messaging must be done from an Agent's run() method is what I am trying to get around.
Any other ideas would be much appreciated!
I fully agree with Amy: Lime is definitely overkill for your application (as far as I understand) if a client-server architecture is what you have in mind, i.e., if tasks must sit on a server, and be picked up by client. For this task, LighTS (with some hacking to enable remote access) might be useful to you although---be warned---the current implementation of the tuple space is simply a Vector...
If, instead, your need is to enable multiple process in the distributed system to "advertise" the presence of a new task (by dumping into their local tuple space), react to it by picking it up (wherever it is), and process it, then Lime may actually be the right solution.
As Amy pointed out, a key question is about the architecture you have in mind: if it's client-server, then it's doable but somewhat overkill in Lime. If instead you want to free yourself from a centralized server, then Lime may be the right tool.
Hope this helps.
Regards,
Gian Pietro