I'd like to get something like this going:
1. client machines periodically collect data (easy with scheduling:timer) and when collection completes, notify a central server that the data is ready (using jmx:client).
2. the central server should have a pool of processing jobs so that it can process some number of client data files in parallel, but not too many so it doesn't overload the system. parallel and throttle are obviously of use here.
Is this something I can accomplish with OddJob? I've got the first part working, but I'm struggling on how to configure the second part.
Hi - The idea of job queue came up a few weeks ago. Please see this thread: http://sourceforge.net/projects/ojob/forums/forum/496818/topic/4881571 to see if it might be along the lines of what you want.
It wouldn't be too hard to extend the idea to use JMS but we would need to think about how resilient this would be because Oddjob has no transaction support and so recovery would be difficult.
Unfortunately I can't think of a simple out of the box solution that Oddjob could provide for your problem.
Wow, thank you! This is looking very encouraging. I got your example from that thread going without a hitch, then modified it so that I have clients pushing many jobs into the queue and watched the server handle them as desired using <throttle>.
Is there a way to set a maximum queue size, or a way to trigger a job if the queue hits a certain size? Just looking for protection against unexpected cases like a runaway client flooding the queue.
Glad you got it working. The quick solution to monitor the queue would be to use a timer and a sequential job with a child check job followed by a mail job. The check job would use 'gt' to check the size property of the queue and if it succeeds then the sequential would move on and the mail would be sent.
See http://rgordon.co.uk/oddjob/reference/org/oddjob/jobs/CheckJob.html for some example of how to use the check job.
Log in to post a comment.