#21 Excessive communications when lots of pending jobs

[Section8]

As part of my testing for writing documentation, I set up
a grid and loaded it up a bit.

I noticed when there were lots of jobs (100+ in this
case) there is a lot of redundant traffic between the grid
and the engine. Each and every suitable job is
transmitted to each engine when the engine needs a
new job.

There needs to be a cache of available jobs on each
engine to avoid all this traffic or this project is not going
to scale well at all.

Ideally each job will generate at most 2 messages to
each engine (an announcement and a
completion/removal message). Its probably possible to
submit each job to a subset of the engines and only tell
the other engines if none of those can complete it
(although there is a traffic/response time trade off here)