I announce a pre-alpha version of TaskPoolKit:
TaskPoolKit is a package that mantains a TaskPool, a ThreadPool and Workers.
The Workers process tasks, the ThreadPool maintains a dynamic number of
and TaskPool maintain a queue of tasks to be processed.
The public interface of the package is TaskPool, the other are of
intrapackage use only.
TaskPool has this public methods:
addTask(task, *args, **kargs): add a task to be procesed by a worker
if there are to many tasks
in the queue, wait, if you can't
add even waiting then raise
addTaskNoWait: the same as addTask but raises Full without retrying
asyncCall(function, *args, **kargs): return a ReturnValue instance:
true if the work is finished
returnvalue() return the
The function is procesed
The number of workers are dynamic, they are maintained between minThread and
In the __init__ file is a fast hack that adds a TaskPool instance to
you can do:
application.addTask(tas, *arg, **kargs) and the other methods
You can process long jobs in your servlet (for example indexing a database,
do ftp, start sending emails, ...)
It is simple, write in your code:
app = self.application()
app.addTask(self.doLongJob, arg1, arg2)
A worker will execute doLongJob(arg1, arg2)
My idea is that WebKit can use internally TaskPool ( self.addTask
request.handleRequest) ) .
I think that this is the first step to a MultiAppServer arquitecture. Each
addTask from a unique taskPool. Then all the threads are shared between all
TaskPool.py has a test() function, try it.
Make a better integration of Application and TaskPool:
include in Application self._taskPool and the public methods of TaskPool
directly accesible from Application.
Include TaskPool.shutDown() in Application.shutDown()
Include TaskPool parameters in config file
What do you think? I need feedback.
Can this be included in Webware?
If all is correct I will work in a MultiAppServer Webware. My idea is:
AppServer------> Application <---- TaskPool
And application has multiple appservers. Each appserver listen a port
in an eternal cicle.
Each appserver add tasks to the TaskPool, the workers execute the task.
All the AppServers share the same TaskPool, and thus they share
the same threads.