RE: [Babeldoc-devel] Late night musings on parallel execution of pipeline stage results

[Leech, J. <jl...@vi...>]

Bruce,

I haven't looked at the refactoring yet but that sounds excellent.  I had
thought about using a proper ThreadPool but didn't have one and didn't think
to look for one.  The problem with not letting a PipelineStage specify
maxThreads is that you can end up with one PipelineStage using all the
available threads, and starving other stages that you want to run
concurrently.  Also, if you are using up a limited resource in the stage
e.g. database connections, it doesn't make sense to have more threads
running than connections.  But once you go with a thread pool, there should
also be a maxThreads there as well.

-Jonathan

-----Original Message-----
From: Bruce McDonald [mailto:br...@mc...]
Sent: Tuesday, July 15, 2003 11:29 PM
To: bab...@li...
Subject: [Babeldoc-devel] Late night musings on parallel execution of
pipeline stage results


Jonathan, All:

I have refactored the PipelineStageFactory so that methods:

processPipelineStageResults    		(non-threaded)
processPipelineStageResultsParallel	(threaded)

are as similar as possible and all possible code overlaps are extracted to 
other methods.  At the core, the difference between the two of them is the 
maxThreads argument.  This argument indicates the number of threads to be 
spawned to handle the processing of the pipeline stage results.  I contend 
that this is an attribute of the pipeline stage factory and should be
managed 
by a threadpool.  In fact, in direct contradiction to my prior email on this

matter, an even simpler implementation would be to subclass the non-threaded

implementation and override the processPipelineStageResults method to handle

all the threaded complexity. 

thoughts?

regards,
Bruce.

PS.  My thanks to Jonathan for stinging my lazy butt out of a destructive 
cycle of Rise Of Nations multiplayer games :)    

PPS.  I would seriously advise everyone to look at this code - it looks
solid.  
That ThreadLocal thing was new to me.


-------------------------------------------------------
This SF.net email is sponsored by: VM Ware
With VMware you can run multiple operating systems on a single machine.
WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
_______________________________________________
Babeldoc-devel mailing list
Bab...@li...
https://lists.sourceforge.net/lists/listinfo/babeldoc-devel