babeldoc-devel Mailing List for Babeldoc: Universal Document Processor (Page 95)

Brought to you by: dejank, santibegue, triphop, vharcq

babeldoc-devel — Babeldoc developer list

You can subscribe to this list here.

2003	Jan	Feb (14)	Mar (107)	Apr (211)	May (93)	Jun (158)	Jul (159)	Aug (368)	Sep (188)	Oct (151)	Nov (115)	Dec (98)
2004	Jan (25)	Feb	Mar (33)	Apr (28)	May (116)	Jun (2)	Jul (117)	Aug (19)	Sep (9)	Oct (2)	Nov	Dec (4)
2005	Jan	Feb	Mar	Apr	May (1)	Jun	Jul	Aug	Sep	Oct (2)	Nov (9)	Dec
2006	Jan	Feb	Mar (22)	Apr	May	Jun	Jul	Aug	Sep	Oct	Nov	Dec
2007	Jan	Feb	Mar (6)	Apr	May	Jun	Jul	Aug (267)	Sep	Oct	Nov (6)	Dec (512)
2008	Jan (187)	Feb	Mar	Apr	May	Jun (3)	Jul (6)	Aug	Sep	Oct	Nov	Dec
2011	Jan	Feb	Mar	Apr	May	Jun	Jul	Aug	Sep	Oct (2)	Nov	Dec
2012	Jan	Feb	Mar	Apr (1)	May	Jun	Jul	Aug	Sep	Oct	Nov	Dec

Flat | Threaded

<< < 1 .. 93 94 95 96 97 .. 122 > >> (Page 95 of 122)

[Babeldoc-devel] babeldoc/modules/conversion/src/com/babeldoc/conversion/flatfile/digester - New directory

From: <tr...@us...> - 2003-07-19 12:47:31

Update of /cvsroot/babeldoc/babeldoc/modules/conversion/src/com/babeldoc/conversion/flatfile/digester
In directory sc8-pr-cvs1:/tmp/cvs-serv22005/digester

Log Message:
Directory /cvsroot/babeldoc/babeldoc/modules/conversion/src/com/babeldoc/conversion/flatfile/digester added to the repository

[Babeldoc-devel] babeldoc/modules/conversion/src/com/babeldoc/conversion/flatfile - New directory

From: <tr...@us...> - 2003-07-19 01:28:06

Update of /cvsroot/babeldoc/babeldoc/modules/conversion/src/com/babeldoc/conversion/flatfile
In directory sc8-pr-cvs1:/tmp/cvs-serv4180/flatfile

Log Message:
Directory /cvsroot/babeldoc/babeldoc/modules/conversion/src/com/babeldoc/conversion/flatfile added to the repository

[Babeldoc-devel] babeldoc/modules/conversion/src/com/babeldoc/conversion/pipeline/stage XlsToXmlPipelineStage.java,1.1,1.2

From: <de...@us...> - 2003-07-18 11:58:19

Update of /cvsroot/babeldoc/babeldoc/modules/conversion/src/com/babeldoc/conversion/pipeline/stage
In directory sc8-pr-cvs1:/tmp/cvs-serv29766/modules/conversion/src/com/babeldoc/conversion/pipeline/stage

Modified Files:
	XlsToXmlPipelineStage.java 
Log Message:
Fixed some bugs. Removed some unnecessery stuff

Index: XlsToXmlPipelineStage.java
===================================================================
RCS file: /cvsroot/babeldoc/babeldoc/modules/conversion/src/com/babeldoc/conversion/pipeline/stage/XlsToXmlPipelineStage.java,v
retrieving revision 1.1
retrieving revision 1.2
diff -C2 -d -r1.1 -r1.2
*** XlsToXmlPipelineStage.java	17 Jul 2003 14:45:23 -0000	1.1
--- XlsToXmlPipelineStage.java	18 Jul 2003 11:54:49 -0000	1.2
***************
*** 1,11 ****
! /*
!  * Created on Jul 16, 2003
   *
-  * To change this generated comment go to 
-  * Window>Preferences>Java>Code Generation>Code and Comments
   */
  package com.babeldoc.conversion.pipeline.stage;
  
  import java.io.IOException;
  import java.util.Collection;
  import org.apache.poi.hssf.usermodel.HSSFCell;
--- 1,71 ----
! /* ====================================================================
!  * The Apache Software License, Version 1.1
!  *
!  * Copyright (c) 2000 The Apache Software Foundation.  All rights
!  * reserved.
!  *
!  * Redistribution and use in source and binary forms, with or without
!  * modification, are permitted provided that the following conditions
!  * are met:
!  *
!  * 1. Redistributions of source code must retain the above copyright
!  *    notice, this list of conditions and the following disclaimer.
!  *
!  * 2. Redistributions in binary form must reproduce the above copyright
!  *    notice, this list of conditions and the following disclaimer in
!  *    the documentation and/or other materials provided with the
!  *    distribution.
!  *
!  * 3. The end-user documentation included with the redistribution,
!  *    if any, must include the following acknowledgment:
!  *       "This product includes software developed by the
!  *        Apache Software Foundation (http://www.apache.org/)."
!  *    Alternately, this acknowledgment may appear in the software itself,
!  *    if and wherever such third-party acknowledgments normally appear.
!  *
!  * 4. The names "Apache" and "Apache Software Foundation" must
!  *    not be used to endorse or promote products derived from this
!  *    software without prior written permission. For written
!  *    permission, please contact ap...@ap....
!  *
!  * 5. Products derived from this software may not be called "Apache",
!  *    nor may "Apache" appear in their name, without prior written
!  *    permission of the Apache Software Foundation.
!  *
!  * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
!  * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
!  * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
!  * DISCLAIMED.  IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR
!  * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
!  * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
!  * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
!  * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
!  * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
!  * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
!  * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
!  * SUCH DAMAGE.
!  * ====================================================================
!  *
!  * This software consists of voluntary contributions made by many
!  * individuals on behalf of the Apache Software Foundation.  For more
!  * information on the Apache Software Foundation, please see
!  * <http://www.apache.org/>.
!  *
!  * Portions of this software are based upon public domain software
!  * originally written at the National Center for Supercomputing Applications,
!  * University of Illinois, Urbana-Champaign.
!  * ====================================================================
!  *
!  * Babeldoc:  The Universal Document Processor
!  *
!  * $Header$
!  * $DateTime$
!  * $Author$
   *
   */
  package com.babeldoc.conversion.pipeline.stage;
  
  import java.io.IOException;
+ import java.util.ArrayList;
  import java.util.Collection;
  import org.apache.poi.hssf.usermodel.HSSFCell;
***************
*** 18,22 ****
  import org.dom4j.Element;
  
- import com.babeldoc.core.option.IConfigInfo;
  import com.babeldoc.core.pipeline.PipelineDocument;
  import com.babeldoc.core.pipeline.PipelineException;
--- 78,81 ----
***************
*** 33,39 ****
  public class XlsToXmlPipelineStage extends PipelineStage {
  
! 	public static String DATE_FORMAT = "dd.mm.yyyy";
! 
! 	public XlsToXmlPipelineStage(IConfigInfo info) {
  		super(new PipelineStageInfo() {
  			public String getName() {
--- 92,96 ----
  public class XlsToXmlPipelineStage extends PipelineStage {
  
! 	public XlsToXmlPipelineStage() {
  		super(new PipelineStageInfo() {
  			public String getName() {
***************
*** 46,51 ****
  
  			public Collection getTypeSpecificOptions() {
! 
! 				return null;
  			}
  		});
--- 103,107 ----
  
  			public Collection getTypeSpecificOptions() {
! 				return new ArrayList();
  			}
  		});
***************
*** 92,96 ****
  					if (row == null)
  						continue;
! 					Element rowElement = document.addElement("row");
  					rowElement.addAttribute(
  						"rowNum",
--- 148,152 ----
  					if (row == null)
  						continue;
! 					Element rowElement = sheetElement.addElement("row");
  					rowElement.addAttribute(
  						"rowNum",
***************
*** 107,114 ****
  						cellCount++;
  						Element cellElement = rowElement.addElement("cell");
! 						cellElement.addAttribute("colNum", String.valueOf(p));
! 						cellElement.addAttribute(
! 							"col",
! 							String.valueOf(cell.getCellNum()));
  
  						int cellType = cell.getCellType();
--- 163,167 ----
  						cellCount++;
  						Element cellElement = rowElement.addElement("cell");
! 						cellElement.addAttribute("colNum", String.valueOf(p));						
  
  						int cellType = cell.getCellType();
***************
*** 143,151 ****
  					String.valueOf(rowCount));
  			}
! 
  			PipelineDocument doc =
  				new PipelineDocument(
  					this.getDocument(),
! 					document.toString().getBytes());
  			return processHelper(doc);
  		} catch (IOException e) {
--- 196,204 ----
  					String.valueOf(rowCount));
  			}
! 			
  			PipelineDocument doc =
  				new PipelineDocument(
  					this.getDocument(),
! 						document.asXML().getBytes());
  			return processHelper(doc);
  		} catch (IOException e) {

RE: [Babeldoc-devel] Late night musings on parallel execution ofp ipeline stage results

From: Leech, J. <jl...@vi...> - 2003-07-17 19:49:33

Bruce,

The default maxThreads for a pipelinestage definitely belongs in its own
parameter.  
It looks to me from running pipelines and looking at the code that the
AsyncPipelineStageProcessor doesn't adhere to the overall maxThreads
parameter.
And it looks to me like the ThreadPooledPipelineStageProcessor doesn't
adhere to the maxThreads specified in pipeline stages...
I think that we don't need both processors, and it would be harder to fix
AsyncPipelineStageProcessor than it would be to fix
ThreadPooledPipelineStageProcessor. 
What do you think?

-Jonathan

-----Original Message-----
From: McDonald, Bruce [mailto:Bru...@ba...]
Sent: Thursday, July 17, 2003 11:30 AM
To: Leech, Jonathan; Bruce McDonald;
bab...@li...
Subject: RE: [Babeldoc-devel] Late night musings on parallel execution
ofpipeline stage results

Its an upper limit on the number of threads that can be spawned.  At the
moment its also the number of threads that will be spawned if the maxThreads
is not specified at the pipeline stage level.  I think that this is probably
something that needs its own parameter.

-----Original Message-----
From: Leech, Jonathan [mailto:jl...@vi...]
Sent: Thursday, July 17, 2003 1:27 PM
To: McDonald, Bruce; Bruce McDonald;
bab...@li...
Subject: RE: [Babeldoc-devel] Late night musings on parallel execution
ofpipeline stage results

OK.  That makes sense.  I have another question: Is the other maxThreads --
the one for the pipeline -- an overall maxThreads, or a default for stages
that don't specify maxThreads?

-Jonathan

-----Original Message-----
From: McDonald, Bruce [mailto:Bru...@ba...]
Sent: Thursday, July 17, 2003 11:23 AM
To: Leech, Jonathan; Bruce McDonald;
bab...@li...
Subject: RE: [Babeldoc-devel] Late night musings on parallel execution
ofpipeline stage results

No. then in those stages that need to not run multithreaded, put
maxThreads=1

-----Original Message-----
From: Leech, Jonathan [mailto:jl...@vi...]
Sent: Thursday, July 17, 2003 1:17 PM
To: 'Bruce McDonald'; bab...@li...
Subject: RE: [Babeldoc-devel] Late night musings on parallel execution
ofpipeline stage results

Bruce,

I need to be able to mix parallel and serial pipeline processing, as I have
stages that depend on other stages.  To accomplish this, in my pipeline I
have some non-threaded, ordered router stages that start threaded stages.
With your changes, it looks like I can't have the non-threaded stages,
unless I want the whole pipeline to be non-threaded.  Am I missing
something?

-Jonathan

-----Original Message-----
From: Bruce McDonald [mailto:br...@mc...]
Sent: Wednesday, July 16, 2003 4:55 PM
To: Leech, Jonathan; bab...@li...
Subject: Re: [Babeldoc-devel] Late night musings on parallel execution
of pipeline stage results

Ok all,

I have been kinda busy today working on this.  I have checked in code that 
will make the threading in the pipelines a policy of the pipeline stage 
factory and configurable from that point.  Please review the affected files.

To get a good idea of how to use this - please take a look at the 
core/examples/threads example.  This contains three pipelines (with a single

definition file) which implements all of the three threading processors.

1.  SyncPipelineStageProcessor  (this is the current implementation and is
the 
default use unless the pipeline is configured otherwise.)
2.  AsyncPipelineStageProcessor (this is basically Jonathans code extracted 
from the the PipelineStageFactory.  This creates threads as necessary and 
then waits for them to complete)
3.  ThreadPooledPipelineStageProcessor  (this is a thread pooled 
implementation that I whipped into shape today - it still leaves something
to 
be desired)

Here are some notes:

1.  The standard behavior is retained unless you configure it differently.
2.  All the configuration of the threading for pipelines is done in the 
pipeline/config.properties file. 
3.  The Async processor accepts hints from the pipeline definition file 
(maxThreads) 
4.  I have added three required properties to the services/query.properties 
file to look up the following service names: sync; async; threadpool

regards,
Bruce.

On Wednesday 16 July 2003 10:45 am, Leech, Jonathan wrote:
> Bruce,
>
> I haven't looked at the refactoring yet but that sounds excellent.  I had
> thought about using a proper ThreadPool but didn't have one and didn't
> think to look for one.  The problem with not letting a PipelineStage
> specify maxThreads is that you can end up with one PipelineStage using all
> the available threads, and starving other stages that you want to run
> concurrently.  Also, if you are using up a limited resource in the stage
> e.g. database connections, it doesn't make sense to have more threads
> running than connections.  But once you go with a thread pool, there
should
> also be a maxThreads there as well.
>
> -Jonathan
>
> -----Original Message-----
> From: Bruce McDonald [mailto:br...@mc...]
> Sent: Tuesday, July 15, 2003 11:29 PM
> To: bab...@li...
> Subject: [Babeldoc-devel] Late night musings on parallel execution of
> pipeline stage results
>
>
> Jonathan, All:
>
> I have refactored the PipelineStageFactory so that methods:
>
> processPipelineStageResults    		(non-threaded)
> processPipelineStageResultsParallel	(threaded)
>
> are as similar as possible and all possible code overlaps are extracted to
> other methods.  At the core, the difference between the two of them is the
> maxThreads argument.  This argument indicates the number of threads to be
> spawned to handle the processing of the pipeline stage results.  I contend
> that this is an attribute of the pipeline stage factory and should be
> managed
> by a threadpool.  In fact, in direct contradiction to my prior email on
> this
>
> matter, an even simpler implementation would be to subclass the
> non-threaded
>
> implementation and override the processPipelineStageResults method to
> handle
>
> all the threaded complexity.
>
> thoughts?
>
> regards,
> Bruce.
>
> PS.  My thanks to Jonathan for stinging my lazy butt out of a destructive
> cycle of Rise Of Nations multiplayer games :)
>
> PPS.  I would seriously advise everyone to look at this code - it looks
> solid.
> That ThreadLocal thing was new to me.
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

-------------------------------------------------------
This SF.net email is sponsored by: VM Ware
With VMware you can run multiple operating systems on a single machine.
WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
_______________________________________________
Babeldoc-devel mailing list
Bab...@li...
https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

-------------------------------------------------------
This SF.net email is sponsored by: VM Ware
With VMware you can run multiple operating systems on a single machine.
WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
_______________________________________________
Babeldoc-devel mailing list
Bab...@li...
https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

RE: [Babeldoc-devel] Late night musings on parallel execution ofpipeline stage results

From: McDonald, B. <Bru...@ba...> - 2003-07-17 17:30:14

Its an upper limit on the number of threads that can be spawned.  At the moment its also the number of threads that will be spawned if the maxThreads is not specified at the pipeline stage level.  I think that this is probably something that needs its own parameter.

-----Original Message-----
From: Leech, Jonathan [mailto:jl...@vi...]
Sent: Thursday, July 17, 2003 1:27 PM
To: McDonald, Bruce; Bruce McDonald;
bab...@li...
Subject: RE: [Babeldoc-devel] Late night musings on parallel execution
ofpipeline stage results

OK.  That makes sense.  I have another question: Is the other maxThreads --
the one for the pipeline -- an overall maxThreads, or a default for stages
that don't specify maxThreads?

-Jonathan

-----Original Message-----
From: McDonald, Bruce [mailto:Bru...@ba...]
Sent: Thursday, July 17, 2003 11:23 AM
To: Leech, Jonathan; Bruce McDonald;
bab...@li...
Subject: RE: [Babeldoc-devel] Late night musings on parallel execution
ofpipeline stage results

No. then in those stages that need to not run multithreaded, put
maxThreads=1

-----Original Message-----
From: Leech, Jonathan [mailto:jl...@vi...]
Sent: Thursday, July 17, 2003 1:17 PM
To: 'Bruce McDonald'; bab...@li...
Subject: RE: [Babeldoc-devel] Late night musings on parallel execution
ofpipeline stage results

Bruce,

I need to be able to mix parallel and serial pipeline processing, as I have
stages that depend on other stages.  To accomplish this, in my pipeline I
have some non-threaded, ordered router stages that start threaded stages.
With your changes, it looks like I can't have the non-threaded stages,
unless I want the whole pipeline to be non-threaded.  Am I missing
something?

-Jonathan

-----Original Message-----
From: Bruce McDonald [mailto:br...@mc...]
Sent: Wednesday, July 16, 2003 4:55 PM
To: Leech, Jonathan; bab...@li...
Subject: Re: [Babeldoc-devel] Late night musings on parallel execution
of pipeline stage results

Ok all,

I have been kinda busy today working on this.  I have checked in code that 
will make the threading in the pipelines a policy of the pipeline stage 
factory and configurable from that point.  Please review the affected files.

To get a good idea of how to use this - please take a look at the 
core/examples/threads example.  This contains three pipelines (with a single

definition file) which implements all of the three threading processors.

1.  SyncPipelineStageProcessor  (this is the current implementation and is
the 
default use unless the pipeline is configured otherwise.)
2.  AsyncPipelineStageProcessor (this is basically Jonathans code extracted 
from the the PipelineStageFactory.  This creates threads as necessary and 
then waits for them to complete)
3.  ThreadPooledPipelineStageProcessor  (this is a thread pooled 
implementation that I whipped into shape today - it still leaves something
to 
be desired)

Here are some notes:

1.  The standard behavior is retained unless you configure it differently.
2.  All the configuration of the threading for pipelines is done in the 
pipeline/config.properties file. 
3.  The Async processor accepts hints from the pipeline definition file 
(maxThreads) 
4.  I have added three required properties to the services/query.properties 
file to look up the following service names: sync; async; threadpool

regards,
Bruce.

On Wednesday 16 July 2003 10:45 am, Leech, Jonathan wrote:
> Bruce,
>
> I haven't looked at the refactoring yet but that sounds excellent.  I had
> thought about using a proper ThreadPool but didn't have one and didn't
> think to look for one.  The problem with not letting a PipelineStage
> specify maxThreads is that you can end up with one PipelineStage using all
> the available threads, and starving other stages that you want to run
> concurrently.  Also, if you are using up a limited resource in the stage
> e.g. database connections, it doesn't make sense to have more threads
> running than connections.  But once you go with a thread pool, there
should
> also be a maxThreads there as well.
>
> -Jonathan
>
> -----Original Message-----
> From: Bruce McDonald [mailto:br...@mc...]
> Sent: Tuesday, July 15, 2003 11:29 PM
> To: bab...@li...
> Subject: [Babeldoc-devel] Late night musings on parallel execution of
> pipeline stage results
>
>
> Jonathan, All:
>
> I have refactored the PipelineStageFactory so that methods:
>
> processPipelineStageResults    		(non-threaded)
> processPipelineStageResultsParallel	(threaded)
>
> are as similar as possible and all possible code overlaps are extracted to
> other methods.  At the core, the difference between the two of them is the
> maxThreads argument.  This argument indicates the number of threads to be
> spawned to handle the processing of the pipeline stage results.  I contend
> that this is an attribute of the pipeline stage factory and should be
> managed
> by a threadpool.  In fact, in direct contradiction to my prior email on
> this
>
> matter, an even simpler implementation would be to subclass the
> non-threaded
>
> implementation and override the processPipelineStageResults method to
> handle
>
> all the threaded complexity.
>
> thoughts?
>
> regards,
> Bruce.
>
> PS.  My thanks to Jonathan for stinging my lazy butt out of a destructive
> cycle of Rise Of Nations multiplayer games :)
>
> PPS.  I would seriously advise everyone to look at this code - it looks
> solid.
> That ThreadLocal thing was new to me.
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

-------------------------------------------------------
This SF.net email is sponsored by: VM Ware
With VMware you can run multiple operating systems on a single machine.
WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
_______________________________________________
Babeldoc-devel mailing list
Bab...@li...
https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

-------------------------------------------------------
This SF.net email is sponsored by: VM Ware
With VMware you can run multiple operating systems on a single machine.
WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
_______________________________________________
Babeldoc-devel mailing list
Bab...@li...
https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

RE: [Babeldoc-devel] Late night musings on parallel execution ofp ipeline stage results

From: Leech, J. <jl...@vi...> - 2003-07-17 17:28:26

OK.  That makes sense.  I have another question: Is the other maxThreads --
the one for the pipeline -- an overall maxThreads, or a default for stages
that don't specify maxThreads?

-Jonathan

-----Original Message-----
From: McDonald, Bruce [mailto:Bru...@ba...]
Sent: Thursday, July 17, 2003 11:23 AM
To: Leech, Jonathan; Bruce McDonald;
bab...@li...
Subject: RE: [Babeldoc-devel] Late night musings on parallel execution
ofpipeline stage results


No. then in those stages that need to not run multithreaded, put
maxThreads=1

-----Original Message-----
From: Leech, Jonathan [mailto:jl...@vi...]
Sent: Thursday, July 17, 2003 1:17 PM
To: 'Bruce McDonald'; bab...@li...
Subject: RE: [Babeldoc-devel] Late night musings on parallel execution
ofpipeline stage results


Bruce,

I need to be able to mix parallel and serial pipeline processing, as I have
stages that depend on other stages.  To accomplish this, in my pipeline I
have some non-threaded, ordered router stages that start threaded stages.
With your changes, it looks like I can't have the non-threaded stages,
unless I want the whole pipeline to be non-threaded.  Am I missing
something?

-Jonathan

-----Original Message-----
From: Bruce McDonald [mailto:br...@mc...]
Sent: Wednesday, July 16, 2003 4:55 PM
To: Leech, Jonathan; bab...@li...
Subject: Re: [Babeldoc-devel] Late night musings on parallel execution
of pipeline stage results


Ok all,

I have been kinda busy today working on this.  I have checked in code that 
will make the threading in the pipelines a policy of the pipeline stage 
factory and configurable from that point.  Please review the affected files.

To get a good idea of how to use this - please take a look at the 
core/examples/threads example.  This contains three pipelines (with a single

definition file) which implements all of the three threading processors.

1.  SyncPipelineStageProcessor  (this is the current implementation and is
the 
default use unless the pipeline is configured otherwise.)
2.  AsyncPipelineStageProcessor (this is basically Jonathans code extracted 
from the the PipelineStageFactory.  This creates threads as necessary and 
then waits for them to complete)
3.  ThreadPooledPipelineStageProcessor  (this is a thread pooled 
implementation that I whipped into shape today - it still leaves something
to 
be desired)

Here are some notes:

1.  The standard behavior is retained unless you configure it differently.
2.  All the configuration of the threading for pipelines is done in the 
pipeline/config.properties file. 
3.  The Async processor accepts hints from the pipeline definition file 
(maxThreads) 
4.  I have added three required properties to the services/query.properties 
file to look up the following service names: sync; async; threadpool

regards,
Bruce.

On Wednesday 16 July 2003 10:45 am, Leech, Jonathan wrote:
> Bruce,
>
> I haven't looked at the refactoring yet but that sounds excellent.  I had
> thought about using a proper ThreadPool but didn't have one and didn't
> think to look for one.  The problem with not letting a PipelineStage
> specify maxThreads is that you can end up with one PipelineStage using all
> the available threads, and starving other stages that you want to run
> concurrently.  Also, if you are using up a limited resource in the stage
> e.g. database connections, it doesn't make sense to have more threads
> running than connections.  But once you go with a thread pool, there
should
> also be a maxThreads there as well.
>
> -Jonathan
>
> -----Original Message-----
> From: Bruce McDonald [mailto:br...@mc...]
> Sent: Tuesday, July 15, 2003 11:29 PM
> To: bab...@li...
> Subject: [Babeldoc-devel] Late night musings on parallel execution of
> pipeline stage results
>
>
> Jonathan, All:
>
> I have refactored the PipelineStageFactory so that methods:
>
> processPipelineStageResults    		(non-threaded)
> processPipelineStageResultsParallel	(threaded)
>
> are as similar as possible and all possible code overlaps are extracted to
> other methods.  At the core, the difference between the two of them is the
> maxThreads argument.  This argument indicates the number of threads to be
> spawned to handle the processing of the pipeline stage results.  I contend
> that this is an attribute of the pipeline stage factory and should be
> managed
> by a threadpool.  In fact, in direct contradiction to my prior email on
> this
>
> matter, an even simpler implementation would be to subclass the
> non-threaded
>
> implementation and override the processPipelineStageResults method to
> handle
>
> all the threaded complexity.
>
> thoughts?
>
> regards,
> Bruce.
>
> PS.  My thanks to Jonathan for stinging my lazy butt out of a destructive
> cycle of Rise Of Nations multiplayer games :)
>
> PPS.  I would seriously advise everyone to look at this code - it looks
> solid.
> That ThreadLocal thing was new to me.
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel


-------------------------------------------------------
This SF.net email is sponsored by: VM Ware
With VMware you can run multiple operating systems on a single machine.
WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
_______________________________________________
Babeldoc-devel mailing list
Bab...@li...
https://lists.sourceforge.net/lists/listinfo/babeldoc-devel


-------------------------------------------------------
This SF.net email is sponsored by: VM Ware
With VMware you can run multiple operating systems on a single machine.
WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
_______________________________________________
Babeldoc-devel mailing list
Bab...@li...
https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

RE: [Babeldoc-devel] Late night musings on parallel execution ofpipeline stage results

From: McDonald, B. <Bru...@ba...> - 2003-07-17 17:23:21

No. then in those stages that need to not run multithreaded, put maxThreads=1

-----Original Message-----
From: Leech, Jonathan [mailto:jl...@vi...]
Sent: Thursday, July 17, 2003 1:17 PM
To: 'Bruce McDonald'; bab...@li...
Subject: RE: [Babeldoc-devel] Late night musings on parallel execution
ofpipeline stage results


Bruce,

I need to be able to mix parallel and serial pipeline processing, as I have
stages that depend on other stages.  To accomplish this, in my pipeline I
have some non-threaded, ordered router stages that start threaded stages.
With your changes, it looks like I can't have the non-threaded stages,
unless I want the whole pipeline to be non-threaded.  Am I missing
something?

-Jonathan

-----Original Message-----
From: Bruce McDonald [mailto:br...@mc...]
Sent: Wednesday, July 16, 2003 4:55 PM
To: Leech, Jonathan; bab...@li...
Subject: Re: [Babeldoc-devel] Late night musings on parallel execution
of pipeline stage results


Ok all,

I have been kinda busy today working on this.  I have checked in code that 
will make the threading in the pipelines a policy of the pipeline stage 
factory and configurable from that point.  Please review the affected files.

To get a good idea of how to use this - please take a look at the 
core/examples/threads example.  This contains three pipelines (with a single

definition file) which implements all of the three threading processors.

1.  SyncPipelineStageProcessor  (this is the current implementation and is
the 
default use unless the pipeline is configured otherwise.)
2.  AsyncPipelineStageProcessor (this is basically Jonathans code extracted 
from the the PipelineStageFactory.  This creates threads as necessary and 
then waits for them to complete)
3.  ThreadPooledPipelineStageProcessor  (this is a thread pooled 
implementation that I whipped into shape today - it still leaves something
to 
be desired)

Here are some notes:

1.  The standard behavior is retained unless you configure it differently.
2.  All the configuration of the threading for pipelines is done in the 
pipeline/config.properties file. 
3.  The Async processor accepts hints from the pipeline definition file 
(maxThreads) 
4.  I have added three required properties to the services/query.properties 
file to look up the following service names: sync; async; threadpool

regards,
Bruce.

On Wednesday 16 July 2003 10:45 am, Leech, Jonathan wrote:
> Bruce,
>
> I haven't looked at the refactoring yet but that sounds excellent.  I had
> thought about using a proper ThreadPool but didn't have one and didn't
> think to look for one.  The problem with not letting a PipelineStage
> specify maxThreads is that you can end up with one PipelineStage using all
> the available threads, and starving other stages that you want to run
> concurrently.  Also, if you are using up a limited resource in the stage
> e.g. database connections, it doesn't make sense to have more threads
> running than connections.  But once you go with a thread pool, there
should
> also be a maxThreads there as well.
>
> -Jonathan
>
> -----Original Message-----
> From: Bruce McDonald [mailto:br...@mc...]
> Sent: Tuesday, July 15, 2003 11:29 PM
> To: bab...@li...
> Subject: [Babeldoc-devel] Late night musings on parallel execution of
> pipeline stage results
>
>
> Jonathan, All:
>
> I have refactored the PipelineStageFactory so that methods:
>
> processPipelineStageResults    		(non-threaded)
> processPipelineStageResultsParallel	(threaded)
>
> are as similar as possible and all possible code overlaps are extracted to
> other methods.  At the core, the difference between the two of them is the
> maxThreads argument.  This argument indicates the number of threads to be
> spawned to handle the processing of the pipeline stage results.  I contend
> that this is an attribute of the pipeline stage factory and should be
> managed
> by a threadpool.  In fact, in direct contradiction to my prior email on
> this
>
> matter, an even simpler implementation would be to subclass the
> non-threaded
>
> implementation and override the processPipelineStageResults method to
> handle
>
> all the threaded complexity.
>
> thoughts?
>
> regards,
> Bruce.
>
> PS.  My thanks to Jonathan for stinging my lazy butt out of a destructive
> cycle of Rise Of Nations multiplayer games :)
>
> PPS.  I would seriously advise everyone to look at this code - it looks
> solid.
> That ThreadLocal thing was new to me.
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel


-------------------------------------------------------
This SF.net email is sponsored by: VM Ware
With VMware you can run multiple operating systems on a single machine.
WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
_______________________________________________
Babeldoc-devel mailing list
Bab...@li...
https://lists.sourceforge.net/lists/listinfo/babeldoc-devel


-------------------------------------------------------
This SF.net email is sponsored by: VM Ware
With VMware you can run multiple operating systems on a single machine.
WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
_______________________________________________
Babeldoc-devel mailing list
Bab...@li...
https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

RE: [Babeldoc-devel] Late night musings on parallel execution ofpipeline stage results

From: McDonald, B. <Bru...@ba...> - 2003-07-17 17:18:50

Take a look at that example in the core module - in the pipeline/config.properties:

#pipeline/config configuration
#Thu Jun 05 10:31:49 EDT 2003

pipeline.type=simple
pipeline.configFile=pipeline/pipeline

asyncpipeline.type=simple
asyncpipeline.configFile=pipeline/pipeline
asyncpipeline.processor.type=async
asyncpipeline.processor.maxThreads=4

pooledpipeline.type=simple
pooledpipeline.configFile=pipeline/pipeline
pooledpipeline.processor.type=threadpool
pooledpipeline.processor.poolSize=10

This defines three pipelines using the three different pipelinestage processors.  The first pipeline does not actually define a processor - it gets the synchronous processor.  This means that existing pipelines work as before.  The other two pipelines:  asyncpipeline and pooledpipeline used the async processor and the pooled processor respectively.

regards,
Bruce.


-----Original Message-----
From: Leech, Jonathan [mailto:jl...@vi...]
Sent: Thursday, July 17, 2003 12:56 PM
To: 'Bruce McDonald'; bab...@li...
Subject: RE: [Babeldoc-devel] Late night musings on parallel execution
ofpipeline stage results


Quick question -- how do I choose which processor to use?

-----Original Message-----
From: Bruce McDonald [mailto:br...@mc...]
Sent: Wednesday, July 16, 2003 4:55 PM
To: Leech, Jonathan; bab...@li...
Subject: Re: [Babeldoc-devel] Late night musings on parallel execution
of pipeline stage results


Ok all,

I have been kinda busy today working on this.  I have checked in code that 
will make the threading in the pipelines a policy of the pipeline stage 
factory and configurable from that point.  Please review the affected files.

To get a good idea of how to use this - please take a look at the 
core/examples/threads example.  This contains three pipelines (with a single

definition file) which implements all of the three threading processors.

1.  SyncPipelineStageProcessor  (this is the current implementation and is
the 
default use unless the pipeline is configured otherwise.)
2.  AsyncPipelineStageProcessor (this is basically Jonathans code extracted 
from the the PipelineStageFactory.  This creates threads as necessary and 
then waits for them to complete)
3.  ThreadPooledPipelineStageProcessor  (this is a thread pooled 
implementation that I whipped into shape today - it still leaves something
to 
be desired)

Here are some notes:

1.  The standard behavior is retained unless you configure it differently.
2.  All the configuration of the threading for pipelines is done in the 
pipeline/config.properties file. 
3.  The Async processor accepts hints from the pipeline definition file 
(maxThreads) 
4.  I have added three required properties to the services/query.properties 
file to look up the following service names: sync; async; threadpool

regards,
Bruce.

On Wednesday 16 July 2003 10:45 am, Leech, Jonathan wrote:
> Bruce,
>
> I haven't looked at the refactoring yet but that sounds excellent.  I had
> thought about using a proper ThreadPool but didn't have one and didn't
> think to look for one.  The problem with not letting a PipelineStage
> specify maxThreads is that you can end up with one PipelineStage using all
> the available threads, and starving other stages that you want to run
> concurrently.  Also, if you are using up a limited resource in the stage
> e.g. database connections, it doesn't make sense to have more threads
> running than connections.  But once you go with a thread pool, there
should
> also be a maxThreads there as well.
>
> -Jonathan
>
> -----Original Message-----
> From: Bruce McDonald [mailto:br...@mc...]
> Sent: Tuesday, July 15, 2003 11:29 PM
> To: bab...@li...
> Subject: [Babeldoc-devel] Late night musings on parallel execution of
> pipeline stage results
>
>
> Jonathan, All:
>
> I have refactored the PipelineStageFactory so that methods:
>
> processPipelineStageResults    		(non-threaded)
> processPipelineStageResultsParallel	(threaded)
>
> are as similar as possible and all possible code overlaps are extracted to
> other methods.  At the core, the difference between the two of them is the
> maxThreads argument.  This argument indicates the number of threads to be
> spawned to handle the processing of the pipeline stage results.  I contend
> that this is an attribute of the pipeline stage factory and should be
> managed
> by a threadpool.  In fact, in direct contradiction to my prior email on
> this
>
> matter, an even simpler implementation would be to subclass the
> non-threaded
>
> implementation and override the processPipelineStageResults method to
> handle
>
> all the threaded complexity.
>
> thoughts?
>
> regards,
> Bruce.
>
> PS.  My thanks to Jonathan for stinging my lazy butt out of a destructive
> cycle of Rise Of Nations multiplayer games :)
>
> PPS.  I would seriously advise everyone to look at this code - it looks
> solid.
> That ThreadLocal thing was new to me.
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel


-------------------------------------------------------
This SF.net email is sponsored by: VM Ware
With VMware you can run multiple operating systems on a single machine.
WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
_______________________________________________
Babeldoc-devel mailing list
Bab...@li...
https://lists.sourceforge.net/lists/listinfo/babeldoc-devel


-------------------------------------------------------
This SF.net email is sponsored by: VM Ware
With VMware you can run multiple operating systems on a single machine.
WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
_______________________________________________
Babeldoc-devel mailing list
Bab...@li...
https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

RE: [Babeldoc-devel] Late night musings on parallel execution of pipeline stage results

From: Leech, J. <jl...@vi...> - 2003-07-17 17:17:51

Bruce,

I need to be able to mix parallel and serial pipeline processing, as I have
stages that depend on other stages.  To accomplish this, in my pipeline I
have some non-threaded, ordered router stages that start threaded stages.
With your changes, it looks like I can't have the non-threaded stages,
unless I want the whole pipeline to be non-threaded.  Am I missing
something?

-Jonathan

-----Original Message-----
From: Bruce McDonald [mailto:br...@mc...]
Sent: Wednesday, July 16, 2003 4:55 PM
To: Leech, Jonathan; bab...@li...
Subject: Re: [Babeldoc-devel] Late night musings on parallel execution
of pipeline stage results

Ok all,

I have been kinda busy today working on this.  I have checked in code that 
will make the threading in the pipelines a policy of the pipeline stage 
factory and configurable from that point.  Please review the affected files.

To get a good idea of how to use this - please take a look at the 
core/examples/threads example.  This contains three pipelines (with a single

definition file) which implements all of the three threading processors.

1.  SyncPipelineStageProcessor  (this is the current implementation and is
the 
default use unless the pipeline is configured otherwise.)
2.  AsyncPipelineStageProcessor (this is basically Jonathans code extracted 
from the the PipelineStageFactory.  This creates threads as necessary and 
then waits for them to complete)
3.  ThreadPooledPipelineStageProcessor  (this is a thread pooled 
implementation that I whipped into shape today - it still leaves something
to 
be desired)

Here are some notes:

1.  The standard behavior is retained unless you configure it differently.
2.  All the configuration of the threading for pipelines is done in the 
pipeline/config.properties file. 
3.  The Async processor accepts hints from the pipeline definition file 
(maxThreads) 
4.  I have added three required properties to the services/query.properties 
file to look up the following service names: sync; async; threadpool

regards,
Bruce.

On Wednesday 16 July 2003 10:45 am, Leech, Jonathan wrote:
> Bruce,
>
> I haven't looked at the refactoring yet but that sounds excellent.  I had
> thought about using a proper ThreadPool but didn't have one and didn't
> think to look for one.  The problem with not letting a PipelineStage
> specify maxThreads is that you can end up with one PipelineStage using all
> the available threads, and starving other stages that you want to run
> concurrently.  Also, if you are using up a limited resource in the stage
> e.g. database connections, it doesn't make sense to have more threads
> running than connections.  But once you go with a thread pool, there
should
> also be a maxThreads there as well.
>
> -Jonathan
>
> -----Original Message-----
> From: Bruce McDonald [mailto:br...@mc...]
> Sent: Tuesday, July 15, 2003 11:29 PM
> To: bab...@li...
> Subject: [Babeldoc-devel] Late night musings on parallel execution of
> pipeline stage results
>
>
> Jonathan, All:
>
> I have refactored the PipelineStageFactory so that methods:
>
> processPipelineStageResults    		(non-threaded)
> processPipelineStageResultsParallel	(threaded)
>
> are as similar as possible and all possible code overlaps are extracted to
> other methods.  At the core, the difference between the two of them is the
> maxThreads argument.  This argument indicates the number of threads to be
> spawned to handle the processing of the pipeline stage results.  I contend
> that this is an attribute of the pipeline stage factory and should be
> managed
> by a threadpool.  In fact, in direct contradiction to my prior email on
> this
>
> matter, an even simpler implementation would be to subclass the
> non-threaded
>
> implementation and override the processPipelineStageResults method to
> handle
>
> all the threaded complexity.
>
> thoughts?
>
> regards,
> Bruce.
>
> PS.  My thanks to Jonathan for stinging my lazy butt out of a destructive
> cycle of Rise Of Nations multiplayer games :)
>
> PPS.  I would seriously advise everyone to look at this code - it looks
> solid.
> That ThreadLocal thing was new to me.
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

-------------------------------------------------------
This SF.net email is sponsored by: VM Ware
With VMware you can run multiple operating systems on a single machine.
WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
_______________________________________________
Babeldoc-devel mailing list
Bab...@li...
https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

RE: [Babeldoc-devel] Late night musings on parallel execution of pipeline stage results

From: Leech, J. <jl...@vi...> - 2003-07-17 16:57:05

Quick question -- how do I choose which processor to use?

-----Original Message-----
From: Bruce McDonald [mailto:br...@mc...]
Sent: Wednesday, July 16, 2003 4:55 PM
To: Leech, Jonathan; bab...@li...
Subject: Re: [Babeldoc-devel] Late night musings on parallel execution
of pipeline stage results


Ok all,

I have been kinda busy today working on this.  I have checked in code that 
will make the threading in the pipelines a policy of the pipeline stage 
factory and configurable from that point.  Please review the affected files.

To get a good idea of how to use this - please take a look at the 
core/examples/threads example.  This contains three pipelines (with a single

definition file) which implements all of the three threading processors.

1.  SyncPipelineStageProcessor  (this is the current implementation and is
the 
default use unless the pipeline is configured otherwise.)
2.  AsyncPipelineStageProcessor (this is basically Jonathans code extracted 
from the the PipelineStageFactory.  This creates threads as necessary and 
then waits for them to complete)
3.  ThreadPooledPipelineStageProcessor  (this is a thread pooled 
implementation that I whipped into shape today - it still leaves something
to 
be desired)

Here are some notes:

1.  The standard behavior is retained unless you configure it differently.
2.  All the configuration of the threading for pipelines is done in the 
pipeline/config.properties file. 
3.  The Async processor accepts hints from the pipeline definition file 
(maxThreads) 
4.  I have added three required properties to the services/query.properties 
file to look up the following service names: sync; async; threadpool

regards,
Bruce.

On Wednesday 16 July 2003 10:45 am, Leech, Jonathan wrote:
> Bruce,
>
> I haven't looked at the refactoring yet but that sounds excellent.  I had
> thought about using a proper ThreadPool but didn't have one and didn't
> think to look for one.  The problem with not letting a PipelineStage
> specify maxThreads is that you can end up with one PipelineStage using all
> the available threads, and starving other stages that you want to run
> concurrently.  Also, if you are using up a limited resource in the stage
> e.g. database connections, it doesn't make sense to have more threads
> running than connections.  But once you go with a thread pool, there
should
> also be a maxThreads there as well.
>
> -Jonathan
>
> -----Original Message-----
> From: Bruce McDonald [mailto:br...@mc...]
> Sent: Tuesday, July 15, 2003 11:29 PM
> To: bab...@li...
> Subject: [Babeldoc-devel] Late night musings on parallel execution of
> pipeline stage results
>
>
> Jonathan, All:
>
> I have refactored the PipelineStageFactory so that methods:
>
> processPipelineStageResults    		(non-threaded)
> processPipelineStageResultsParallel	(threaded)
>
> are as similar as possible and all possible code overlaps are extracted to
> other methods.  At the core, the difference between the two of them is the
> maxThreads argument.  This argument indicates the number of threads to be
> spawned to handle the processing of the pipeline stage results.  I contend
> that this is an attribute of the pipeline stage factory and should be
> managed
> by a threadpool.  In fact, in direct contradiction to my prior email on
> this
>
> matter, an even simpler implementation would be to subclass the
> non-threaded
>
> implementation and override the processPipelineStageResults method to
> handle
>
> all the threaded complexity.
>
> thoughts?
>
> regards,
> Bruce.
>
> PS.  My thanks to Jonathan for stinging my lazy butt out of a destructive
> cycle of Rise Of Nations multiplayer games :)
>
> PPS.  I would seriously advise everyone to look at this code - it looks
> solid.
> That ThreadLocal thing was new to me.
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel


-------------------------------------------------------
This SF.net email is sponsored by: VM Ware
With VMware you can run multiple operating systems on a single machine.
WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
_______________________________________________
Babeldoc-devel mailing list
Bab...@li...
https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

[Babeldoc-devel] FW: Me (from work)

From: McDonald, B. <Bru...@ba...> - 2003-07-17 15:04:07

All,

We need to start getting ready for 1.2.  This means that the current 1.1 development branch needs to start stabilizing.  To this end, I propose that we impose a feature freeze and then a code freeze on MAIN.  This will released to the sourceforce site as interim releases.  Here is my proposal:

0.  Series of interim releases (probably about 4-6)
1.  Feature freeze (2003-10-1)
2.  Release of two candidates RC1, RC2
3.  Code freeze (2003-11-1)
4.  Release of candidate RC3
5.  Release of 1.2 final version.

Please comment - votes, etc

regards,
Bruce.

[Babeldoc-devel] babeldoc/modules/conversion/config/service query.properties,1.3,1.4

From: <de...@us...> - 2003-07-17 14:49:48

Update of /cvsroot/babeldoc/babeldoc/modules/conversion/config/service
In directory sc8-pr-cvs1:/tmp/cvs-serv18130/modules/conversion/config/service

Modified Files:
	query.properties 
Log Message:
Excel files to XML conversion pipeline stage

Index: query.properties
===================================================================
RCS file: /cvsroot/babeldoc/babeldoc/modules/conversion/config/service/query.properties,v
retrieving revision 1.3
retrieving revision 1.4
diff -C2 -d -r1.3 -r1.4
*** query.properties	21 May 2003 22:07:11 -0000	1.3
--- query.properties	17 Jul 2003 14:49:45 -0000	1.4
***************
*** 1,3 ****
  Command.flat2xml=com.babeldoc.conversion.ConversionClient
  PipelineStage.FlatToXml=com.babeldoc.conversion.pipeline.stage.FlatToXmlPipelineStage
! 
--- 1,3 ----
  Command.flat2xml=com.babeldoc.conversion.ConversionClient
  PipelineStage.FlatToXml=com.babeldoc.conversion.pipeline.stage.FlatToXmlPipelineStage
! PipelineStage.XlsToXml=com.babeldoc.conversion.pipeline.stage.XlsToXmlPipelineStage

[Babeldoc-devel] babeldoc/modules/conversion build.xml,1.8,1.9

From: <de...@us...> - 2003-07-17 14:49:48

Update of /cvsroot/babeldoc/babeldoc/modules/conversion
In directory sc8-pr-cvs1:/tmp/cvs-serv18130/modules/conversion

Modified Files:
	build.xml 
Log Message:
Excel files to XML conversion pipeline stage

Index: build.xml
===================================================================
RCS file: /cvsroot/babeldoc/babeldoc/modules/conversion/build.xml,v
retrieving revision 1.8
retrieving revision 1.9
diff -C2 -d -r1.8 -r1.9
*** build.xml	27 Jun 2003 01:44:06 -0000	1.8
--- build.xml	17 Jul 2003 14:49:45 -0000	1.9
***************
*** 27,30 ****
--- 27,33 ----
  				  <include name="*"/>
  			  </fileset>
+ 			  <fileset dir="lib">
+ 				  <include name="*"/>
+ 			  	</fileset>
  			  <pathelement location="${base_dir}/support/ant/lib/junit.jar"/>
  		  </classpath>		  
***************
*** 33,41 ****
      <!-- Copy the configuration files to build/core/... -->
      <mkdir dir="${module_path}/${Module}"/>
! 		<copy toDir="${module_path}/${Module}">
! 			<fileset dir="config">
! 				<include name="*/**"/>
! 			</fileset>
! 		</copy>
  
      <!-- copy over the services file -->
--- 36,45 ----
      <!-- Copy the configuration files to build/core/... -->
      <mkdir dir="${module_path}/${Module}"/>
! 	<copy toDir="${module_path}/${Module}">
! 		<fileset dir="config">
! 			<include name="*/**"/>
! 		</fileset>
! 	</copy>
! 
  
      <!-- copy over the services file -->

[Babeldoc-devel] babeldoc/modules/conversion/lib poi.jar,NONE,1.1

From: <de...@us...> - 2003-07-17 14:49:48

Update of /cvsroot/babeldoc/babeldoc/modules/conversion/lib
In directory sc8-pr-cvs1:/tmp/cvs-serv18130/modules/conversion/lib

Added Files:
	poi.jar 
Log Message:
Excel files to XML conversion pipeline stage

--- NEW FILE: poi.jar ---
(This appears to be a binary file; contents omitted.)

[Babeldoc-devel] babeldoc/modules/conversion/lib - New directory

From: <de...@us...> - 2003-07-17 14:47:39

Update of /cvsroot/babeldoc/babeldoc/modules/conversion/lib
In directory sc8-pr-cvs1:/tmp/cvs-serv18047/modules/conversion/lib

Log Message:
Directory /cvsroot/babeldoc/babeldoc/modules/conversion/lib added to the repository

[Babeldoc-devel] babeldoc/modules/conversion/src/com/babeldoc/conversion/pipeline/stage XlsToXmlPipelineStage.java,NONE,1.1

From: <de...@us...> - 2003-07-17 14:45:29

Update of /cvsroot/babeldoc/babeldoc/modules/conversion/src/com/babeldoc/conversion/pipeline/stage
In directory sc8-pr-cvs1:/tmp/cvs-serv17735/modules/conversion/src/com/babeldoc/conversion/pipeline/stage

Added Files:
	XlsToXmlPipelineStage.java 
Log Message:
Excel files to XML conversion pipeline stage

--- NEW FILE: XlsToXmlPipelineStage.java ---
/*
 * Created on Jul 16, 2003
 *
 * To change this generated comment go to 
 * Window>Preferences>Java>Code Generation>Code and Comments
 */
package com.babeldoc.conversion.pipeline.stage;

import java.io.IOException;
import java.util.Collection;
import org.apache.poi.hssf.usermodel.HSSFCell;
import org.apache.poi.hssf.usermodel.HSSFRow;
import org.apache.poi.hssf.usermodel.HSSFSheet;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
import org.apache.poi.poifs.filesystem.POIFSFileSystem;
import org.dom4j.Document;
import org.dom4j.DocumentHelper;
import org.dom4j.Element;

import com.babeldoc.core.option.IConfigInfo;
import com.babeldoc.core.pipeline.PipelineDocument;
import com.babeldoc.core.pipeline.PipelineException;
import com.babeldoc.core.pipeline.PipelineStage;
import com.babeldoc.core.pipeline.PipelineStageInfo;
import com.babeldoc.core.pipeline.PipelineStageResult;

/**
 * @author dejank
 *
 * To change this generated comment go to 
 * Window>Preferences>Java>Code Generation>Code and Comments
 */
public class XlsToXmlPipelineStage extends PipelineStage {

	public static String DATE_FORMAT = "dd.mm.yyyy";

	public XlsToXmlPipelineStage(IConfigInfo info) {
		super(new PipelineStageInfo() {
			public String getName() {
				return "XlsToXml";
			}

			public String getDescription() {
				return "Converts Microsoft Excel files to XML format.";
			}

			public Collection getTypeSpecificOptions() {

				return null;
			}
		});

	}

	/* (non-Javadoc)
	 * @see com.babeldoc.core.pipeline.PipelineStage#process()
	 */
	public PipelineStageResult[] process() throws PipelineException {

		POIFSFileSystem fs;
		try {
			fs = new POIFSFileSystem(this.getDocument().getInputStream());
			HSSFWorkbook wb = new HSSFWorkbook(fs);

			HSSFSheet sheet = null;
			HSSFRow row = null;
			HSSFCell cell = null;

			Document document = DocumentHelper.createDocument();
			//
			Element root = document.addElement("workbook");

			int numberOfSheets = wb.getNumberOfSheets();
			root.addAttribute(
				"number-of-sheets",
				String.valueOf(numberOfSheets));

			for (int i = 0; i < numberOfSheets; i++) {
				Element sheetElement = root.addElement("sheet");
				sheet = wb.getSheetAt(i);
				String sheetName = wb.getSheetName(i);
				sheetElement.addAttribute("name", sheetName);
				sheetElement.addAttribute(
					"sheet-number",
					(new Integer(i)).toString());
				int rowCount = 0;
				int firstRowNum = sheet.getFirstRowNum();
				int lastRowNum = sheet.getLastRowNum();

				for (int m = firstRowNum; m <= lastRowNum; m++) {
					row = sheet.getRow(m);
					if (row == null)
						continue;
					Element rowElement = document.addElement("row");
					rowElement.addAttribute(
						"rowNum",
						(new Integer(m)).toString());
					rowCount++;
					int firstCellNum = row.getFirstCellNum();
					int lastCellNum = row.getLastCellNum();
					int cellCount = 0;
					for (int p = firstCellNum; p <= lastCellNum; p++) {
						cell = row.getCell((short) p);
						// if (cell == null || cell.getCellType() == HSSFCell.CELL_TYPE_BLANK) continue; 
						if (cell == null)
							continue;
						cellCount++;
						Element cellElement = rowElement.addElement("cell");
						cellElement.addAttribute("colNum", String.valueOf(p));
						cellElement.addAttribute(
							"col",
							String.valueOf(cell.getCellNum()));

						int cellType = cell.getCellType();
						switch (cellType) {
							case HSSFCell.CELL_TYPE_NUMERIC :
								cellElement.addAttribute("type", "Numeric");
								cellElement.addText(
									String.valueOf(cell.getNumericCellValue()));
								break;
							case HSSFCell.CELL_TYPE_STRING :
								cellElement.addAttribute("type", "String");
								cellElement.addText(cell.getStringCellValue());
								break;
							case HSSFCell.CELL_TYPE_BOOLEAN :
								cellElement.addAttribute("type", "Boolean");
								cellElement.addText(
									String.valueOf(cell.getBooleanCellValue()));
								break;
							case HSSFCell.CELL_TYPE_BLANK :
								cellElement.addAttribute("type", "N/A");
								break;
							default :
								break;
						}
					}
					rowElement.addAttribute(
						"number-of-cells",
						String.valueOf(cellCount));
				}
				sheetElement.addAttribute(
					"number-of-rows",
					String.valueOf(rowCount));
			}

			PipelineDocument doc =
				new PipelineDocument(
					this.getDocument(),
					document.toString().getBytes());
			return processHelper(doc);
		} catch (IOException e) {
			// TODO Auto-generated catch block
			throw new PipelineException("Error converting XLS file to XML");
		}

	}

}

[Babeldoc-devel] babeldoc/readme CHANGES,1.21,1.22

From: <tr...@us...> - 2003-07-17 03:18:20

Update of /cvsroot/babeldoc/babeldoc/readme
In directory sc8-pr-cvs1:/tmp/cvs-serv12431

Modified Files:
	CHANGES 
Log Message:
update the change log

Index: CHANGES
===================================================================
RCS file: /cvsroot/babeldoc/babeldoc/readme/CHANGES,v
retrieving revision 1.21
retrieving revision 1.22
diff -C2 -d -r1.21 -r1.22
*** CHANGES	11 Jun 2003 03:45:12 -0000	1.21
--- CHANGES	17 Jul 2003 03:18:18 -0000	1.22
***************
*** 1,2 ****
--- 1,862 ----
+ 2003-07-16 20:22  triphop
+ 
+ 	* modules/core/src/com/babeldoc/core/pipeline/:
+ 	IPipelineStage.java, PipelineStage.java: removed the threaded
+ 	attribute on the pipeline stage
+ 
+ 2003-07-16 18:45  triphop
+ 
+ 	* build.properties, modules/core/config/service/query.properties,
+ 	modules/core/examples/threads/data.txt,
+ 	modules/core/examples/threads/flatfile.xml,
+ 	modules/core/examples/threads/pipeline/config.properties,
+ 	modules/core/examples/threads/pipeline/pipeline.properties,
+ 	modules/core/src/com/babeldoc/core/TieredConfigurationHelper.java,
+ 	modules/core/src/com/babeldoc/core/pipeline/IPipelineStageFactory.java,
+ 	modules/core/src/com/babeldoc/core/pipeline/PipelineStageFactory.java,
+ 	modules/core/src/com/babeldoc/core/pipeline/command/PipelineCommand.java,
+ 	modules/core/src/com/babeldoc/core/pipeline/processor/AsyncPipelineStageProcessor.java,
+ 	modules/core/src/com/babeldoc/core/pipeline/processor/IPipelineStageProcessor.java,
+ 	modules/core/src/com/babeldoc/core/pipeline/processor/PipelineStageProcessorFactory.java,
+ 	modules/core/src/com/babeldoc/core/pipeline/processor/SyncPipelineStageProcessor.java,
+ 	modules/core/src/com/babeldoc/core/pipeline/processor/ThreadPooledPipelineStageProcessor.java,
+ 	modules/core/src/com/babeldoc/core/pipeline/simple/SimplePipelineStageResolver.java,
+ 	modules/web/src/com/babeldoc/web/context/PipelineListingContext.java:
+ 	New pipeline stage factory threading model has been implemented.
+ 
+ 2003-07-16 02:04  triphop
+ 
+ 	* modules/core/src/com/babeldoc/core/pipeline/:
+ 	PipelineFactory.java, PipelineStageFactory.java,
+ 	simple/SimplePipelineStageFactory.java: fixed error in
+ 	SimplePipelineStageFactory where parent method.setOptions was not
+ 	called in overridden method.  This cause options to not ever be set
+ 	on PipelineStageFactory.
+ 
+ 2003-07-16 01:46  triphop
+ 
+ 	* modules/core/src/com/babeldoc/core/:
+ 	TieredConfigurationHelper.java, pipeline/PipelineFactory.java: now
+ 	use the TieredConfigurationHelper in PIpelineFactory.
+ 
+ 2003-07-16 01:17  triphop
+ 
+ 	* modules/core/src/com/babeldoc/core/:
+ 	TieredConfigurationHelper.java, pipeline/IPipelineStage.java,
+ 	pipeline/PipelineStageFactory.java,
+ 	pipeline/feeder/MemoryQueue.java: refactorings and meanderings. 
+ 	Serious looking at the PipelineStageFactory to arrive a simplest
+ 	yet most adequate interface to facilitate parallel execution of
+ 	multiple pipeline stage results.
+ 
+ 2003-07-15 22:33  triphop
+ 
+ 	* modules/core/src/com/babeldoc/core/: Pair.java,
+ 	pipeline/xml/XmlPipelineStageResolver.java,
+ 	pipeline/xml/XmlPipelineUnmarshaller.java: removed old code and
+ 	changed implementation to interface in an interface.
+ 
+ 2003-07-15 22:26  triphop
+ 
+ 	* modules/core/src/com/babeldoc/core/GeneralException.java: Now
+ 	uses the Avalon context object instead of map
+ 
+ 2003-07-15 22:12  triphop
+ 
+ 	*
+ 	modules/sql/src/com/babeldoc/sql/pipeline/stage/SqlEnrichPipelineStage.java:
+ 	removed unnecessary throws clause
+ 
+ 2003-07-15 22:10  triphop
+ 
+ 	*
+ 	modules/sql/src/com/babeldoc/sql/pipeline/stage/SqlQueryPipelineStage.java:
+ 	updated so that the output xml can have a schema applied to it.
+ 
+ 2003-07-15 18:46  triphop
+ 
+ 	* modules/:
+ 	sql/src/com/babeldoc/sql/pipeline/stage/SqlQueryPipelineStage.java,
+ 	web/src/com/babeldoc/web/pipeline/stage/HttpClientPipelineStage.java:
+ 	added the header and some small reformattings.
+ 
+ 2003-07-15 11:07  jonathan_leech
+ 
+ 	* modules/core/src/com/babeldoc/core/pipeline/:
+ 	IPipelineStage.java, PipelineStage.java, PipelineStageFactory.java,
+ 	stage/RouterPipelineStage.java: Threading via the 'threaded' and
+ 	'maxThreads' attributes.  'splitAttributes' copies document
+ 	attributes to split pipeline stages.
+ 
+ 2003-07-14 18:43  jonathan_leech
+ 
+ 	* modules/sql/: config/service/query.properties,
+ 	config/sql/query.properties,
+ 	src/com/babeldoc/sql/journal/GenericSqlJournal.java,
+ 	src/com/babeldoc/sql/pipeline/stage/SqlQueryPipelineStage.java:
+ 	Added SqlQueryPipelineStage, and fixed a couple of bugs in the
+ 	GenericSQL and Oracle journals.
+ 
+ 2003-07-14 16:49  jonathan_leech
+ 
+ 	* modules/web/: config/i18n/messages.properties,
+ 	config/service/query.properties,
+ 	src/com/babeldoc/web/pipeline/stage/HttpClientPipelineStage.java:
+ 	Initial version of HttpClientPipelineStage supporting http get,
+ 	post.
+ 
+ 2003-07-14 13:27  jonathan_leech
+ 
+ 	*
+ 	modules/core/src/com/babeldoc/core/pipeline/stage/XpathExtractPipelineStage.java:
+ 	Allow all types of XPath expressions, e.g. count() etc.
+ 
+ 2003-07-14 13:02  jonathan_leech
+ 
+ 	* modules/sql/src/com/babeldoc/sql/resource/PooledJdbc.java:
+ 	removed synchronization from checkOut() and checkIn() to eliminate
+ 	deadlock when pool is exhausted.
+ 
+ 2003-07-07 22:46  triphop
+ 
+ 	* modules/core/src/com/babeldoc/core/: ResourceLoader.java,
+ 	config/light/LightConfigService.java: Now the getUrl uses the
+ 	modules list to search for names...
+ 
+ 2003-07-02 08:29  dejank
+ 
+ 	*
+ 	modules/core/src/com/babeldoc/core/pipeline/feeder/AsynchronousFeeder.java:
+ 	Fixed bug that causes initialization error when using DiskQueue
+ 
+ 2003-07-01 10:28  dejank
+ 
+ 	*
+ 	modules/gui/src/com/babeldoc/gui/pipeline/command/SwingFeederPanel.java:
+ 	Commented one of catch clasuses to prevent compiler errors. We
+ 	should reconsider this later...
+ 
+ 2003-07-01 08:48  triphop
+ 
+ 	*
+ 	modules/core/src/com/babeldoc/core/TieredConfigurationHelper.java:
+ 	[no log message]
+ 
+ 2003-07-01 08:45  triphop
+ 
+ 	*
+ 	modules/core/src/com/babeldoc/core/pipeline/feeder/FeederFactory.java:
+ 	missing file
+ 
+ 2003-07-01 08:11  dejank
+ 
+ 	* modules/core/lib/: commons-net-1.0.0-dev.jar, commons-net.jar:
+ 	Updated commons-net library to 1.0 final release for branch_1_0
+ 
+ 2003-06-30 18:16  triphop
+ 
+ 	* modules/scanner/: config/feeder/config.properties,
+ 	src/com/babeldoc/scanner/Scanner.java,
+ 	src/com/babeldoc/scanner/ScannerFactory.java: updated the scanner
+ 	code to use the feeder factory code.
+ 
+ 2003-06-30 18:14  triphop
+ 
+ 	* modules/core/: config/feeder/config.properties,
+ 	config/service/query.properties,
+ 	src/com/babeldoc/core/pipeline/command/PipelineFeeder.java,
+ 	src/com/babeldoc/core/pipeline/feeder/AsynchronousFeeder.java,
+ 	src/com/babeldoc/core/pipeline/feeder/IFeeder.java,
+ 	src/com/babeldoc/core/pipeline/feeder/SynchronousFeeder.java: Added
+ 	the feeder factory and the abstraction of access to the feeders.
+ 
+ 2003-06-28 14:13  triphop
+ 
+ 	* readme/CHANGES: updated the changes file.
+ 
+ 2003-06-28 14:02  triphop
+ 
+ 	* build.properties: updated the version number
+ 
+ 2003-06-27 10:00  dejank
+ 
+ 	* modules/scanner/src/com/babeldoc/scanner/: ScannerWorker.java,
+ 	IScannerWorker.java, Scanner.java,
+ 	ScannerConfigurationException.java, ScannerFactoryTest.java,
+ 	ScannerThread.java, IScanDocuments.java, ScanDocument.java,
+ 	ScanDocuments.java, ScannerException.java, ScannerFactory.java,
+ 	ScannerSchedule.java, ScannerWorkerConfig.java,
+ 	ScannerWorkerInfo.java, ScannerWorkerScheduler.java,
+ 	ScannerWorkerStatus.java, ScannerWorkerTest.java,
+ 	ScannerWorkerType.java, ScannerWorkerVO.java,
+ 	gui/ConnectionDlg.java, gui/ScannerMonitor.java,
+ 	gui/ScannerMonitorFrame.java,
+ 	gui/ScannerMonitorFrame_AboutBoxPanel.java,
+ 	gui/ScannerOptionNode.java, gui/ScannerWorkerNode.java,
+ 	worker/DirectoryScanner.java, worker/FtpScanner.java,
+ 	worker/MailboxScanner.java, jmx/ScannerService.java,
+ 	jmx/ScannerServiceMBean.java, rmi/RemoteScannerBridge.java,
+ 	rmi/RemoteScannerClient.java,
+ 	worker/ExternalApplicationScanner.java, worker/HttpScanner.java,
+ 	worker/SimpleScanDocuments.java, worker/SqlScanner.java,
+ 	module/ScannerModule.java, rmi/IRemoteScanner.java: Changing
+ 	licence to Apache and fixed MailboxScanner
+ 
+ 2003-06-27 09:49  dejank
+ 
+ 	*
+ 	modules/core/src/com/babeldoc/core/pipeline/feeder/DiskQueue.java:
+ 	Removed unecessery System.out.println statement.
+ 
+ 2003-06-26 23:36  triphop
+ 
+ 	* modules/core/: config/i18n/messages.properties,
+ 	src/com/babeldoc/core/module/BabeldocModuleCommand.java,
+ 	src/com/babeldoc/core/module/BabeldocModuleList.java: update the
+ 	runtime module handling to use same algorithm as build module
+ 	handler.  Also placed missing i18n messages for ModuleCommand.
+ 
+ 2003-06-26 23:10  triphop
+ 
+ 	* modules/core/config/pipeline/simple/documentation.properties: use
+ 	"real" comment characters.
+ 
+ 2003-06-26 22:43  triphop
+ 
+ 	* quickstart-en.txt, modules/crypto/build.xml,
+ 	modules/init/build.xml, readme/userguide/appendix1.xml: updated the
+ 	license file
+ 
+ 2003-06-26 22:19  triphop
+ 
+ 	* modules/core/: build.xml,
+ 	src/com/babeldoc/core/BabeldocCommand.java,
+ 	src/com/babeldoc/core/EnvironmentLoader.java,
+ 	src/com/babeldoc/core/GeneralException.java,
+ 	src/com/babeldoc/core/I18n.java, src/com/babeldoc/core/INamed.java,
+ 	src/com/babeldoc/core/LogService.java,
+ 	src/com/babeldoc/core/Main.java,
+ 	src/com/babeldoc/core/NameValuePair.java,
+ 	src/com/babeldoc/core/Named.java, src/com/babeldoc/core/Pair.java,
+ 	src/com/babeldoc/core/ResourceLoader.java,
+ 	src/com/babeldoc/core/RmiRegistry.java,
+ 	src/com/babeldoc/core/Type.java,
+ 	src/com/babeldoc/core/VariableProcessor.java,
+ 	src/com/babeldoc/core/VelocityUtilityContext.java,
+ 	src/com/babeldoc/core/config/ConfigService.java,
+ 	src/com/babeldoc/core/config/IConfig.java,
+ 	src/com/babeldoc/core/config/IConfigService.java,
+ 	src/com/babeldoc/core/config/light/LightConfig.java,
+ 	src/com/babeldoc/core/config/light/LightConfigService.java,
+ 	src/com/babeldoc/core/journal/IJournal.java,
+ 	src/com/babeldoc/core/journal/IJournalStatus.java,
+ 	src/com/babeldoc/core/journal/IJournalTicket.java,
+ 	src/com/babeldoc/core/journal/Journal.java,
+ 	src/com/babeldoc/core/journal/JournalException.java,
+ 	src/com/babeldoc/core/journal/JournalFactory.java,
+ 	src/com/babeldoc/core/journal/JournalOperation.java,
+ 	src/com/babeldoc/core/journal/JournalStatus.java,
+ 	src/com/babeldoc/core/journal/JournalTest.java,
+ 	src/com/babeldoc/core/journal/JournalTicket.java,
+ 	src/com/babeldoc/core/journal/JournalType.java,
+ 	src/com/babeldoc/core/journal/command/JournalCommand.java,
+ 	src/com/babeldoc/core/journal/query/JournalQuery.java,
+ 	src/com/babeldoc/core/journal/query/JournalQueryException.java,
+ 	src/com/babeldoc/core/journal/query/JournalQueryTest.java,
+ 	src/com/babeldoc/core/journal/query/QueryOption.java,
+ 	src/com/babeldoc/core/journal/query/QueryTicket.java,
+ 	src/com/babeldoc/core/journal/simple/SimpleJournal.java,
+ 	src/com/babeldoc/core/module/BabeldocModule.java,
+ 	src/com/babeldoc/core/module/BabeldocModuleCommand.java,
+ 	src/com/babeldoc/core/module/BabeldocModuleList.java,
+ 	src/com/babeldoc/core/module/CoreModule.java,
+ 	src/com/babeldoc/core/option/BooleanConfigOptionType.java,
+ 	src/com/babeldoc/core/option/ComplexConfigOptionType.java,
+ 	src/com/babeldoc/core/option/ConfigInfo.java,
+ 	src/com/babeldoc/core/option/ConfigOption.java,
+ 	src/com/babeldoc/core/option/DirectoryConfigOptionType.java,
+ 	src/com/babeldoc/core/option/FilenameConfigOptionType.java,
+ 	src/com/babeldoc/core/option/IConfigInfo.java,
+ 	src/com/babeldoc/core/option/IConfigOptionType.java,
+ 	src/com/babeldoc/core/option/IConfigurable.java,
+ 	src/com/babeldoc/core/option/IntegerConfigOptionType.java,
+ 	src/com/babeldoc/core/option/MultiLineConfigOptionType.java,
+ 	src/com/babeldoc/core/option/ServiceTypeConfigOptionType.java,
+ 	src/com/babeldoc/core/option/StringConfigOptionType.java,
+ 	src/com/babeldoc/core/option/UrlConfigOptionType.java,
+ 	src/com/babeldoc/core/option/ValueListConfigOptionType.java,
+ 	src/com/babeldoc/core/option/XpathConfigOptionType.java,
+ 	src/com/babeldoc/core/pipeline/DefaultPipelineStageErrorHandler.java,
+ 	src/com/babeldoc/core/pipeline/IPipelineFactory.java,
+ 	src/com/babeldoc/core/pipeline/IPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/IPipelineStageErrorHandler.java,
+ 	src/com/babeldoc/core/pipeline/IPipelineStageFactory.java,
+ 	src/com/babeldoc/core/pipeline/IPipelineStageResolver.java,
+ 	src/com/babeldoc/core/pipeline/PipelineConfigException.java,
+ 	src/com/babeldoc/core/pipeline/PipelineDocument.java,
+ 	src/com/babeldoc/core/pipeline/PipelineDocumentTest.java,
+ 	src/com/babeldoc/core/pipeline/PipelineException.java,
+ 	src/com/babeldoc/core/pipeline/PipelineFactory.java,
+ 	src/com/babeldoc/core/pipeline/PipelineFactoryFactory.java,
+ 	src/com/babeldoc/core/pipeline/PipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/PipelineStageConfig.java,
+ 	src/com/babeldoc/core/pipeline/PipelineStageConnection.java,
+ 	src/com/babeldoc/core/pipeline/PipelineStageFactory.java,
+ 	src/com/babeldoc/core/pipeline/PipelineStageFactoryType.java,
+ 	src/com/babeldoc/core/pipeline/PipelineStageInfo.java,
+ 	src/com/babeldoc/core/pipeline/PipelineStageResolver.java,
+ 	src/com/babeldoc/core/pipeline/PipelineStageResult.java,
+ 	src/com/babeldoc/core/pipeline/PipelineStageType.java,
+ 	src/com/babeldoc/core/pipeline/command/PipelineCommand.java,
+ 	src/com/babeldoc/core/pipeline/command/PipelineCommandTest.java,
+ 	src/com/babeldoc/core/pipeline/command/PipelineFeeder.java,
+ 	src/com/babeldoc/core/pipeline/compiler/PipelineClassLoader.java,
+ 	src/com/babeldoc/core/pipeline/compiler/PipelineStageCompiler.java,
+ 	src/com/babeldoc/core/pipeline/error/RedirectErrorHandler.java,
+ 	src/com/babeldoc/core/pipeline/feeder/AsynchronousFeeder.java,
+ 	src/com/babeldoc/core/pipeline/feeder/DiskQueue.java,
+ 	src/com/babeldoc/core/pipeline/feeder/DiskQueueTest.java,
+ 	src/com/babeldoc/core/pipeline/feeder/FeedDocument.java,
+ 	src/com/babeldoc/core/pipeline/feeder/IFeeder.java,
+ 	src/com/babeldoc/core/pipeline/feeder/IFeederQueue.java,
+ 	src/com/babeldoc/core/pipeline/feeder/MemoryQueue.java,
+ 	src/com/babeldoc/core/pipeline/feeder/SynchronousFeeder.java,
+ 	src/com/babeldoc/core/pipeline/simple/SimplePipelineStageFactory.java,
+ 	src/com/babeldoc/core/pipeline/simple/SimplePipelineStageResolver.java,
+ 	src/com/babeldoc/core/pipeline/stage/CallStagePipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/CompressionPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/DecompressionPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/DomifyPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/EnrichDocumentPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/ExternalApplicationPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/FileWriterPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/FtpWriterPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/GenericWriterPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/JTidyPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/JavaXmlDecoderPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/NullPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/RSSChannelPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/RouterPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/ScriptingPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/SmtpWriterPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/SocketWriterPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/VelocityTemplatizePipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/XpathExtractPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/XpathSplitterPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/XslTransformPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/stage/ZipArchiveWriterPipelineStage.java,
+ 	src/com/babeldoc/core/pipeline/xml/IUnmarshaller.java,
+ 	src/com/babeldoc/core/pipeline/xml/XmlPipelineStageFactory.java,
+ 	src/com/babeldoc/core/pipeline/xml/XmlPipelineStageResolver.java,
+ 	src/com/babeldoc/core/pipeline/xml/XmlPipelineUnmarshaller.java,
+ 	src/com/babeldoc/core/pipeline/xml/digester/Connection.java,
+ 	src/com/babeldoc/core/pipeline/xml/digester/DigesterPipelineUnmarshaller.java,
+ 	src/com/babeldoc/core/pipeline/xml/digester/Dynamic.java,
+ 	src/com/babeldoc/core/pipeline/xml/digester/Option.java,
+ 	src/com/babeldoc/core/pipeline/xml/digester/Pipeline.java,
+ 	src/com/babeldoc/core/pipeline/xml/digester/StageInst.java,
+ 	src/com/babeldoc/core/resource/IResource.java,
+ 	src/com/babeldoc/core/resource/Resource.java,
+ 	src/com/babeldoc/core/resource/ResourceException.java,
+ 	src/com/babeldoc/core/resource/ResourceFactory.java,
+ 	src/com/babeldoc/core/resource/ResourceType.java,
+ 	src/com/babeldoc/core/service/ServiceException.java,
+ 	src/com/babeldoc/core/service/ServiceFactory.java,
+ 	src/com/babeldoc/core/user/IOrganization.java,
+ 	src/com/babeldoc/core/user/IUser.java,
+ 	src/com/babeldoc/core/user/IUserContext.java,
+ 	src/com/babeldoc/core/user/IUserCredential.java,
+ 	src/com/babeldoc/core/user/IUserFactory.java,
+ 	src/com/babeldoc/core/user/IUserOperation.java,
+ 	src/com/babeldoc/core/user/IUserResourceFactory.java,
+ 	src/com/babeldoc/core/user/PasswordUserCredential.java,
+ 	src/com/babeldoc/core/user/User.java,
+ 	src/com/babeldoc/core/user/UserContext.java,
+ 	src/com/babeldoc/core/user/UserCreateException.java,
+ 	src/com/babeldoc/core/user/UserCredentialException.java,
+ 	src/com/babeldoc/core/user/UserException.java,
+ 	src/com/babeldoc/core/user/UserFactory.java,
+ 	src/com/babeldoc/core/user/UserFactoryType.java,
+ 	src/com/babeldoc/core/user/UserResource.java,
+ 	src/com/babeldoc/core/user/UserResourceException.java,
+ 	src/com/babeldoc/core/user/UserResourceFactory.java,
+ 	src/com/babeldoc/core/user/UserResourceFactoryType.java,
+ 	src/com/babeldoc/core/user/UserType.java,
+ 	src/com/babeldoc/core/user/simple/SimpleUserContext.java,
+ 	src/com/babeldoc/core/user/simple/SimpleUserFactory.java,
+ 	src/com/babeldoc/core/user/simple/SimpleUserResourceFactory.java:
+ 	reformatted and updated license header
+ 
+ 2003-06-26 22:12  triphop
+ 
+ 	* modules/gui/build.xml: moved the echo into the commented area
+ 
+ 2003-06-26 22:09  triphop
+ 
+ 	* modules/web/: build.xml,
+ 	src/com/babeldoc/web/context/BabelContext.java,
+ 	src/com/babeldoc/web/context/JournalListingContext.java,
+ 	src/com/babeldoc/web/context/ModuleListingContext.java,
+ 	src/com/babeldoc/web/context/PipelineListingContext.java,
+ 	src/com/babeldoc/web/main/ConsoleServlet.java,
+ 	src/com/babeldoc/web/module/WebModule.java,
+ 	src/com/babeldoc/web/pipeline/servlet/HttpPostPipelineFeeder.java:
+ 	Reformatted and added new license header
+ 
+ 2003-06-26 22:05  triphop
+ 
+ 	* modules/sql/: build.xml,
+ 	src/com/babeldoc/sql/config/SqlConfig.java,
+ 	src/com/babeldoc/sql/config/SqlConfigService.java,
+ 	src/com/babeldoc/sql/config/SqlConfigurationLoader.java,
+ 	src/com/babeldoc/sql/journal/GenericSqlJournal.java,
+ 	src/com/babeldoc/sql/journal/MySqlJournal.java,
+ 	src/com/babeldoc/sql/journal/OracleJournal.java,
+ 	src/com/babeldoc/sql/module/SqlModule.java,
+ 	src/com/babeldoc/sql/pipeline/stage/SqlEnrichPipelineStage.java,
+ 	src/com/babeldoc/sql/pipeline/stage/SqlWriterPipelineStage.java,
+ 	src/com/babeldoc/sql/resource/PooledJdbc.java,
+ 	src/com/babeldoc/sql/resource/SimpleJdbc.java,
+ 	src/com/babeldoc/sql/user/SqlOrganization.java,
+ 	src/com/babeldoc/sql/user/SqlUser.java,
+ 	src/com/babeldoc/sql/user/SqlUserContext.java,
+ 	src/com/babeldoc/sql/user/SqlUserFactory.java,
+ 	src/com/babeldoc/sql/user/SqlUserFactoryTest.java,
+ 	src/com/babeldoc/sql/user/SqlUserResourceFactory.java,
+ 	src/com/babeldoc/sql/user/SqlUserResourceFactoryTest.java,
+ 	src/com/babeldoc/sql/util/SqlQueryManager.java,
+ 	src/com/babeldoc/sql/util/TableKeyHelper.java,
+ 	src/com/babeldoc/sql/util/VelocitySqlContext.java: Reformatted and
+ 	updated license header
+ 
+ 2003-06-26 21:50  triphop
+ 
+ 	* modules/crypto/: build.xml,
+ 	src/com/babeldoc/crypto/module/CryptoModule.java,
+ 	src/com/babeldoc/crypto/pipeline/stage/CipherPipelineStage.java,
+ 	src/com/babeldoc/crypto/pipeline/stage/CryptoPipelineStage.java,
+ 	src/com/babeldoc/crypto/pipeline/stage/SignerPipelineStage.java,
+ 	src/com/babeldoc/crypto/pipeline/stage/XmlSignerPipelineStage.java:
+ 	reformatted and changed header
+ 
+ 2003-06-26 21:44  triphop
+ 
+ 	* modules/conversion/: build.xml,
+ 	src/com/babeldoc/conversion/ConversionClient.java,
+ 	src/com/babeldoc/conversion/ConversionException.java,
+ 	src/com/babeldoc/conversion/ConversionUnmarshaller.java,
+ 	src/com/babeldoc/conversion/FieldData.java,
+ 	src/com/babeldoc/conversion/LineSegmentData.java,
+ 	src/com/babeldoc/conversion/digester/Conversion.java,
+ 	src/com/babeldoc/conversion/digester/DigesterConversionUnmarshaller.java,
+ 	src/com/babeldoc/conversion/digester/Field.java,
+ 	src/com/babeldoc/conversion/digester/Header.java,
+ 	src/com/babeldoc/conversion/digester/InputDocument.java,
+ 	src/com/babeldoc/conversion/digester/LineSegment.java,
+ 	src/com/babeldoc/conversion/digester/OutputDocument.java,
+ 	src/com/babeldoc/conversion/module/ConversionModule.java,
+ 	src/com/babeldoc/conversion/pipeline/stage/FlatToXmlPipelineStage.java:
+ 	Reformatted and changed the license header - also added javadoc
+ 	comments.
+ 
+ 2003-06-26 20:47  triphop
+ 
+ 	* modules/babelfish/: build.xml,
+ 	src/com/babeldoc/babelfish/BabelfishTranslator.java,
+ 	src/com/babeldoc/babelfish/module/BabelfishModule.java: Updated the
+ 	formatting and license header in the babelfish module
+ 
+ 2003-06-26 20:35  triphop
+ 
+ 	* modules/soap/: build.xml,
+ 	src/com/babeldoc/soap/module/SoapModule.java,
+ 	src/com/babeldoc/soap/pipeline/command/SoapPipelineFeeder.java,
+ 	src/com/babeldoc/soap/pipeline/servlet/SoapPipelineFeeder.java,
+ 	src/com/babeldoc/soap/pipeline/stage/SoapWriterPipelineStage.java,
+ 	src/com/babeldoc/soap/pipeline/util/SoapHelper.java: Formatted the
+ 	soap module
+ 
+ 2003-06-26 20:31  triphop
+ 
+ 	* modules/xslfo/: build.xml,
+ 	src/com/babeldoc/xslfo/module/XslFoModule.java,
+ 	src/com/babeldoc/xslfo/pipeline/stage/SvgTranscodePipelineStage.java,
+ 	src/com/babeldoc/xslfo/pipeline/stage/XslFoTransformPipelineStage.java:
+ 	Formatted the XSLFO module.
+ 
+ 2003-06-26 20:14  triphop
+ 
+ 	* build.xml,
+ 	modules/gui/src/com/babeldoc/gui/config/BooleanConfigOptionComponent.java,
+ 	modules/gui/src/com/babeldoc/gui/config/ConfigOptionActionHandler.java,
+ 	modules/gui/src/com/babeldoc/gui/config/ConfigOptionActionListener.java,
+ 	modules/gui/src/com/babeldoc/gui/config/ConfigOptionComponent.java,
+ 	modules/gui/src/com/babeldoc/gui/config/ConfigOptionComponentFactory.java,
+ 	modules/gui/src/com/babeldoc/gui/config/ConfigOptionFocusListener.java,
+ 	modules/gui/src/com/babeldoc/gui/config/ConfigOptionListenerBase.java,
+ 	modules/gui/src/com/babeldoc/gui/config/ConfigOptionPanel.java,
+ 	modules/gui/src/com/babeldoc/gui/config/DirectoryConfigOptionComponent.java,
+ 	modules/gui/src/com/babeldoc/gui/config/FilenameConfigOptionComponent.java,
+ 	modules/gui/src/com/babeldoc/gui/config/MultiLineConfigOptionComponent.java,
+ 	modules/gui/src/com/babeldoc/gui/config/StringConfigOptionComponent.java,
+ 	modules/gui/src/com/babeldoc/gui/config/ValueListConfigOptionComponent.java,
+ 	modules/init/build.xml,
+ 	modules/soap/src/com/babeldoc/soap/module/SoapModule.java,
+ 	modules/soap/src/com/babeldoc/soap/pipeline/command/SoapPipelineFeeder.java,
+ 	modules/soap/src/com/babeldoc/soap/pipeline/servlet/SoapPipelineFeeder.java,
+ 	modules/soap/src/com/babeldoc/soap/pipeline/stage/SoapWriterPipelineStage.java,
+ 	modules/soap/src/com/babeldoc/soap/pipeline/util/SoapHelper.java,
+ 	support/ant/lib/babeldoc_bootstrap.jar,
+ 	support/src/com/babeldoc/bootstrap/ant/ModuleFinder.java: Updated
+ 	the build system using an explicit modular ant task.  This is
+ 	better than the method previously employed which added "hidden"
+ 	tasks to the build and clean etc targets.  Additionally using a
+ 	more robust module sorting algorithm.
+ 
+ 2003-06-25 17:11  vharcq
+ 
+ 	* modules/web/docs/journal/all-tickets.vm: file all-tickets.vm was
+ 	initially added on branch branch_1_0.
+ 
+ 2003-06-25 17:11  vharcq
+ 
+ 	* modules/web/docs/journal/failed-status.vm: file failed-status.vm
+ 	was initially added on branch branch_1_0.
+ 
+ 2003-06-25 17:11  vharcq
+ 
+ 	* modules/web/docs/journal/: failed-status.vm, ticketstep.vm,
+ 	all-tickets.vm, index.vm, view-document.vm: - Add a way to query
+ 	journal on ticket's STATUS - The view of a Ticket shows a bit more
+ 	information
+ 
+ 2003-06-25 17:05  vharcq
+ 
+ 	* modules/:
+ 	core/src/com/babeldoc/core/journal/query/JournalQuery.java,
+ 	core/src/com/babeldoc/core/journal/query/QueryOption.java,
+ 	core/src/com/babeldoc/core/journal/simple/SimpleJournal.java,
+ 	web/src/com/babeldoc/web/context/JournalListingContext.java: Add a
+ 	way to query journal on ticket's STATUS
+ 
+ 2003-06-25 17:00  vharcq
+ 
+ 	*
+ 	modules/core/src/com/babeldoc/core/pipeline/stage/FtpWriterPipelineStage.java:
+ 	- Remove an uneeded "static" on a method - print a message with
+ 	this classname and not another one
+ 
+ 2003-06-25 16:58  vharcq
+ 
+ 	*
+ 	modules/core/src/com/babeldoc/core/pipeline/stage/SmtpWriterPipelineStage.java:
+ 	Avoid to create an empty body when ther is no "smtpMessage" defined
+ 
+ 2003-06-25 16:56  vharcq
+ 
+ 	*
+ 	modules/core/src/com/babeldoc/core/pipeline/DefaultPipelineStageErrorHandler.java:
+ 	Save the complete StackTrace  in addMessage() for better analyse of
+ 	errors afterwards
+ 
+ 2003-06-25 16:51  vharcq
+ 
+ 	* modules/core/src/com/babeldoc/core/ResourceLoader.java: Add the
+ 	"J2EE way" of ClassLoading in getClass
+ 
+ 2003-06-25 16:46  vharcq
+ 
+ 	* modules/core/config/service/query.properties: I can not
+ 	understand how the web console could work without this line
+ 
+ 2003-06-25 16:35  vharcq
+ 
+ 	*
+ 	modules/scanner/src/com/babeldoc/scanner/worker/DirectoryScanner.java:
+ 	Change a method to protected to allow subclassing
+ 
+ 2003-06-25 10:00  dejank
+ 
+ 	* modules/core/config/service/query.properties: Added default error
+ 	handler entry in service/config file. Note that this information is
+ 	not used yet...
+ 
+ 2003-06-23 23:27  triphop
+ 
+ 	* support/: ant/lib/babeldoc_bootstrap.jar,
+ 	src/com/babeldoc/bootstrap/ant/DependencyBuilder.java,
+ 	src/com/babeldoc/bootstrap/ant/ModuleFinder.java,
+ 	src/com/babeldoc/bootstrap/ant/ModuleRunnerTask.java,
+ 	src/com/babeldoc/bootstrap/ant/PathTrimmer.java: changed the
+ 	recursive build arrangement to use an explicit target that calls
+ 	each of the modules in turn.
+ 
+ 2003-06-23 23:26  triphop
+ 
+ 	*
+ 	modules/core/src/com/babeldoc/core/pipeline/feeder/DiskQueueTest.java:
+ 	bad import - nothing in core should include anything from other
+ 	modules.
+ 
+ 2003-06-23 07:37  dejank
+ 
+ 	* modules/scanner/src/com/babeldoc/scanner/: Scanner.java,
+ 	ScannerFactory.java, ScannerWorker.java, worker/HttpScanner.java,
+ 	worker/MailboxScanner.java: Changing initialization order in
+ 	scanner module Changing worker classes to conform new scanner
+ 	module architecture
+ 
+ 2003-06-23 07:29  dejank
+ 
+ 	*
+ 	modules/core/src/com/babeldoc/core/pipeline/feeder/DiskQueue.java:
+ 	Loading queue at startup
+ 
+ 2003-06-18 10:08  dejank
+ 
+ 	* modules/scanner/src/com/babeldoc/scanner/: Scanner.java,
+ 	ScannerFactory.java, ScannerWorker.java,
+ 	ScannerWorkerScheduler.java: - Changed queue type from MemoryQueue
+ 	to DiskQueue for testing purpose. This should be configurable,
+ 	anyway - Changed initialization behaviour. Just refactoring...	-
+ 	Implementing stop method in Scanner
+ 
+ 2003-06-18 08:32  dejank
+ 
+ 	* modules/core/src/com/babeldoc/core/BabeldocCommand.java:
+ 	Formatting..
+ 
+ 2003-06-18 08:15  dejank
+ 
+ 	* modules/scanner/src/com/babeldoc/scanner/worker/FtpScanner.java:
+ 	Removing inner class from ftpScanner.
+ 
+ 2003-06-17 11:21  dejank
+ 
+ 	*
+ 	modules/core/src/com/babeldoc/core/pipeline/feeder/AsynchronousFeeder.java:
+ 	Changing logging level
+ 
+ 2003-06-17 11:19  dejank
+ 
+ 	* modules/scanner/src/com/babeldoc/scanner/: ScannerFactory.java,
+ 	ScannerWorker.java, worker/DirectoryScanner.java,
+ 	worker/FtpScanner.java: Fixing few bugs...
+ 
+ 2003-06-16 20:13  kgeis
+ 
+ 	* modules/core/src/com/babeldoc/core/VariableProcessor.java: Take
+ 	the synchronized off getContext(), add synchronized to
+ 	addToContext(name, object).
+ 
+ 2003-06-16 15:24  dejank
+ 
+ 	* modules/scanner/src/com/babeldoc/scanner/ScannerWorker.java:
+ 	Implementing new method for feeding document
+ 
+ 2003-06-16 15:07  dejank
+ 
+ 	* modules/scanner/src/com/babeldoc/scanner/ScannerWorker.java:
+ 	Implementing new method for feeding document
+ 
+ 2003-06-16 15:05  dejank
+ 
+ 	* modules/scanner/src/com/babeldoc/scanner/worker/:
+ 	DirectoryScanner.java, FtpScanner.java: ...
+ 
+ 2003-06-16 10:46  dejank
+ 
+ 	* modules/scanner/src/com/babeldoc/scanner/: ScanDocument.java,
+ 	ScannerWorker.java: Not finished yet!
+ 
+ 2003-06-16 02:53  dejank
+ 
+ 	*
+ 	modules/core/src/com/babeldoc/core/pipeline/PipelineDocument.java:
+ 	Adding getAttributes method. This can probably be avoided if not
+ 	using ScanDocument object...
+ 
+ 2003-06-15 17:18  dejank
+ 
+ 	* modules/: scanner/src/com/babeldoc/scanner/ScanDocument.java,
+ 	scanner/src/com/babeldoc/scanner/Scanner.java,
+ 	scanner/src/com/babeldoc/scanner/ScannerWorker.java,
+ 	core/src/com/babeldoc/core/pipeline/feeder/AsynchronousFeeder.java:
+ 	Implementing asynchronous feeder. Just commiting changes I have
+ 	made at home so I can continue at work...
+ 
+ 2003-06-15 10:35  dejank
+ 
+ 	* modules/core/src/com/babeldoc/core/pipeline/feeder/:
+ 	AsynchronousFeeder.java, DiskQueue.java, DiskQueueTest.java,
+ 	FeedDocument.java, IFeederQueue.java, MemoryQueue.java,
+ 	SynchronousFeeder.java: Few changes..
+ 
+ 2003-06-15 10:33  dejank
+ 
+ 	* modules/scanner/src/com/babeldoc/scanner/: Scanner.java,
+ 	ScannerFactory.java, ScannerFactoryTest.java, ScannerWorker.java,
+ 	ScannerWorkerFactory.java, ScannerWorkerFactoryTest.java,
+ 	ScannerWorkerTest.java, worker/MailboxScanner.java: Renamed from
+ 	ScannerWorkerFactory to ScannerFactory since it does not handle
+ 	only workers
+ 
+ 2003-06-12 08:16  dejank
+ 
+ 	* modules/scanner/src/com/babeldoc/scanner/:
+ 	ScannerWorkerScheduler.java, Scanner.java,
+ 	worker/MailboxScanner.java: Fixing some bugs. Scanner should scan
+ 	now. I guess there are still many things left for implementation
+ 
+ 2003-06-12 01:00  triphop
+ 
+ 	* modules/gui/src/com/babeldoc/gui/config/ConfigOptionPanel.java:
+ 	First attempt at getting values
+ 
+ 2003-06-12 00:51  triphop
+ 
+ 	* modules/gui/: build.xml,
+ 	src/com/babeldoc/gui/config/ComplexConfigOptionComponent.java,
+ 	src/com/babeldoc/gui/config/ConfigOptionModel.java,
+ 	src/com/babeldoc/gui/config/ConfigOptionPanel.java,
+ 	src/com/babeldoc/gui/treetable/AbstractCellEditor.java,
+ 	src/com/babeldoc/gui/treetable/AbstractTreeTableModel.java,
+ 	src/com/babeldoc/gui/treetable/JTreeTable.java,
+ 	src/com/babeldoc/gui/treetable/TreeTableModel.java,
+ 	src/com/babeldoc/gui/treetable/TreeTableModelAdapter.java: Lots of
+ 	experimentation!
+ 
+ 2003-06-11 22:47  triphop
+ 
+ 	* modules/core/src/com/babeldoc/core/option/ConfigOption.java:
+ 	Changed the interface from the getEntry set.
+ 
+ 2003-06-11 22:43  triphop
+ 
+ 	*
+ 	modules/crypto/src/com/babeldoc/crypto/pipeline/stage/CipherPipelineStage.java:
+ 	Changed the importation of the pipeline.
+ 
+ 2003-06-11 19:37  triphop
+ 
+ 	* support/jalopy/babeldocconvention.xml: slightly updated
+ 	convention.
+ 
+ 2003-06-11 19:34  triphop
+ 
+ 	* modules/: init/src/com/babeldoc/init/BabeldocClassloader.java,
+ 	init/src/com/babeldoc/init/Main.java, gui/build.xml,
+ 	gui/src/com/babeldoc/gui/config/BooleanConfigOptionComponent.java,
+ 	gui/src/com/babeldoc/gui/config/ComplexConfigOptionComponent.java,
+ 	gui/src/com/babeldoc/gui/config/ConfigOptionActionHandler.java,
+ 	gui/src/com/babeldoc/gui/config/ConfigOptionComponent.java,
+ 	gui/src/com/babeldoc/gui/config/ConfigOptionComponentFactory.java,
+ 	gui/src/com/babeldoc/gui/config/ConfigOptionPanel.java,
+ 	gui/src/com/babeldoc/gui/config/DirectoryConfigOptionComponent.java,
+ 	gui/src/com/babeldoc/gui/config/FilenameConfigOptionComponent.java,
+ 	gui/src/com/babeldoc/gui/config/MultiLineConfigOptionComponent.java,
+ 	gui/src/com/babeldoc/gui/config/StringConfigOptionComponent.java,
+ 	gui/src/com/babeldoc/gui/config/ValueListConfigOptionComponent.java,
+ 	gui/src/com/babeldoc/gui/journal/browser/Command.java,
+ 	gui/src/com/babeldoc/gui/journal/browser/JournalBrowser.java,
+ 	gui/src/com/babeldoc/gui/journal/browser/JournalBrowserAction.java,
+ 	gui/src/com/babeldoc/gui/journal/browser/JournalBrowserFrame.java,
+ 	gui/src/com/babeldoc/gui/journal/browser/JournalBrowserModel.java,
+ 	gui/src/com/babeldoc/gui/journal/browser/JournalTree.java,
+ 	gui/src/com/babeldoc/gui/module/GuiModule.java,
+ 	gui/src/com/babeldoc/gui/pipeline/builder/AddPipelineAction.java,
+ 	gui/src/com/babeldoc/gui/pipeline/builder/AddPipelineStageAction.java,
+ 	gui/src/com/babeldoc/gui/pipeline/builder/AddPipelineStageConfigAction.java,
+ 	gui/src/com/babeldoc/gui/pipeline/builder/BuilderAction.java,
+ 	gui/src/com/babeldoc/gui/pipeline/builder/Command.java,
+ 	gui/src/com/babeldoc/gui/pipeline/builder/PipelineBuilder.java,
+ 	gui/src/com/babeldoc/gui/pipeline/builder/PipelineBuilderController.java,
+ 	gui/src/com/babeldoc/gui/pipeline/builder/PipelineBuilderFrame.java,
+ 	gui/src/com/babeldoc/gui/pipeline/builder/PipelineBuilderModel.java,
+ 	gui/src/com/babeldoc/gui/pipeline/builder/PipelineTree.java,
+ 	gui/src/com/babeldoc/gui/pipeline/builder/RemovePipelineAction.java,
+ 	gui/src/com/babeldoc/gui/pipeline/builder/RemovePipelineStageAction.java,
+ 	gui/src/com/babeldoc/gui/pipeline/builder/SetEntryStageAction.java,
+ 	gui/src/com/babeldoc/gui/pipeline/command/SwingFeeder.java,
+ 	gui/src/com/babeldoc/gui/pipeline/command/SwingFeederPanel.java,
+ 	gui/src/com/babeldoc/gui/util/ConsoleOutStream.java,
+ 	gui/src/com/babeldoc/gui/wizard/IWizardController.java,
+ 	gui/src/com/babeldoc/gui/wizard/SimpleWizardController.java,
+ 	gui/src/com/babeldoc/gui/wizard/StandardWizardPanel.java,
+ 	gui/src/com/babeldoc/gui/wizard/WizardButtonPanel.java,
+ 	gui/src/com/babeldoc/gui/wizard/WizardDialog.java,
+ 	gui/src/com/babeldoc/gui/wizard/WizardModel.java,
+ 	gui/src/com/babeldoc/gui/wizard/WizardPanel.java,
+ 	gui/src/com/babeldoc/gui/wizard/addstage/AddPipelineStageController.java,
+ 	gui/src/com/babeldoc/gui/wizard/addstage/AddPipelineStageModel.java,
+ 	gui/src/com/babeldoc/gui/wizard/addstage/AddPipelineStageWizard.java,
+ 	gui/src/com/babeldoc/gui/wizard/addstage/AddPipelineStageWizardPanel.java,
+ 	gui/src/com/babeldoc/gui/wizard/addstage/Command.java,
+ 	gui/src/com/babeldoc/gui/wizard/addstage/GetPipelineNameWizardPanel.java,
+ 	gui/src/com/babeldoc/gui/wizard/addstage/GetStageConfigWizardPanel.java,
+ 	gui/src/com/babeldoc/gui/wizard/addstage/GetStageDetailsWizardPanel.java,
+ 	gui/src/com/babeldoc/gui/wizard/setentrystage/Command.java,
+ 	gui/src/com/babeldoc/gui/wizard/setentrystage/GetPipelineNameWizardPanel.java,
+ 	gui/src/com/babeldoc/gui/wizard/setentrystage/SetEntryPipelineStageWizardPanel.java,
+ 	gui/src/com/babeldoc/gui/wizard/setentrystage/SetEntryStageController.java,
+ 	gui/src/com/babeldoc/gui/wizard/setentrystage/SetEntryStageModel.java,
+ 	gui/src/com/babeldoc/gui/wizard/setentrystage/SetEntryStageWizard.java,
+ 	gui/src/com/babeldoc/gui/wizard/setentrystage/SetEntryStageWizardPanel.java,
+ 	gui/src/com/babeldoc/gui/wizard/setup/Command.java,
+ 	gui/src/com/babeldoc/gui/wizard/setup/FinalizeWizardPanel.java,
+ 	gui/src/com/babeldoc/gui/wizard/setup/IntroWizardPanel.java,
+ 	gui/src/com/babeldoc/gui/wizard/setup/ProjectSetupController.java,
+ 	gui/src/com/babeldoc/gui/wizard/setup/ProjectSetupModel.java,
+ 	gui/src/com/babeldoc/gui/wizard/setup/ProjectSetupWizard.java,
+ 	gui/src/com/babeldoc/gui/wizard/setup/ProjectSetupWizardPanel.java,
+ 	gui/src/com/babeldoc/gui/wizard/setup/RootDirectoryWizardPanel.java,
+ 	gui/src/com/babeldoc/gui/wizard/setup/SelectComponentsWizardPanel.java:
+ 	Reformatted the GUI and init classes
+ 
+ 2003-06-11 18:24  triphop
+ 
+ 	* support/jalopy/babeldocconvention.xml: Added the license back in.
+ 
+ 2003-06-11 10:29  dejank
+ 
+ 	* modules/init/bin/babeldoc.bat: Bruce's changes...
+ 
+ 2003-06-11 07:27  hbenedict
+ 
+ 	* support/jalopy/babeldocconvention.xml: remove "Apache Software
+ 	License" header before adding it (again and again and again...)
+ 
+ 2003-06-11 07:02  hbenedict
+ 
+ 	* modules/: babelfish/build.xml, conversion/build.xml,
+ 	core/build.xml, crypto/build.xml, gui/build.xml, scanner/build.xml,
+ 	sql/build.xml, web/build.xml, xslfo/build.xml: one &gt; to much in
+ 	jalopy tag
+ 
+ 2003-06-11 05:01  dejank
+ 
+ 	* modules/scanner/: src/com/babeldoc/scanner/ScannerWorker.java,
+ 	src/com/babeldoc/scanner/ScannerWorkerStatus.java,
+ 	src/com/babeldoc/scanner/Scanner.java,
+ 	src/com/babeldoc/scanner/ScannerThread.java,
+ 	src/com/babeldoc/scanner/ScannerWorkerScheduler.java, build.xml:
+ 	[no log message]
+ 
+ 2003-06-11 04:23  dejank
+ 
+ 	* modules/scanner/src/com/babeldoc/scanner/:
+ 	worker/DirectoryScanner.java, worker/FtpScanner.java,
+ 	gui/ScannerOptionNode.java: Scanner module changes.
+ 
+ 2003-06-11 04:12  dejank
+ 
+ 	* modules/scanner/src/com/babeldoc/scanner/:
+ 	ScannerWorkerScheduler.java, ScannerWorkerFactory.java: Scanner
+ 	module changes.
+ 
+ 2003-06-10 23:45  triphop
+ 
+ 	* readme/CHANGES: Autogenerated CHANGES
+ 
  2003-06-10 23:24  triphop

[Babeldoc-devel] babeldoc/modules/core/src/com/babeldoc/core/pipeline PipelineStage.java,1.15,1.16 IPipelineStage.java,1.12,1.13

From: <tr...@us...> - 2003-07-17 00:22:13

Update of /cvsroot/babeldoc/babeldoc/modules/core/src/com/babeldoc/core/pipeline
In directory sc8-pr-cvs1:/tmp/cvs-serv21295

Modified Files:
	PipelineStage.java IPipelineStage.java 
Log Message:
removed the threaded attribute on the pipeline stage

Index: PipelineStage.java
===================================================================
RCS file: /cvsroot/babeldoc/babeldoc/modules/core/src/com/babeldoc/core/pipeline/PipelineStage.java,v
retrieving revision 1.15
retrieving revision 1.16
diff -C2 -d -r1.15 -r1.16
*** PipelineStage.java	15 Jul 2003 15:07:53 -0000	1.15
--- PipelineStage.java	17 Jul 2003 00:22:10 -0000	1.16
***************
*** 118,124 ****
    public static final String SPLIT_ATTRIBUTES = "splitAttributes";
    
-   /** constant: nextStages are threaded */
-   public static final String THREADED = "threaded";
-   
    /** constant: maximum number of threads for nextStages */
    public static final String MAX_THREADS = "maxThreads";
--- 118,121 ----
***************
*** 223,233 ****
    }
  
-   /**
-    * Return true if nextStages are threaded
-    **/
-   public boolean getThreaded() {
-      return "true".equalsIgnoreCase(this.getOptions(THREADED));   
-   }
-   
    /**
     * Return the maxThreads for the nextStages
--- 220,223 ----

Index: IPipelineStage.java
===================================================================
RCS file: /cvsroot/babeldoc/babeldoc/modules/core/src/com/babeldoc/core/pipeline/IPipelineStage.java,v
retrieving revision 1.12
retrieving revision 1.13
diff -C2 -d -r1.12 -r1.13
*** IPipelineStage.java	16 Jul 2003 05:17:39 -0000	1.12
--- IPipelineStage.java	17 Jul 2003 00:22:10 -0000	1.13
***************
*** 140,152 ****
  
    /**
-    * Return true if nextStages are threaded
-    **/
-   public boolean getThreaded();
-   
-   /**
     * Return the maxThreads for the nextStages
     **/
    public int getMaxThreads();
!   
    /**
     * pipeline stage processing core.  This is the heart of the process for the
--- 140,147 ----
  
    /**
     * Return the maxThreads for the nextStages
     **/
    public int getMaxThreads();
! 
    /**
     * pipeline stage processing core.  This is the heart of the process for the

Re: [Babeldoc-devel] Late night musings on parallel execution of pipeline stage results

From: Bruce M. <br...@mc...> - 2003-07-16 23:18:41

Ok all,

I have been kinda busy today working on this.  I have checked in code that 
will make the threading in the pipelines a policy of the pipeline stage 
factory and configurable from that point.  Please review the affected files.  
To get a good idea of how to use this - please take a look at the 
core/examples/threads example.  This contains three pipelines (with a single 
definition file) which implements all of the three threading processors.

1.  SyncPipelineStageProcessor  (this is the current implementation and is the 
default use unless the pipeline is configured otherwise.)
2.  AsyncPipelineStageProcessor (this is basically Jonathans code extracted 
from the the PipelineStageFactory.  This creates threads as necessary and 
then waits for them to complete)
3.  ThreadPooledPipelineStageProcessor  (this is a thread pooled 
implementation that I whipped into shape today - it still leaves something to 
be desired)

Here are some notes:

1.  The standard behavior is retained unless you configure it differently.
2.  All the configuration of the threading for pipelines is done in the 
pipeline/config.properties file. 
3.  The Async processor accepts hints from the pipeline definition file 
(maxThreads) 
4.  I have added three required properties to the services/query.properties 
file to look up the following service names: sync; async; threadpool

regards,
Bruce.

On Wednesday 16 July 2003 10:45 am, Leech, Jonathan wrote:
> Bruce,
>
> I haven't looked at the refactoring yet but that sounds excellent.  I had
> thought about using a proper ThreadPool but didn't have one and didn't
> think to look for one.  The problem with not letting a PipelineStage
> specify maxThreads is that you can end up with one PipelineStage using all
> the available threads, and starving other stages that you want to run
> concurrently.  Also, if you are using up a limited resource in the stage
> e.g. database connections, it doesn't make sense to have more threads
> running than connections.  But once you go with a thread pool, there should
> also be a maxThreads there as well.
>
> -Jonathan
>
> -----Original Message-----
> From: Bruce McDonald [mailto:br...@mc...]
> Sent: Tuesday, July 15, 2003 11:29 PM
> To: bab...@li...
> Subject: [Babeldoc-devel] Late night musings on parallel execution of
> pipeline stage results
>
>
> Jonathan, All:
>
> I have refactored the PipelineStageFactory so that methods:
>
> processPipelineStageResults    		(non-threaded)
> processPipelineStageResultsParallel	(threaded)
>
> are as similar as possible and all possible code overlaps are extracted to
> other methods.  At the core, the difference between the two of them is the
> maxThreads argument.  This argument indicates the number of threads to be
> spawned to handle the processing of the pipeline stage results.  I contend
> that this is an attribute of the pipeline stage factory and should be
> managed
> by a threadpool.  In fact, in direct contradiction to my prior email on
> this
>
> matter, an even simpler implementation would be to subclass the
> non-threaded
>
> implementation and override the processPipelineStageResults method to
> handle
>
> all the threaded complexity.
>
> thoughts?
>
> regards,
> Bruce.
>
> PS.  My thanks to Jonathan for stinging my lazy butt out of a destructive
> cycle of Rise Of Nations multiplayer games :)
>
> PPS.  I would seriously advise everyone to look at this code - it looks
> solid.
> That ThreadLocal thing was new to me.
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines at the
> same time. Free trial click here: http://www.vmware.com/wl/offer/345/0
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

[Babeldoc-devel] babeldoc/modules/core/config/service query.properties,1.8,1.9

From: <tr...@us...> - 2003-07-16 22:45:58

Update of /cvsroot/babeldoc/babeldoc/modules/core/config/service
In directory sc8-pr-cvs1:/tmp/cvs-serv4858/modules/core/config/service

Modified Files:
	query.properties 
Log Message:
New pipeline stage factory threading model has been implemented.

Index: query.properties
===================================================================
RCS file: /cvsroot/babeldoc/babeldoc/modules/core/config/service/query.properties,v
retrieving revision 1.8
retrieving revision 1.9
diff -C2 -d -r1.8 -r1.9
*** query.properties	30 Jun 2003 22:14:56 -0000	1.8
--- query.properties	16 Jul 2003 22:45:26 -0000	1.9
***************
*** 38,39 ****
--- 38,45 ----
  Feeder.asynchronous=com.babeldoc.core.pipeline.feeder.AsynchronousFeeder
  
+ 
+ 
+ PipelineStageProcessor.sync=com.babeldoc.core.pipeline.processor.SyncPipelineStageProcessor
+ PipelineStageProcessor.async=com.babeldoc.core.pipeline.processor.AsyncPipelineStageProcessor
+ PipelineStageProcessor.threadpool=com.babeldoc.core.pipeline.processor.ThreadPooledPipelineStageProcessor
+

[Babeldoc-devel] babeldoc build.properties,1.11,1.12

From: <tr...@us...> - 2003-07-16 22:45:58

Update of /cvsroot/babeldoc/babeldoc
In directory sc8-pr-cvs1:/tmp/cvs-serv4858

Modified Files:
	build.properties 
Log Message:
New pipeline stage factory threading model has been implemented.

Index: build.properties
===================================================================
RCS file: /cvsroot/babeldoc/babeldoc/build.properties,v
retrieving revision 1.11
retrieving revision 1.12
diff -C2 -d -r1.11 -r1.12
*** build.properties	10 Jun 2003 15:57:21 -0000	1.11
--- build.properties	16 Jul 2003 22:45:25 -0000	1.12
***************
*** 1,4 ****
  src = ./src
! babeldoc_version = 1.1.0
  config = ./config
  modules = ./modules
--- 1,4 ----
  src = ./src
! babeldoc_version = 1.1.3-dev
  config = ./config
  modules = ./modules

[Babeldoc-devel] babeldoc/modules/core/src/com/babeldoc/core/pipeline/processor AsyncPipelineStageProcessor.java,NONE,1.1 IPipelineStageProcessor.java,NONE,1.1 PipelineStageProcessorFactory.java,NONE,1.1 SyncPipelineStageProcessor.java,NONE,1.1 ThreadPooledPipelineStageProcessor.java,NONE,1.1

From: <tr...@us...> - 2003-07-16 22:45:33

Update of /cvsroot/babeldoc/babeldoc/modules/core/src/com/babeldoc/core/pipeline/processor
In directory sc8-pr-cvs1:/tmp/cvs-serv4858/modules/core/src/com/babeldoc/core/pipeline/processor

Added Files:
	AsyncPipelineStageProcessor.java IPipelineStageProcessor.java 
	PipelineStageProcessorFactory.java 
	SyncPipelineStageProcessor.java 
	ThreadPooledPipelineStageProcessor.java 
Log Message:
New pipeline stage factory threading model has been implemented.

--- NEW FILE: AsyncPipelineStageProcessor.java ---
/* ====================================================================
 * The Apache Software License, Version 1.1
 *
 * Copyright (c) 2000 The Apache Software Foundation.  All rights
 * reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in
 *    the documentation and/or other materials provided with the
 *    distribution.
 *
 * 3. The end-user documentation included with the redistribution,
 *    if any, must include the following acknowledgment:
 *       "This product includes software developed by the
 *        Apache Software Foundation (http://www.apache.org/)."
 *    Alternately, this acknowledgment may appear in the software itself,
 *    if and wherever such third-party acknowledgments normally appear.
 *
 * 4. The names "Apache" and "Apache Software Foundation" must
 *    not be used to endorse or promote products derived from this
 *    software without prior written permission. For written
 *    permission, please contact ap...@ap....
 *
 * 5. Products derived from this software may not be called "Apache",
 *    nor may "Apache" appear in their name, without prior written
 *    permission of the Apache Software Foundation.
 *
 * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
 * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
 * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED.  IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR
 * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
 * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
 * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
 * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 * SUCH DAMAGE.
 * ====================================================================
 *
 * This software consists of voluntary contributions made by many
 * individuals on behalf of the Apache Software Foundation.  For more
 * information on the Apache Software Foundation, please see
 * <http://www.apache.org/>.
 *
 * Portions of this software are based upon public domain software
 * originally written at the National Center for Supercomputing Applications,
 * University of Illinois, Urbana-Champaign.
 * ====================================================================
 *
 * Babeldoc:  The Universal Document Processor
 *
 * $Header: /cvsroot/babeldoc/babeldoc/modules/core/src/com/babeldoc/core/pipeline/processor/AsyncPipelineStageProcessor.java,v 1.1 2003/07/16 22:45:28 triphop Exp $
 * $DateTime$
 * $Author: triphop $
 *
 */
package com.babeldoc.core.pipeline.processor;

import com.babeldoc.core.pipeline.*;

import java.util.*;

import org.apache.commons.lang.NumberUtils;

/**
 * The asynchronous pipeline stage processor is capable of spawning threads to
 * handle multiple pipeline stage results as a result of pipeline stage processing.
 */
public class AsyncPipelineStageProcessor
    extends SyncPipelineStageProcessor {

  private int globalMaxThreads;
  public static final String MAX_THREADS = "maxThreads";
  public static final int DEFAULT_MAX_THREADS = 5;

  /**
   * This method is call prior to any processing - it is a necessary configuration
   * prequisite.
   *
   * @param stageFactory set the associated stage factory for this processor
   */
  public void configure(IPipelineStageFactory stageFactory, Map config) {
    super.configure(stageFactory, config);

    globalMaxThreads = NumberUtils.stringToInt((String)config.get(MAX_THREADS), DEFAULT_MAX_THREADS);
  }


  /**
   * Process the pipeline stage and then defer to eithe the multithreaded or single threaded
   * implementation - this needs to be extracted.
   *
   * @param pstage The pipeline stage to process
   * @param finalResults collection of documents after all processing is completed.
   * @throws PipelineException
   */
  protected void processPipelineStage(IPipelineStage pstage, Collection finalResults)
      throws PipelineException {

    // Process the pipeline stage, get the results of the processing
    PipelineStageResult[] psResults = pstage.processStage();

    if (psResults != null && psResults.length > 0) {
      if (psResults.length > 1) {
        int maxThreads = pstage.getMaxThreads();
        if (maxThreads <= 0) {
          maxThreads = 2; // default to 2 if <= 0
        }
        if(maxThreads>getGlobalMaxThreads()) {
          maxThreads = getGlobalMaxThreads();
        }
        processPipelineStageResultsParallel(psResults, finalResults, maxThreads);
      } else {
        processPipelineStageResults(psResults, finalResults);
      }
    }
  }

  /**
   * Execute the finalResults using threads in parallel.  This uses the maxThreads argument
   * to specify the maximum number of threads to spawn.  There is a slight impedance mismatch here.
   * The processPipelineStageResults methods (parallel and non-parallel) differ only in the
   * max threads argument.  This indicates the number of threads to spawn to handle the results.
   * Now, if we could consider this an attribute of this factory (or even of some threadpool) this
   * we can remove it from the interface - and we can conflate the interface.
   *
   * @param psResults results from the current pipeline stage process
   * @param finalResults result of the current pipeline stage process
   * @param maxThreads max number of threads to spawn to process the finalResults
   */
  protected void processPipelineStageResultsParallel(final PipelineStageResult[] psResults,
                                                     Collection finalResults,
                                                     final int maxThreads)
      throws PipelineException {
    // ensure synchronized access to finalResults Collection if threads are spawned
    final Collection syncResults = Collections.synchronizedCollection(finalResults);
    final int numResults = psResults.length;
    final Vector exceptions = new Vector();
    Vector threads = new Vector();
    for (int i = 0; i < maxThreads; i++) {
      final int index = i;
      Runnable r = new Runnable() {
        public void run() {
          for (int j = 0; j < numResults; j++) {
            if ((j % maxThreads) == index) {
              try {
                processPipelineStageResult(psResults[j], syncResults);
              } catch (PipelineException p) {
                exceptions.add(p);
              }
            }
          }
        }
      };
      Thread t = new Thread(r);
      threads.add(t);
      t.start();
    }

    // wait for any threads to finish before returning
    for (int i = 0; i < threads.size(); i++) {
      try {
        Thread t = (Thread) threads.get(i);
        t.join();
      } catch (InterruptedException ie) {
      }
    }
    // see if any of the threads threw any exceptions, and if so, re-throw the first one
    // TODO: throw an exception encapsulating all the exceptions that were thrown?
    // This might be harder than it appears unless we extend the PipelineException to
    // Add this capability
    if (exceptions.size() > 0) {
      throw (PipelineException) exceptions.get(0);
    }
  }

  /**
   * Get the globally configured number of threads allowed.
   *
   * @return
   */
  public int getGlobalMaxThreads() {
    return globalMaxThreads;
  }
}

--- NEW FILE: IPipelineStageProcessor.java ---
/* ====================================================================
 * The Apache Software License, Version 1.1
 *
 * Copyright (c) 2000 The Apache Software Foundation.  All rights
 * reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in
 *    the documentation and/or other materials provided with the
 *    distribution.
 *
 * 3. The end-user documentation included with the redistribution,
 *    if any, must include the following acknowledgment:
 *       "This product includes software developed by the
 *        Apache Software Foundation (http://www.apache.org/)."
 *    Alternately, this acknowledgment may appear in the software itself,
 *    if and wherever such third-party acknowledgments normally appear.
 *
 * 4. The names "Apache" and "Apache Software Foundation" must
 *    not be used to endorse or promote products derived from this
 *    software without prior written permission. For written
 *    permission, please contact ap...@ap....
 *
 * 5. Products derived from this software may not be called "Apache",
 *    nor may "Apache" appear in their name, without prior written
 *    permission of the Apache Software Foundation.
 *
 * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
 * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
 * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED.  IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR
 * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
 * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
 * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
 * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 * SUCH DAMAGE.
 * ====================================================================
 *
 * This software consists of voluntary contributions made by many
 * individuals on behalf of the Apache Software Foundation.  For more
 * information on the Apache Software Foundation, please see
 * <http://www.apache.org/>.
 *
 * Portions of this software are based upon public domain software
 * originally written at the National Center for Supercomputing Applications,
 * University of Illinois, Urbana-Champaign.
 * ====================================================================
 *
 * Babeldoc:  The Universal Document Processor
 *
 * $Header: /cvsroot/babeldoc/babeldoc/modules/core/src/com/babeldoc/core/pipeline/processor/IPipelineStageProcessor.java,v 1.1 2003/07/16 22:45:28 triphop Exp $
 * $DateTime$
 * $Author: triphop $
 *
 */
package com.babeldoc.core.pipeline.processor;

import com.babeldoc.core.pipeline.IPipelineStage;
import com.babeldoc.core.pipeline.PipelineException;
import com.babeldoc.core.pipeline.PipelineDocument;
import com.babeldoc.core.pipeline.IPipelineStageFactory;
import com.babeldoc.core.journal.IJournalTicket;

import java.util.Collection;
import java.util.Map;

/**
 * This interface represents that functionality required to execute a pipeline of
 * stages from a certain stage.  It has addition functionality for the maintainence
 * and configuration of state.
 *
 */
public interface IPipelineStageProcessor {
  public static final String PROCESSOR = "processor";

  /**
   * Get and instantiate the named pipeline stage.
   *
   * @param stageName the unique name of the stage
   *
   * @return a reference to the pipeline stage
   *
   * @throws PipelineException DOCUMENT ME!
   */
  public IPipelineStage getPipelineStage(String stageName)
      throws PipelineException;

  /**
   * Process documents - this is the entry point for pipeline processing.  If
   * the name of the next stage is not null, then process normally, otherwise
   * add them to the finalResults collection if the finalResults collection is not null.
   *
   * @param name of the next stage
   * @param document to process
   * @param ticket the tracker ticket
   * @param finalResults the resulting processed documents after all processing is completed
   *
   * @throws com.babeldoc.core.pipeline.PipelineException DOCUMENT ME!
   */
  public void process(String name, PipelineDocument document,
    IJournalTicket ticket, Collection finalResults) throws PipelineException;

  /**
   * This method is call prior to any processing - it is a necessary configuration
   * prequisite.
   *
   * @param stageFactory set the associated stage factory for this processor
   * @param options - the map of options specific to this process - this might be null
   */
  public void configure(IPipelineStageFactory stageFactory, Map options);
}

--- NEW FILE: PipelineStageProcessorFactory.java ---
package com.babeldoc.core.pipeline.processor;

import com.babeldoc.core.pipeline.IPipelineStageFactory;
import com.babeldoc.core.GeneralException;
import com.babeldoc.core.service.ServiceFactory;

import java.util.Map;

/**
 */
public class PipelineStageProcessorFactory {
  public static final String PROCESSOR_TYPE = "type";
  public static final String SERVICE_PROCESSOR = "PipelineStageProcessor.";

  /**
   * Quick and easy way to get a Pipeline stage processor implementation.  If the
   * config object is null (no processor specified in the pipeline configuration,
   * then the default processor is assigned.  If the configuration is given and is a
   * map (nested/tiered configuration), then the type is determined and a
   * processor is created and initialized.
   *
   * @param psFactory the pipeline stage factory associated with this processor
   * @param config the configuration object - this must either be a null or a map
   * @return a completely constructed Processor - ready to go.
   * @throws GeneralException
   */
  public static IPipelineStageProcessor getProcessor(IPipelineStageFactory psFactory, Object config)
    throws GeneralException {
    IPipelineStageProcessor processor = null;
    if(config == null) {
//      System.out.println("Config is null");
      processor = new SyncPipelineStageProcessor();
    } else if(!(config instanceof Map)) {
//      System.out.println("Config is bad");
      throw new GeneralException("The configuration settings for the processor must be null or a nested group");
    } else {
//      System.out.println("Config is good (I think)");
      Map map = (Map)config;
      String processorType = (String)map.get(PROCESSOR_TYPE);
      processor = (IPipelineStageProcessor)(ServiceFactory.getService(SERVICE_PROCESSOR+processorType));
    }

    processor.configure(psFactory, (Map)config);
    return processor;
  }
}

--- NEW FILE: SyncPipelineStageProcessor.java ---
/* ====================================================================
 * The Apache Software License, Version 1.1
 *
 * Copyright (c) 2000 The Apache Software Foundation.  All rights
 * reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in
 *    the documentation and/or other materials provided with the
 *    distribution.
 *
 * 3. The end-user documentation included with the redistribution,
 *    if any, must include the following acknowledgment:
 *       "This product includes software developed by the
 *        Apache Software Foundation (http://www.apache.org/)."
 *    Alternately, this acknowledgment may appear in the software itself,
 *    if and wherever such third-party acknowledgments normally appear.
 *
 * 4. The names "Apache" and "Apache Software Foundation" must
 *    not be used to endorse or promote products derived from this
 *    software without prior written permission. For written
 *    permission, please contact ap...@ap....
 *
 * 5. Products derived from this software may not be called "Apache",
 *    nor may "Apache" appear in their name, without prior written
 *    permission of the Apache Software Foundation.
 *
 * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
 * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
 * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED.  IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR
 * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
 * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
 * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
 * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 * SUCH DAMAGE.
 * ====================================================================
 *
 * This software consists of voluntary contributions made by many
 * individuals on behalf of the Apache Software Foundation.  For more
 * information on the Apache Software Foundation, please see
 * <http://www.apache.org/>.
 *
 * Portions of this software are based upon public domain software
 * originally written at the National Center for Supercomputing Applications,
 * University of Illinois, Urbana-Champaign.
 * ====================================================================
 *
 * Babeldoc:  The Universal Document Processor
 *
 * $Header: /cvsroot/babeldoc/babeldoc/modules/core/src/com/babeldoc/core/pipeline/processor/SyncPipelineStageProcessor.java,v 1.1 2003/07/16 22:45:28 triphop Exp $
 * $DateTime$
 * $Author: triphop $
 *
 */
package com.babeldoc.core.pipeline.processor;

import com.babeldoc.core.pipeline.*;
import com.babeldoc.core.journal.IJournalTicket;
import com.babeldoc.core.journal.JournalFactory;
import com.babeldoc.core.journal.JournalException;
import com.babeldoc.core.I18n;
import com.babeldoc.core.LogService;

import java.util.*;

/**
 * The default pipline stage processor executes ***EVERYTHING*** serialially.
 * This is of concern when a pipeline returns a number of results.  Most of this
 * code used to live in the pipeline stage factory.
 */
public class SyncPipelineStageProcessor
    implements IPipelineStageProcessor {

  private IPipelineStageFactory stageFactory;

  // this is put in a ThreadLocal, since with Threads the same stage can be called by several Threads at once
  private ThreadLocal localStages = new ThreadLocal() {
    protected synchronized Object initialValue() {
      LogService.getInstance().logDebug(I18n.get("019005", "new"));
      return new HashMap();
    }
  };


  /**
   * Get the "owning" pipeline stage factory.
   *
   * @return
   */
  public IPipelineStageFactory getStageFactory() {
    return stageFactory;
  }

  /**
   * Returns the pipeline stage type for this particular pipeline
   *
   * @param stageName
   *
   * @return
   *
   * @throws PipelineException
   */
  public PipelineStageType getPipelineStageType(String stageName)
      throws PipelineException {
    return getStageFactory().getResolver().getPipelineStageType(stageName);
  }

  /**
   * Get and instantiate the named pipeline stage.
   *
   * @param stageName the unique name of the stage
   *
   * @return a reference to the pipeline stage
   *
   * @throws PipelineException DOCUMENT ME!
   */
  public IPipelineStage getPipelineStage(String stageName)
      throws PipelineException {
    Map stages = (Map)localStages.get();

    if (!stages.containsKey(stageName)) {
      LogService.getInstance().logDebug(I18n.get("019005", stageName));

      try {
        PipelineStageType pipelineStageType = getPipelineStageType(stageName);
        Class stage = pipelineStageType.getTypeClass();

        if (stage != null) {
          IPipelineStage pstage = (IPipelineStage) (stage.newInstance());
          pstage.setName(stageName);
          pstage.setResolver(getStageFactory().getResolver());
          pstage.setPipelineName(getStageFactory().getName());
          stages.put(stageName, pstage);
        } else {
          throw new PipelineException(I18n.get("019010",
              pipelineStageType.getTypeName()));
        }
      } catch (InstantiationException ie) {
        throw new PipelineException(ie.getMessage(), ie);
      } catch (IllegalAccessException iae) {
        throw new PipelineException(iae.getMessage(), iae);
      }
    }

    return (IPipelineStage) stages.get(stageName);
  }


  /**
   * Process documents - this is the entry point for pipeline processing.  If
   * the name of the next stage is not null, then process normally, otherwise
   * add them to the finalResults collection if the finalResults collection is not null.
   *
   * @param name of the next stage
   * @param document to process
   * @param ticket the tracker ticket
   * @param finalResults the resulting processed documents after all processing is completed
   *
   * @throws com.babeldoc.core.pipeline.PipelineException DOCUMENT ME!
   */
  public void process(String name, PipelineDocument document,
    IJournalTicket ticket, Collection finalResults) throws PipelineException {
    if ((name != null) && !name.equals("null") && (document != null)) {
      LogService.getInstance().logInfo(I18n.get("019008", name));

      // Get the pipeline stage for this name
      IPipelineStage pstage = getPipelineStage(name);

      if (pstage != null) {
        try {
          pstage.initialize();
        } catch (Exception e) {
          throw new PipelineException("Exception while initializing", e);
        }
        pstage.setDocument(document);
        pstage.setTicket(ticket);
        trackDocument(pstage);

        processPipelineStage(pstage, finalResults);
      } else {
        LogService.getInstance().logError(I18n.get("019008", name), null);
      }
    } else {
      if (finalResults != null) {
        finalResults.add(new PipelineStageResult(null, document, ticket));
      }
    }
  }

  /**
   * This method is call prior to any processing - it is a necessary configuration
   * prequisite.
   *
   * @param stageFactory set the associated stage factory for this processor
   */
  public void configure(IPipelineStageFactory stageFactory, Map config) {
    this.stageFactory = stageFactory;
  }

  /**
   * Process the pipeline stage and then defer to eithe the multithreaded or single threaded
   * implementation - this needs to be extracted.
   *
   * @param pstage The pipeline stage to process
   * @param finalResults collection of documents after all processing is completed.
   * @throws com.babeldoc.core.pipeline.PipelineException
   */
  protected void processPipelineStage(IPipelineStage pstage, Collection finalResults)
      throws PipelineException {

    // Process the pipeline stage, get the results of the processing
    PipelineStageResult [] psResults = pstage.processStage();

    if(psResults!=null && psResults.length>0) {
      processPipelineStageResults(psResults, finalResults);
    }
  }

  /**
   * Process the pipeline stage finalResults.  The gets called from process and
   * handles the finalResults from the processing.
   *
   * @param psResults The results of running the current stage on this document
   * @param finalResults Collection of documents after all processing done (after null)
   *
   * @throws com.babeldoc.core.pipeline.PipelineException
   */
  protected void processPipelineStageResults(PipelineStageResult[] psResults,
    Collection finalResults) throws PipelineException {

    int numResults = psResults.length;
    for (int i = 0; i < numResults; ++i) {
      processPipelineStageResult(psResults[i], finalResults);
    }
  }

  /**
   * Process a single PipelineStageResult.
   *
   * @param psResult The pipeline stage result to process
   * @param finalResults the collection of processed documents.
   * @throws com.babeldoc.core.pipeline.PipelineException
   */
  protected void processPipelineStageResult(PipelineStageResult psResult, Collection finalResults)
      throws PipelineException {
    String name = psResult.getNamePipelineStage();
    PipelineDocument document = psResult.getDocument();
    IJournalTicket ticket = psResult.getTicket();

    this.process(name, document, ticket, finalResults);
  }

  /**
   * Track the document on the pipeline stage
   *
   * @param pstage
   *
   * @throws PipelineException DOCUMENT ME!
   */
  protected void trackDocument(IPipelineStage pstage)
      throws PipelineException {
    if (pstage.isTracked()) {
      try {
        JournalFactory.getJournal().updateDocument(pstage.getTicket(),
          pstage.getDocument(), getStageFactory().getName() + "." + pstage.getName());
      } catch (JournalException e) {
        LogService.getInstance().logError(e);
        throw new PipelineException(e.getMessage(), e);
      }
    }
  }
}

--- NEW FILE: ThreadPooledPipelineStageProcessor.java ---
/* ====================================================================
 * The Apache Software License, Version 1.1
 *
 * Copyright (c) 2000 The Apache Software Foundation.  All rights
 * reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in
 *    the documentation and/or other materials provided with the
 *    distribution.
 *
 * 3. The end-user documentation included with the redistribution,
 *    if any, must include the following acknowledgment:
 *       "This product includes software developed by the
 *        Apache Software Foundation (http://www.apache.org/)."
 *    Alternately, this acknowledgment may appear in the software itself,
 *    if and wherever such third-party acknowledgments normally appear.
 *
 * 4. The names "Apache" and "Apache Software Foundation" must
 *    not be used to endorse or promote products derived from this
 *    software without prior written permission. For written
 *    permission, please contact ap...@ap....
 *
 * 5. Products derived from this software may not be called "Apache",
 *    nor may "Apache" appear in their name, without prior written
 *    permission of the Apache Software Foundation.
 *
 * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
 * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
 * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED.  IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR
 * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
 * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
 * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
 * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 * SUCH DAMAGE.
 * ====================================================================
 *
 * This software consists of voluntary contributions made by many
 * individuals on behalf of the Apache Software Foundation.  For more
 * information on the Apache Software Foundation, please see
 * <http://www.apache.org/>.
 *
 * Portions of this software are based upon public domain software
 * originally written at the National Center for Supercomputing Applications,
 * University of Illinois, Urbana-Champaign.
 * ====================================================================
 *
 * Babeldoc:  The Universal Document Processor
 *
 * $Header: /cvsroot/babeldoc/babeldoc/modules/core/src/com/babeldoc/core/pipeline/processor/ThreadPooledPipelineStageProcessor.java,v 1.1 2003/07/16 22:45:28 triphop Exp $
 * $DateTime$
 * $Author: triphop $
 *
 */
package com.babeldoc.core.pipeline.processor;

import com.babeldoc.core.pipeline.*;
import com.babeldoc.core.LogService;

import java.util.*;

import org.apache.commons.lang.NumberUtils;
import org.apache.commons.threadpool.ThreadPool;
import org.apache.commons.threadpool.DefaultThreadPool;

/**
 * The thread pooled pipeline stage processor is a processor that maintains a
 * pool of threads that can be assigned processing.
 */
public class ThreadPooledPipelineStageProcessor
    extends SyncPipelineStageProcessor {

  public static final String POOL_SIZE = "poolSize";
  public static final int DEFAULT_POOLSIZE = 5;

  private int poolsize;
  private ThreadPool threadPool;

  /**
   * This method is call prior to any processing - it is a necessary configuration
   * prequisite.
   *
   * @param stageFactory set the associated stage factory for this processor
   */
  public void configure(IPipelineStageFactory stageFactory, Map config) {
    super.configure(stageFactory, config);

    poolsize = NumberUtils.stringToInt((String)config.get(POOL_SIZE), DEFAULT_POOLSIZE);
//    System.out.println("Setting up a pool with size of: "+poolsize);
    threadPool = new DefaultThreadPool(poolsize);
  }

  /**
   * Execute the pipeline stage result by adding runnables to the thread pool and
   * returning.  This means that this processing does return immediately after the
   * processing has completed.
   *
   * TODO:  finalResults document processing.
   * TODO:  Handle exceptions - at the moment it is logged.
   *
   * @param psResults results from the current pipeline stage process
   * @param finalResults result of the current pipeline stage process
   */
  protected void processPipelineStageResults(final PipelineStageResult[] psResults, Collection finalResults)
      throws PipelineException {
    // ensure synchronized access to finalResults Collection if threads are spawned
    final Collection syncResults = Collections.synchronizedCollection(finalResults);
    final int numResults = psResults.length;
    for (int j = 0; j < numResults; j++) {
      final PipelineStageResult psresult = psResults[j];
      threadPool.invokeLater(new Runnable() {
        public void run() {
          try {
            processPipelineStageResult(psresult, syncResults);
          } catch (PipelineException e) {
            LogService.getInstance().logError(e);
          }
        }
      });
    }
  }
}

[Babeldoc-devel] babeldoc/modules/core/src/com/babeldoc/core/pipeline/simple SimplePipelineStageResolver.java,1.4,1.5

From: <tr...@us...> - 2003-07-16 22:45:33

Update of /cvsroot/babeldoc/babeldoc/modules/core/src/com/babeldoc/core/pipeline/simple
In directory sc8-pr-cvs1:/tmp/cvs-serv4858/modules/core/src/com/babeldoc/core/pipeline/simple

Modified Files:
	SimplePipelineStageResolver.java 
Log Message:
New pipeline stage factory threading model has been implemented.

Index: SimplePipelineStageResolver.java
===================================================================
RCS file: /cvsroot/babeldoc/babeldoc/modules/core/src/com/babeldoc/core/pipeline/simple/SimplePipelineStageResolver.java,v
retrieving revision 1.4
retrieving revision 1.5
diff -C2 -d -r1.4 -r1.5
*** SimplePipelineStageResolver.java	27 Jun 2003 02:19:59 -0000	1.4
--- SimplePipelineStageResolver.java	16 Jul 2003 22:45:28 -0000	1.5
***************
*** 140,146 ****
  
        // HEAVY EXPLAINING:  For each token in the key (last token), create a
!       // Hashtable and map it to that that token in the working hashtable.
!       // point the working hashtable to the newly created hashtable.  If the last
!       // token is reached map the value to the name in the hashtable.
        StringTokenizer st = new StringTokenizer(key, ".");
        int numTokens = st.countTokens();
--- 140,146 ----
  
        // HEAVY EXPLAINING:  For each token in the key (last token), create a
!       // map and map it to that that token in the working map.
!       // point the working hashtable to the newly created map.  If the last
!       // token is reached map the value to the name in the map.
        StringTokenizer st = new StringTokenizer(key, ".");
        int numTokens = st.countTokens();

[Babeldoc-devel] babeldoc/modules/web/src/com/babeldoc/web/context PipelineListingContext.java,1.3,1.4

From: <tr...@us...> - 2003-07-16 22:45:32

Update of /cvsroot/babeldoc/babeldoc/modules/web/src/com/babeldoc/web/context
In directory sc8-pr-cvs1:/tmp/cvs-serv4858/modules/web/src/com/babeldoc/web/context

Modified Files:
	PipelineListingContext.java 
Log Message:
New pipeline stage factory threading model has been implemented.

Index: PipelineListingContext.java
===================================================================
RCS file: /cvsroot/babeldoc/babeldoc/modules/web/src/com/babeldoc/web/context/PipelineListingContext.java,v
retrieving revision 1.3
retrieving revision 1.4
diff -C2 -d -r1.3 -r1.4
*** PipelineListingContext.java	27 Jun 2003 02:09:20 -0000	1.3
--- PipelineListingContext.java	16 Jul 2003 22:45:28 -0000	1.4
***************
*** 186,189 ****
--- 186,190 ----
      try {
        return PipelineFactoryFactory.getPipelineStageFactory(pipeline)
+                                    .getResolver()
                                     .getPipelineStageType(stageName);
      } catch (PipelineException pipex) {

[Babeldoc-devel] babeldoc/modules/core/src/com/babeldoc/core/pipeline/command PipelineCommand.java,1.6,1.7

From: <tr...@us...> - 2003-07-16 22:45:30

Update of /cvsroot/babeldoc/babeldoc/modules/core/src/com/babeldoc/core/pipeline/command
In directory sc8-pr-cvs1:/tmp/cvs-serv4858/modules/core/src/com/babeldoc/core/pipeline/command

Modified Files:
	PipelineCommand.java 
Log Message:
New pipeline stage factory threading model has been implemented.

Index: PipelineCommand.java
===================================================================
RCS file: /cvsroot/babeldoc/babeldoc/modules/core/src/com/babeldoc/core/pipeline/command/PipelineCommand.java,v
retrieving revision 1.6
retrieving revision 1.7
diff -C2 -d -r1.6 -r1.7
*** PipelineCommand.java	27 Jun 2003 02:19:58 -0000	1.6
--- PipelineCommand.java	16 Jul 2003 22:45:27 -0000	1.7
***************
*** 272,276 ****
      IPipelineStageFactory pfactory = getPipelineStageFactory(pair);
      String stage = (String) pair.getSecond();
!     PipelineStageType pstageType = pfactory.getPipelineStageType(stage);
  
      if (pstageType != null) {
--- 272,276 ----
      IPipelineStageFactory pfactory = getPipelineStageFactory(pair);
      String stage = (String) pair.getSecond();
!     PipelineStageType pstageType = pfactory.getResolver().getPipelineStageType(stage);
  
      if (pstageType != null) {
***************
*** 295,299 ****
  
      if (stage != null) {
!       PipelineStageType ptype = pfactory.getPipelineStageType(stage);
  
        if (ptype != null) {
--- 295,299 ----
  
      if (stage != null) {
!       PipelineStageType ptype = pfactory.getResolver().getPipelineStageType(stage);
  
        if (ptype != null) {
***************
*** 407,411 ****
  
          if (flagType) {
!           String typeName = pfactory.getPipelineStageType(name).toString();
            System.out.print(I18n.get("008005", typeName));
          }
--- 407,411 ----
  
          if (flagType) {
!           String typeName = pfactory.getResolver().getPipelineStageType(name).toString();
            System.out.print(I18n.get("008005", typeName));
          }

2 messages has been excluded from this view by a project administrator.

Flat | Threaded

<< < 1 .. 93 94 95 96 97 .. 122 > >> (Page 95 of 122)