Re: [Babeldoc-devel] Re: Pipeline cache refreshing

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

On the whole, I'm happy with Bruce's line of thought.  I think that it
will be useful to build up a library of "local" configuration management
tools, helping components to manage their own configuration after it has
been retrieved using the existing functions.

Initially, I also thought that the existing configuration classes should
be responsible for the versioning and time-control of the config data,
but I think Bruce's idea works well, as it doesn't force complex config
handling where it is not required.

Cheers...

MikeA

On Sun, 2003-12-14 at 23:51, Bruce McDonald wrote:
> Sherman, All
> 
> Unfortunately I did not respond to the list with my email to Mike, but the 
> gist of what I said was this:  Introduce a listener / notification scheme for 
> the configuration server and the configuration clients.  The configuration 
> clients then can listen for changes to changes in the configuration data.  
> They can choose to update their own caches when it is "correct" for them to 
> do so.  This will mean that the pipeline factory will only flush its caches 
> (and subsequently reload from the changed data) after it has finished 
> processing all pipelines.  The scanner would do something similar.
> 
> The point is that the policy of how the configuration clients react to 
> configuration data changes doesnt (and shouldn't) be set globally - it is up 
> to the configuration client code to react in a fashion that is correct for 
> itself.
> 
> I think that this should cover the situation described by you.
> 
> So here is the rough sequence of events:
> 
> 1.  Configuration client (say, PipelineFactory, registers itself as a 
> configuration data change listener on the "pipeline/config" configuration.
> 2.  When the configuration server (though the configuration implementation) 
> notices that the "pipeline/config" has changed, it fires a DataChange event 
> to all listeners.
> 3.  The PipelineFactory then determines whether it can update its own cache.  
> This is probably done using two threads with a semaphore.  The pipeline 
> processing code grabs the semaphore and only releases it when it has finished 
> processing.  The configuration updater then blocks for the semaphore and when 
> the semaphore is free, it updates the configuration and releases the 
> semaphore.
> 
> Comments?
> 
> Bruce.
> 
> On Sunday 14 December 2003 05:21 pm, Sherman Wood wrote:
> > Ken Geis and I are just getting into this, as part of dynamic deployment of
> > new/changed pipeline definitions, scanners and supporting files ie. XSLs.
> >
> > I think that at the point in time where the pipeline(s) is being
> > reconfigured:
> >
> > * The scanner for the pipeline is stopped from accepting new documents
> > * Documents already in the pipeline finish processing
> > * The reconfiguration occurs: new files deployed etc
> > * A clearCache on the effected pipeline happens
> > * The scanner for the pipeline restarts
> >
> > To Mike's point about the "transactionality" of config data, it seems to me
> > that it is an unnecessary complication to implement the listener pattern to
> > reflect config updates.
> >
> > Cheers,
> >
> > Sherman
> >
> > > Subject: RE: [Babeldoc-devel] Pipeline cache refreshing
> > > From: Michael Ansley <mic...@ze...>
> > > To: bab...@li...
> > > Organization:
> > > Date: 11 Dec 2003 18:39:52 +0000
> > >
> > > I think that it requires more than that.  First of all, do we want
> > > pipelines changing while documents are being processed?  If not, then,
> > > do we version configs?  How?  What is the most basic operation that is
> > > allowed a consistent set of config data?  All pipelines should present a
> > > consistent set of config data to any single document.  Once any document
> > > has started processing, the config data used for processing that
> > > document should be fixed, in my opinion (I'm open to arguments to the
> > > contrary, though).  A guess it's a bit like transactioning metadata in a
> > > relational database.  If somebody does an ALTER TABLE that affects data
> > > your query is busy with, you don't want it affecting your data (because
> > > it's likely to be catastrophic).
> > >
> > > So, I'm open to ideas and opinions.
> > >
> > > Cheers...
> > >
> > >
> > > MikeA
> > >
> > > On Thu, 2003-12-11 at 14:17, McDonald, Bruce wrote:
> > > > Yes,
> > > >
> > > > Do that.
> > > >
> > > > At a more fundamental level, we probably need to implement an listener
> >
> > pattern on the configserver.  The client classes of the configuration data
> > (pipelines, journal, etc, etc) add themselves as data change listeners to
> > the configuration classes thereby being notified when the underlying data
> > changes.  They could register for change notices when the config files
> > changes...
> >
> > > > just an idea.
> > > >
> > > > Bruce.
> > > >
> > > > -----Original Message-----
> > > > From: Michael Ansley [mailto:mic...@ze...]
> > > > Sent: Wednesday, December 10, 2003 7:22 PM
> > > > To: bab...@li...
> > > > Subject: [Babeldoc-devel] Pipeline cache refreshing
> > > >
> > > >
> > > > Hi,
> > > >
> > > > I'm back on the trail of config refreshing, and now attacking the
> > > > pipeline configs.  It seems that PipelineFactory has a method
> > > > clearCache() and also clearCache(String name), but these methods are
> > > > not deemed generic to the role of IPipelineFactory.  Can I make them
> > > > generic?  Or are there implementations of IPipelineFactory for which
> > > > they do not make sense?
> > > >
> > > > Cheers...
> > > >
> > > >
> > > > MikeA
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: SF.net Giveback Program.
> > Does SourceForge.net help you be more productive?  Does it
> > help you create better code?  SHARE THE LOVE, and help us help
> > YOU!  Click Here: http://sourceforge.net/donate/
> > _______________________________________________
> > Babeldoc-devel mailing list
> > Bab...@li...
> > https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> Does SourceForge.net help you be more productive?  Does it
> help you create better code?  SHARE THE LOVE, and help us help
> YOU!  Click Here: http://sourceforge.net/donate/
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel