Re: [Babeldoc-devel] Re: Pipeline cache refreshing

[Bruce M. <br...@mc...>]

Sherman, All

Unfortunately I did not respond to the list with my email to Mike, but the 
gist of what I said was this:  Introduce a listener / notification scheme for 
the configuration server and the configuration clients.  The configuration 
clients then can listen for changes to changes in the configuration data.  
They can choose to update their own caches when it is "correct" for them to 
do so.  This will mean that the pipeline factory will only flush its caches 
(and subsequently reload from the changed data) after it has finished 
processing all pipelines.  The scanner would do something similar.

The point is that the policy of how the configuration clients react to 
configuration data changes doesnt (and shouldn't) be set globally - it is up 
to the configuration client code to react in a fashion that is correct for 
itself.

I think that this should cover the situation described by you.

So here is the rough sequence of events:

1.  Configuration client (say, PipelineFactory, registers itself as a 
configuration data change listener on the "pipeline/config" configuration.
2.  When the configuration server (though the configuration implementation) 
notices that the "pipeline/config" has changed, it fires a DataChange event 
to all listeners.
3.  The PipelineFactory then determines whether it can update its own cache.  
This is probably done using two threads with a semaphore.  The pipeline 
processing code grabs the semaphore and only releases it when it has finished 
processing.  The configuration updater then blocks for the semaphore and when 
the semaphore is free, it updates the configuration and releases the 
semaphore.

Comments?

Bruce.

On Sunday 14 December 2003 05:21 pm, Sherman Wood wrote:
> Ken Geis and I are just getting into this, as part of dynamic deployment of
> new/changed pipeline definitions, scanners and supporting files ie. XSLs.
>
> I think that at the point in time where the pipeline(s) is being
> reconfigured:
>
> * The scanner for the pipeline is stopped from accepting new documents
> * Documents already in the pipeline finish processing
> * The reconfiguration occurs: new files deployed etc
> * A clearCache on the effected pipeline happens
> * The scanner for the pipeline restarts
>
> To Mike's point about the "transactionality" of config data, it seems to me
> that it is an unnecessary complication to implement the listener pattern to
> reflect config updates.
>
> Cheers,
>
> Sherman
>
> > Subject: RE: [Babeldoc-devel] Pipeline cache refreshing
> > From: Michael Ansley <mic...@ze...>
> > To: bab...@li...
> > Organization:
> > Date: 11 Dec 2003 18:39:52 +0000
> >
> > I think that it requires more than that.  First of all, do we want
> > pipelines changing while documents are being processed?  If not, then,
> > do we version configs?  How?  What is the most basic operation that is
> > allowed a consistent set of config data?  All pipelines should present a
> > consistent set of config data to any single document.  Once any document
> > has started processing, the config data used for processing that
> > document should be fixed, in my opinion (I'm open to arguments to the
> > contrary, though).  A guess it's a bit like transactioning metadata in a
> > relational database.  If somebody does an ALTER TABLE that affects data
> > your query is busy with, you don't want it affecting your data (because
> > it's likely to be catastrophic).
> >
> > So, I'm open to ideas and opinions.
> >
> > Cheers...
> >
> >
> > MikeA
> >
> > On Thu, 2003-12-11 at 14:17, McDonald, Bruce wrote:
> > > Yes,
> > >
> > > Do that.
> > >
> > > At a more fundamental level, we probably need to implement an listener
>
> pattern on the configserver.  The client classes of the configuration data
> (pipelines, journal, etc, etc) add themselves as data change listeners to
> the configuration classes thereby being notified when the underlying data
> changes.  They could register for change notices when the config files
> changes...
>
> > > just an idea.
> > >
> > > Bruce.
> > >
> > > -----Original Message-----
> > > From: Michael Ansley [mailto:mic...@ze...]
> > > Sent: Wednesday, December 10, 2003 7:22 PM
> > > To: bab...@li...
> > > Subject: [Babeldoc-devel] Pipeline cache refreshing
> > >
> > >
> > > Hi,
> > >
> > > I'm back on the trail of config refreshing, and now attacking the
> > > pipeline configs.  It seems that PipelineFactory has a method
> > > clearCache() and also clearCache(String name), but these methods are
> > > not deemed generic to the role of IPipelineFactory.  Can I make them
> > > generic?  Or are there implementations of IPipelineFactory for which
> > > they do not make sense?
> > >
> > > Cheers...
> > >
> > >
> > > MikeA
>
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> Does SourceForge.net help you be more productive?  Does it
> help you create better code?  SHARE THE LOVE, and help us help
> YOU!  Click Here: http://sourceforge.net/donate/
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel