Re: [Xpipe-users] Starting with XPipe

[Sean M. <sea...@pr...>]

At 09:55 14/10/2003 +0100, Sergey Beryozkin wrote:
> > What is a preferred execution model for XComponents ?
>
>That was probably a silly question. I know that for XComponent to be
>executed it has to be inserted in XPipe, and XPipe engine will execute it.
>I'm just thinking about cooking up my own simple execution engine, if time
>allows.  My thinking was that because XPipe is just a sequence/composite
>list of XComponents, then an execution engine can execute both XPipe and
>XComponent, the same way as JUnit TestRunner can execute TestSuite and
>TestCase. Any other Composite exampel will do as well.

You just need an engine to exectute an XPipe. If you want to execute an 
XComponent
directly, you can just 'wrap' it in an XPipe shell before passing it to the 
execution
engine. That way, there is no need for the core engine to know anything
other than how to execute an XPipe.

>By the way, what is XGrid ?

XML processing, when approaches in term of data flow and pipelines, becomes 
what is
known in parallel computing circles as "trivially parallelisable". i.e. you 
can be pumping
many documents through a pipe at the same time and the execution of stage X 
for document A
is completely unrelated to the execution of stage X for document B. Given 
lots of processors
to do the work, you can get serious throughput this way.

Grid technology is a term increasingly used to describe the notion of 
computational power
available 'on tap'. I think it has a huge future for XML processing.

Note that a common objection to XGrid concept is that all the stages of 
processing document A have
to happen in order. So, if you have a small volume of large documents, 
XGrid buys you very little.

This would be true if not for the fact that most real world XML processing 
tasks operate on a segmented
view of an XML document. e.g. Customer by customer, chapter by chapter, 
widget by widget.

These segments - micro-documents can be processed independently. The 
scatter/gather concept discussed
in the powerpoint on the xpipe site talks about how two XComponents - 
scatter and gather - can work together
to allow documents to be split into segments (scattered) at 'fulcrum' 
points for the purposes of work distribution on
an XGrid and then re-assembled (gathered) before popping out of the end of 
a processing pipeline.

Sean

http://seanmcgrath.blogspot.com