Re: [Exist-development] Execution Pipeline, async xquery execution...

[Thomas W. <tho...@gm...>]

Adam,

Where shell I start, may be from the fact I was not aware there are new
functions in the scheduler. Very lame :-)

I guess when you say "one-shot asynchronous execution" you mean
using schedule-xquery-periodic-job called with $period=0, and $repeat=1.
It seams then I have everything I need to create the batch asynch
functionality I described earlier.

BTW, is there a way to reduce the number remaining calls for *
schedule-xquery-periodic-job* in run time?
Use case: I need to process a dataset running periodically a XQuery function
that process a chunk of that data. I expect to have 5000 calls but the data
set  is smaller this time and I need to cancel it after the 50th call. Can I
do this? If not, I hope we can then extend
*schedule-xquery-periodic-job* function
to cancel the job if the scheduled function returns false().

Regards,
Thomas

------

Thomas White

Mobile:+44 7711 922 966
Skype: thomaswhite
gTalk: thomas.0007
Linked-In:http://www.linkedin.com/in/thomaswhite0007
facebook: http://www.facebook.com/thomas.0007



On 3 February 2010 11:18, Adam Retter <ad...@ex...> wrote:

> > If we have an application where every user can fire a federated search
> > to 25 remote servers, we do need a simple and robust way to cancel
> > incompleted long running jobs, if the user makes a new federated
> > search, logs off prematurely or when the session times out.
>
> This is where I struggle to understand your vision a little bit, and
> maybe I need to revisit the original discussion. But all of what you
> want above can be achieved with the Scheduler module - the Scheduler
> module enables you to immediately Schedule an XQuery for one-shot
> asynchronous execution, it also manages the status of running Queries
> and enables you to cancel long running queries.
>
>
> >
> >
> >
> > ------
> >
> > Thomas White
> >
> > Mobile:+44 7711 922 966
> > Skype: thomaswhite
> > gTalk: thomas.0007
> > Linked-In:http://www.linkedin.com/in/thomaswhite0007
> > facebook: http://www.facebook.com/thomas.0007
> >
> >
> >
> > On 2 February 2010 16:15, Andrzej Jan Taramina <an...@ch...>
> wrote:
> >>
> >> Thomas asked:
> >>
> >> >>  do believe, having asynchronous mechanism for fetching data in eXist
> will
> >> >> > give us foundation for many?interesting new?ideas.
> >> >> > I will need this functionality in about two months time and
> I?really hope
> >> >> > somebody from the dev team will be interested in implementing the
> execution
> >> >> > pipeline.
> >>
> >> To which Adam replied:
> >>
> >> > I think we will be very pushed for time to meet that deadline. The
> >> > roadmap for the next two versions is already laid out - although it is
> >> > subject to change.
> >>
> >> This feature, asynchronous execution of XQueries, is something that I'm
> really interested in having as well.
> >>
> >> And I'm willing to implement this within the next two months (probably
> sooner than that!).
> >>
> >> I was thinking of a new function along the lines of:
> >>
> >>        util:eval-async( ... )
> >>
> >> which would return some sort of id for the spawned xquery that could be
> used to check completion status and obtain the
> >> results of the execution. Same function signatures as the original
> util:eval() function to start.
> >>
> >> Would this kind of new function meet your needs as well, Thomas?
> >>
> >> Thoughts from the crew on such a new function?
> >>
> >>
> >> --
> >> Andrzej Taramina
> >> Chaeron Corporation: Enterprise System Solutions
> >> http://www.chaeron.com
> >
>
>
>
>  --
> Adam Retter
>
> eXist Developer
> { United Kingdom }
> ad...@ex...
> irc://irc.freenode.net/existdb
>