Re: [Exist-development] Execution Pipeline, async xquery execution...

[Adam R. <ad...@ex...>]

On 3 February 2010 15:46, Thomas White <tho...@gm...> wrote:
> Adam,
>
> Where shell I start, may be from the fact I was not aware there are new
> functions in the scheduler. Very lame :-)

I dont think these are new functions, but perhaps I am not aware that
someone else has augmented them?

> I guess when you say "one-shot asynchronous execution" you mean
> using schedule-xquery-periodic-job called with $period=0, and $repeat=1.
> It seams then I have everything I need to create the batch asynch
> functionality I described earlier.

Yes, that is exactly the case.

> BTW, is there a way to reduce the number remaining calls for
> schedule-xquery-periodic-job in run time?
> Use case: I need to process a dataset running periodically a XQuery function
> that process a chunk of that data. I expect to have 5000 calls but the data
> set  is smaller this time and I need to cancel it after the 50th call. Can I
> do this? If not, I hope we can then extend
> schedule-xquery-periodic-job function to cancel the job if the scheduled
> function returns false().

eXist maintains an Abstract idea of a Scheduler and the underlying
implementation is provided by Quartz, it should be fairly easy maybe 1
- 2 hours work to abstract around the Quatz Scheduler.rescheduleJob()
function and expose that functionality via the XQuery Scheduler
module.

> Regards,
> Thomas
>
> ------
>
> Thomas White
>
> Mobile:+44 7711 922 966
> Skype: thomaswhite
> gTalk: thomas.0007
> Linked-In:http://www.linkedin.com/in/thomaswhite0007
> facebook: http://www.facebook.com/thomas.0007
>
>
>
> On 3 February 2010 11:18, Adam Retter <ad...@ex...> wrote:
>>
>> > If we have an application where every user can fire a federated search
>> > to 25 remote servers, we do need a simple and robust way to cancel
>> > incompleted long running jobs, if the user makes a new federated
>> > search, logs off prematurely or when the session times out.
>>
>> This is where I struggle to understand your vision a little bit, and
>> maybe I need to revisit the original discussion. But all of what you
>> want above can be achieved with the Scheduler module - the Scheduler
>> module enables you to immediately Schedule an XQuery for one-shot
>> asynchronous execution, it also manages the status of running Queries
>> and enables you to cancel long running queries.
>>
>>
>> >
>> >
>> >
>> > ------
>> >
>> > Thomas White
>> >
>> > Mobile:+44 7711 922 966
>> > Skype: thomaswhite
>> > gTalk: thomas.0007
>> > Linked-In:http://www.linkedin.com/in/thomaswhite0007
>> > facebook: http://www.facebook.com/thomas.0007
>> >
>> >
>> >
>> > On 2 February 2010 16:15, Andrzej Jan Taramina <an...@ch...>
>> > wrote:
>> >>
>> >> Thomas asked:
>> >>
>> >> >>  do believe, having asynchronous mechanism for fetching data in
>> >> >> eXist will
>> >> >> > give us foundation for many?interesting new?ideas.
>> >> >> > I will need this functionality in about two months time and
>> >> >> > I?really hope
>> >> >> > somebody from the dev team will be interested in implementing the
>> >> >> > execution
>> >> >> > pipeline.
>> >>
>> >> To which Adam replied:
>> >>
>> >> > I think we will be very pushed for time to meet that deadline. The
>> >> > roadmap for the next two versions is already laid out - although it
>> >> > is
>> >> > subject to change.
>> >>
>> >> This feature, asynchronous execution of XQueries, is something that I'm
>> >> really interested in having as well.
>> >>
>> >> And I'm willing to implement this within the next two months (probably
>> >> sooner than that!).
>> >>
>> >> I was thinking of a new function along the lines of:
>> >>
>> >>        util:eval-async( ... )
>> >>
>> >> which would return some sort of id for the spawned xquery that could be
>> >> used to check completion status and obtain the
>> >> results of the execution. Same function signatures as the original
>> >> util:eval() function to start.
>> >>
>> >> Would this kind of new function meet your needs as well, Thomas?
>> >>
>> >> Thoughts from the crew on such a new function?
>> >>
>> >>
>> >> --
>> >> Andrzej Taramina
>> >> Chaeron Corporation: Enterprise System Solutions
>> >> http://www.chaeron.com
>> >
>>
>>
>>
>> --
>> Adam Retter
>>
>> eXist Developer
>> { United Kingdom }
>> ad...@ex...
>> irc://irc.freenode.net/existdb
>
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb