Re: [Exist-open] eXist-db RC2 becomes unresponsive

[Adam R. <ad...@ex...>]

Hi Nick,

eXist's scheduler is really an abstraction around the Terracotta
Quartz Scheduler. Quartz has the notion of StatefulJob which prevents
jobs from overlapping during execution, I should be able to add some
extra function signatures to the the XQuery Scheduler Module so that
you can control whether jobs overlap or not. We already make use of
StatefulJob for the internal SystemTasks which eXist schedules (e.g.
journal flush etc.)

If you dont hear anything in a few days - then prod me ;-)

On 22 January 2013 22:01, Nick Sincaglia <nsi...@nu...> wrote:
> After you sent your email, I started to reconsider my use of this module. The purpose for which I am using this module is that I have a variety of different programs which I schedule to run on fixed intervals using the scheduler module. The runtime duration of these programs can sometimes overlap with the next regularly scheduled interval. I was using the cache module to record when a program was actively running so that the scheduler could abort a regularly scheduled program from launching if the previously launched program was still running.
>
> If the cache module is as unsafe to use as you suggest it is, I guess, I could just write some sort of indicator into the job record document and have the program that the scheduler calls query that record looking for that indicator. I don't know if there is any advantage or disadvantage to using the cache versus a data record in the database. I would be interested in hearing opinions on which is better and why.
>
> Thanks,
>
> Nick
>
>
> On Jan 22, 2013, at 2:08 PM, Adam Retter wrote:
>
>> I dont see that I have to give an alternative, its not my
>> responsibility to maintain or improve other peoples code unless I
>> choose to do so. If the person who wrote this could not be bothered to
>> think about the impact, then that is their issue. I was interested to
>> see what this module did, as I had not heard of it before, when I
>> looked at the code, I saw problems, as someone is trying to use it, I
>> think its fair to let them know that it might not be a good idea to
>> use it.
>>
>> In addition, I dont understand actually why someone would need a
>> module like this? It seems to me like a shortcut around the functional
>> nature of XQuery.
>>
>> Perhaps Nick could explain his intentions to us and we could perhaps
>> advise a better approach...
>>
>> On 22 January 2013 18:49, Dmitriy Shabanov <sha...@gm...> wrote:
>>> I did test $job-id ='', nothing.
>>>
>>> Adam, this staff quite easy to make thread safe. If you advise against, you
>>> should give alternative then.
>>>
>>>
>>>
>>> On Tue, Jan 22, 2013 at 6:22 PM, Nick Sincaglia <nsi...@nu...>
>>> wrote:
>>>>
>>>> I agree, when $job-id has a real value (like '1'), the cache:put()
>>>> function works fine. This I have tested this and it works perfectly.
>>>>
>>>> However, when I accidentally set $job-id ='', the
>>>> cache:put('running-jobs','','true') function, I believe, fails and puts the
>>>> system into a weird state. It seems to get worse every time it is called.
>>>> Obviously, passing in a empty string as a job-id is not something one should
>>>> do but I wanted to bring this issue to surface to see if there are be ways
>>>> to guard against this mistake such that the system does not get into this
>>>> weird state and crash.
>>>>
>>>> I can create a much more basic program to try to eliminate potential other
>>>> function calls and try to recreate the issue.
>>>>
>>>> To call my script I register it with the scheduler module using this call:
>>>> scheduler:schedule-xquery-cron-job($which-remote-poll-query/text(), $cron,
>>>> $party-id, $params)
>>>>
>>>> The ftp:check-for-new-releeases() function connects to a remote ftp site
>>>> and retrieves file and folder listings and returns an XML file and then
>>>> disconnects. However, like I said earlier, this function works without any
>>>> problems when $job-id has a valid value in the variable.
>>>>
>>>> Nick
>>>>
>>>> On Jan 22, 2013, at 9:42 AM, Dmitriy Shabanov wrote:
>>>>
>>>> Running modified script don't give any problem
>>>>
>>>> let $job-id := "1"
>>>>
>>>> let $aaa := if (cache:get('running-jobs', $job-id) = 'true')
>>>>    then util:log-system-out(concat('Job ',$job-id ,' is already
>>>> running.'))
>>>>    else cache:put('running-jobs', $job-id, 'true')
>>>>
>>>> let $log2 := util:log-system-out(concat('Party ID  ', $job-id))
>>>>
>>>> return "done"
>>>>
>>>> Can you describe how do you call your script? And what doing
>>>> ftp:check-for-new-releases ?
>>>>
>>>>
>>>> On Mon, Jan 21, 2013 at 10:21 PM, Nick Sincaglia <nsi...@nu...>
>>>> wrote:
>>>>>
>>>>> I did a little more experimentation today to try to recreate the
>>>>> conditions which cause this issue. I narrowed it down to a small coding
>>>>> error in my code which then calls the cache:put() function.
>>>>>
>>>>> My coding error is that I was accidentally putting an empty string into a
>>>>> variable.(I was debugging my code both by manually passing variables and
>>>>> calling the code through the scheduler. I forgot to comment out some code
>>>>> when I switched between debugging methods.) I would then call the
>>>>> cache:put() function as such:
>>>>>
>>>>> declare variable $local:party-id as xs:string external;
>>>>> let $party-id := $local:party-id
>>>>> let $party-id := request:get-parameter('party-id', '')
>>>>> let $job-id := $party-id
>>>>>
>>>>> return
>>>>>  if (cache:get('running-jobs', $job-id) = 'true')
>>>>>    then util:log-system-out(concat('Job ',$job-id ,' is already
>>>>> running.')) (: if the job is running then exit :)
>>>>>    else
>>>>>    (: continue :)
>>>>>    let $set-running-flag := cache:put('running-jobs', $job-id, 'true')
>>>>>
>>>>> let $log2 := util:log-system-out(concat('Party ID  ', $party-id))
>>>>>
>>>>> My program would fail before reaching the util:log-system-out() function.
>>>>> Nothing ever was written to the log.
>>>>>
>>>>> The scheduler would call this code every 5 minutes. I discovered that
>>>>> after two calls, my system would start to no longer respond.
>>>
>>>
>>> --
>>> Dmitriy Shabanov
>>>
>>> ------------------------------------------------------------------------------
>>> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
>>> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
>>> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
>>> MVPs and experts. ON SALE this month only -- learn more at:
>>> http://p.sf.net/sfu/learnnow-d2d
>>> _______________________________________________
>>> Exist-open mailing list
>>> Exi...@li...
>>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>>
>>
>>
>>
>> --
>> Adam Retter
>>
>> eXist Developer
>> { United Kingdom }
>> ad...@ex...
>> irc://irc.freenode.net/existdb
>



--
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb