taskforest-discuss Mailing List for TaskForest Job Scheduler (Page 2)
Brought to you by:
enoor
Menu
▾
▴
taskforest-discuss — General discussion, questions, feature requests, etc. related to TaskForest
You can subscribe to this list here.
| 2009 |
Jan
|
Feb
|
Mar
(10) |
Apr
(10) |
May
(10) |
Jun
(6) |
Jul
(4) |
Aug
|
Sep
(4) |
Oct
|
Nov
|
Dec
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2010 |
Jan
|
Feb
(10) |
Mar
(4) |
Apr
(9) |
May
|
Jun
|
Jul
|
Aug
|
Sep
(2) |
Oct
|
Nov
|
Dec
|
| 2011 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(1) |
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
(3) |
Nov
|
Dec
(1) |
| 2014 |
Jan
|
Feb
(2) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Steve H. <st...@id...> - 2010-02-12 04:33:42
|
Aijaz, Thanks for the update. TaskForest-1.34 passes all unit tests which were previously failing under similar conditions. Thanks again, Steve Hulet On Mon, Feb 8, 2010 at 9:32 AM, Aijaz Ansari <en...@us...> wrote: > Steve: > > I can confirm that there was a problem with the test cases (as opposed to > the actual program). I've fixed the test cases, and the scenario you were > describing works now. There's an unofficial version 1.34 available at > http://www.taskforest.com/TaskForest-1.34.tar.gz > > It's unofficial because I had already been working on version 1.34 and it's > got a few other enhancements, including the ability to put a job on hold and > release a hold off a job. I haven't finished working on the documentation > of these new features and don't want to release the code officially until > the user documentation is done. I expect it will take a few more days to > complete the docs. If you would like to, feel free to download 1.34 and > test it out. I'll release the official version as 1.35. > > Thanks again for your help in debugging this. > > Aijaz. > |
|
From: Aijaz A. <en...@us...> - 2010-02-08 17:35:15
|
Vitor: As I just mentioned to Steve on the mailing list, version 1.34 (available unofficially at http://www.taskforest.com/TaskForest-1.34.tar.gz ) should fix this problem. Please let me know if it doesn't. I'll release an official version 1.35 in a few days after I complete the user documentation. Thanks, Aijaz. On Tue, Feb 2, 2010 at 11:30 AM, Vitor Ventura <ven...@gm...>wrote: > Hello, > > I had the some problem, on the same platform. I never resolved it. > > ... > >> >> I just downloaded Task Forest 1.33 with high hopes and great >> expectations--I've been looking for a simple yet powerful >> text-file-based job scheduler of this sort for a long time. >> >> I ran `perl Makefile.PL`, `make`, and `make test` but many of the unit >> test fail and t/025_Midnight_Ready hangs after printing 1/19. Is this >> expected? Did I miss a setup or configuration step? >> >> I'm running perl v5.10.0 on Linux version 2.6.31-17-generic (Ubuntu >> 9.10). `make test` output included below. >> >> |
|
From: Aijaz A. <en...@us...> - 2010-02-08 17:32:42
|
Steve: I can confirm that there was a problem with the test cases (as opposed to the actual program). I've fixed the test cases, and the scenario you were describing works now. There's an unofficial version 1.34 available at http://www.taskforest.com/TaskForest-1.34.tar.gz It's unofficial because I had already been working on version 1.34 and it's got a few other enhancements, including the ability to put a job on hold and release a hold off a job. I haven't finished working on the documentation of these new features and don't want to release the code officially until the user documentation is done. I expect it will take a few more days to complete the docs. If you would like to, feel free to download 1.34 and test it out. I'll release the official version as 1.35. Thanks again for your help in debugging this. Aijaz. On Tue, Feb 2, 2010 at 11:19 AM, Aijaz Ansari <en...@us...>wrote: > Excellent detective work. Thanks, Steve. My comments are interspersed > below. > > On Tue, Feb 2, 2010 at 10:55 AM, Steve Hulet <st...@id...> wrote: > ... > >> >> Now for the news: running the exact same commands this morning (`perl >> Makefile.PL`, `make`, & `make test`) does *not* result in the same >> failures experienced last night. That is, all of the tests pass, >> *until* t/025_Midnight_Ready, which still hangs. >> > > I think the test case has a bug. The code uses its own function (called > LocalTime, I think) that returns the current time. That way during testing > I can change the 'current time' so that I can test how the program would > behave in the future. I think t/025_Midnight_Ready may not be using that > function, and therefore must be confused about what day it is. I can test > that out pretty easily. > > >> >> Interestingly, between 10pm and midnight PST and CST are on different >> days. Could this have caused the other `make test` failures? >> > > Yes. I'm pretty sure that's it. I think (hope) it's just a bug in the test > code and not in the actual program itself. I'll verify that over the next > day or two. It will take me a few days to get the new release out > officially, but I'll let you know what I find as I go along. > > >> >> Looking inside t/families/MIDNIGHT I see: >> >> >> start => '00:00', tz=>'GMT', days => 'Mon,Tue,Wed,Thu,Fri,Sat,Sun' >> >> J_005( start=>'04:59') # 11:59pm Chicago time April 19 2009 >> >> >> Based on the comment this test appears to make the assumption that >> 4:59 GMT is 11:59pm Chicago time. While this was true April 19, 2009 >> it is not true today due to Daylight Savings Time not being in effect. >> >> 2009-04-09: >> http://timeanddate.com/worldclock/converted.html?month=4&day=19&year=2009&hour=4&min=55&sec=0&p1=0&p2=64 >> 2010-02-02<http://timeanddate.com/worldclock/converted.html?month=4&day=19&year=2009&hour=4&min=55&sec=0&p1=0&p2=64%0A2010-02-02>: >> >> http://timeanddate.com/worldclock/converted.html?month=2&day=2&year=2010&hour=4&min=55&sec=0&p1=0&p2=64 >> >> Could this have caused t/025_Midnight_Ready to hang? >> > > I think this is ok. I don't have the code in front of me, but if I > remember correctly, the test case sets the 'current time' to some time on > 4/19/2009 (Chicago time). I'll double check this. > > You've given me all the information I need to be able to duplicate this at > home and determine whether there's a bug in the test program or in the > taskforest code itself, or both. > > As for the failure of the taskforestd daemon test, that's probably not > related, but I'll gladly help you get that running once these other issues > are resolved. > > Thanks again, > > Aijaz. > > >> >> Thanks, >> steve >> >> >> >> On Tue, Feb 2, 2010 at 6:17 AM, Aijaz Ansari >> <en...@us...> wrote: >> > Steve: >> > >> > Thanks for your detailed email. There may be a problem with the test >> case >> > or something unexpected in your setup. I'll try to duplicate this on >> my >> > end. Could you please answer these questions for me? It might help >> > identify the problem. >> > >> > What time zone is your computer set at, and at what time were you >> running >> > these tests? It really shouldn't matter, but it may help identify the >> > problem. >> > >> > Were you attempting to install via CPAN or did you download the package >> and >> > and manually type in perl Makefile.PL, make and make test? >> > >> > Did the linux user you were logged in as have write permission to the >> > current directory you were in? >> > >> > Thanks, >> > >> > Aijaz. >> > >> > >> > On Tue, Feb 2, 2010 at 1:23 AM, Steve Hulet <st...@id...> >> wrote: >> >> >> >> Hi, >> >> >> >> I just downloaded Task Forest 1.33 with high hopes and great >> >> expectations--I've been looking for a simple yet powerful >> >> text-file-based job scheduler of this sort for a long time. >> >> >> >> I ran `perl Makefile.PL`, `make`, and `make test` but many of the unit >> >> test fail and t/025_Midnight_Ready hangs after printing 1/19. Is this >> >> expected? Did I miss a setup or configuration step? >> >> >> >> I'm running perl v5.10.0 on Linux version 2.6.31-17-generic (Ubuntu >> >> 9.10). `make test` output included below. >> >> >> >> Thanks, >> >> steve >> >> >> > >> > <snip> >> > > |
|
From: Vitor V. <ven...@gm...> - 2010-02-02 17:30:21
|
Hello, I had the some problem, on the same platform. I never resolved it. On Tue, Feb 2, 2010 at 7:23 AM, Steve Hulet <st...@id...> wrote: > Hi, > > I just downloaded Task Forest 1.33 with high hopes and great > expectations--I've been looking for a simple yet powerful > text-file-based job scheduler of this sort for a long time. > > I ran `perl Makefile.PL`, `make`, and `make test` but many of the unit > test fail and t/025_Midnight_Ready hangs after printing 1/19. Is this > expected? Did I miss a setup or configuration step? > > I'm running perl v5.10.0 on Linux version 2.6.31-17-generic (Ubuntu > 9.10). `make test` output included below. > > Thanks, > steve > > > > > > PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" > "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t > t/001_DateTime.........................ok > t/002_TimeDependency...................ok > t/003_Options..........................ok > t/004_LogDir...........................ok > t/005_Job..............................ok > t/0065_StringHandle....................ok 1/16 > Ignore this line: Hello, world! > > Back to stderr - Ignore this line > Ignore this line: This is the first line > Ignore this line: This is the second line > t/0065_StringHandle....................ok > t/006_Family...........................ok 1/74 > # Failed test ' J1 is now ready' > t/006_Family...........................NOK 62/74# at t/006_Family.t line > 136. > # got: 'Waiting' > # expected: 'Ready' > > # Failed test ' J5 is now ready' > # at t/006_Family.t line 136. > t/006_Family...........................NOK 63/74# got: 'Waiting' > # expected: 'Ready' > > # Failed test ' J10 is now ready' > # at t/006_Family.t line 136. > # got: 'Waiting' > # expected: 'Ready' > > # Failed test ' J2 is has succeeded' > # at t/006_Family.t line 142. > # got: 'Ready' > # expected: 'Success' > > # Failed test ' J3 is has succeeded' > # at t/006_Family.t line 142. > # got: 'Ready' > # expected: 'Success' > > # Failed test ' J6 is has succeeded' > # at t/006_Family.t line 142. > # got: 'Ready' > # expected: 'Success' > > # Failed test ' J7 is has succeeded' > # at t/006_Family.t line 142. > # got: 'Ready' > # expected: 'Success' > > # Failed test ' J9 is has succeeded' > # at t/006_Family.t line 142. > # got: 'Ready' > # expected: 'Success' > # Looks like you failed 8 tests of 74. > t/006_Family...........................dubious > Test returned status 8 (wstat 2048, 0x800) > DIED. FAILED tests 62-64, 70-74 > Failed 8/74 tests, 89.19% okay > t/007_TaskForest.......................ok 1/7# Haven't found job log > files on try 1 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 2 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 3 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 4 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 5 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 6 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 7 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 8 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 9 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 10 of 10. Sleeping another 2 seconds > > # Failed test ' After first cycle, J2 ran successfully' > # at t/007_TaskForest.t line 65. > # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t > t/007_TaskForest.......................NOK 3/7 > # Failed test ' After first cycle, J3 ran successfully' > # at t/007_TaskForest.t line 66. > t/007_TaskForest.......................NOK 4/7# Try to increase the > value of $SLEEP_TIME in t/007_TaskForest.t > > # Failed test ' After first cycle, J6 ran successfully' > # at t/007_TaskForest.t line 67. > t/007_TaskForest.......................NOK 5/7# Try to increase the > value of $SLEEP_TIME in t/007_TaskForest.t > > # Failed test ' After first cycle, J7 ran successfully' > # at t/007_TaskForest.t line 68. > t/007_TaskForest.......................NOK 6/7# Try to increase the > value of $SLEEP_TIME in t/007_TaskForest.t > > # Failed test ' After first cycle, J9 ran successfully' > # at t/007_TaskForest.t line 69. > t/007_TaskForest.......................NOK 7/7# Try to increase the > value of $SLEEP_TIME in t/007_TaskForest.t > # Looks like you failed 5 tests of 7. > t/007_TaskForest.......................dubious > Test returned status 5 (wstat 1280, 0x500) > DIED. FAILED tests 3-7 > Failed 5/7 tests, 28.57% okay > t/008_Repeat...........................ok > t/009_Display..........................ok > t/010_LongRunningJobs..................ok 1/6 > # Failed test 'Got Line LONG_RUNNING::JLongRunning Ready > - America/Chicago 00:00 --:-- --:--' > # at /var/tmp/TaskForest-1.33/blib/lib/TaskForest/Test.pm line 28. > # 'LONG_RUNNING::JLongRunning Ready - > America/Chicago 00:00 --:-- --:--' > # doesn't match '(?-xism:LONG_RUNNING::JLongRunning +Running +- > +America/Chicago +00:00 +\d\d:\d\d +--:--)' > # Looks like you failed 1 test of 6. > t/010_LongRunningJobs..................dubious > Test returned status 1 (wstat 256, 0x100) > DIED. FAILED test 6 > Failed 1/6 tests, 83.33% okay > t/011_NotEnoughTime....................ok > t/012_Collapse.........................ok > t/013_Rerun............................ok > t/014_Mark.............................ok > t/015_Syntax...........................ok 1/18Odd number of elements > in hash assignment at > /var/tmp/TaskForest-1.33/blib/lib/TaskForest/Family.pm line 1155. > Use of uninitialized value $args in list assignment at > /var/tmp/TaskForest-1.33/blib/lib/TaskForest/Family.pm line 1155. > t/015_Syntax...........................ok > t/016_Cascade..........................ok > t/017_IgnoreRegex......................ok 1/12# Haven't found job log > files on try 1 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 2 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 3 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 4 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 5 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 6 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 7 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 8 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 9 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 10 of 10. Sleeping another 2 seconds > > # Failed test ' After first cycle, J1 ran successfully' > # at t/017_IgnoreRegex.t line 67. > # Try to increase the value of $SLEEP_TIME in t/017_IgnoreRegex.t > t/017_IgnoreRegex......................NOK 3/12 > # Failed test ' After first cycle, J2 ran successfully' > # at t/017_IgnoreRegex.t line 68. > t/017_IgnoreRegex......................NOK 4/12# Try to increase the > value of $SLEEP_TIME in t/017_IgnoreRegex.t > t/017_IgnoreRegex......................ok 5/12# Haven't found job log > files on try 1 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 2 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 3 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 4 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 5 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 6 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 7 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 8 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 9 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 10 of 10. Sleeping another 2 seconds > > # Failed test ' After second cycle, J1 ran successfully' > # at t/017_IgnoreRegex.t line 106. > # Try to increase the value of $SLEEP_TIME in t/017_IgnoreRegex.t > t/017_IgnoreRegex......................ok 9/12# Looks like you failed > 3 tests of 12. > t/017_IgnoreRegex......................dubious > Test returned status 3 (wstat 768, 0x300) > DIED. FAILED tests 3-4, 8 > Failed 3/12 tests, 75.00% okay > t/018TaskForestDaemon..................# > # ********************************************************************** > # > # No web server was found at port 1111. Skipping the rest of > # these tests. If you want, you can start the web server and > # rerun 'make test'. > # > # To start the web server enter: > # > # perl -T -I lib ./blib/script/taskforestd > --config_file=./taskforestd.test.cfg > # > # For more help, see http://www.taskforest.com > # > # ********************************************************************** > # > t/018TaskForestDaemon..................ok > 362/362 skipped: various reasons > t/019_FamilyRemoval....................ok > t/020_Run_With_Log.....................ok 1/7# Haven't found job log > files on try 1 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 2 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 3 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 4 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 5 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 6 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 7 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 8 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 9 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 10 of 10. Sleeping another 2 seconds > > # Failed test ' After first cycle, J2 ran successfully' > # at t/020_Run_With_Log.t line 65. > # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t > > # Failed test ' After first cycle, J3 ran successfully' > # at t/020_Run_With_Log.t line 66. > # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t > > # Failed test ' After first cycle, J6 ran successfully' > # at t/020_Run_With_Log.t line 67. > # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t > > # Failed test ' After first cycle, J7 ran successfully' > # at t/020_Run_With_Log.t line 68. > # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t > > # Failed test ' After first cycle, J9 ran successfully' > # at t/020_Run_With_Log.t line 69. > # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t > # Looks like you failed 5 tests of 7. > t/020_Run_With_Log.....................dubious > Test returned status 5 (wstat 1280, 0x500) > DIED. FAILED tests 3-7 > Failed 5/7 tests, 28.57% okay > t/021_Release..........................ok > t/022_Debug............................ok 1/7# Haven't found job log > files on try 1 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 2 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 3 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 4 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 5 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 6 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 7 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 8 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 9 of 10. Sleeping another 2 seconds > # Haven't found job log files on try 10 of 10. Sleeping another 2 seconds > > # Failed test ' After first cycle, J2 ran successfully' > # at t/022_Debug.t line 65. > # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t > t/022_Debug............................NOK 3/7 > # Failed test ' After first cycle, J3 ran successfully' > # at t/022_Debug.t line 66. > # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t > t/022_Debug............................NOK 4/7 > # Failed test ' After first cycle, J6 ran successfully' > # at t/022_Debug.t line 67. > t/022_Debug............................NOK 5/7# Try to increase the > value of $SLEEP_TIME in t/007_TaskForest.t > > # Failed test ' After first cycle, J7 ran successfully' > # at t/022_Debug.t line 68. > t/022_Debug............................NOK 6/7# Try to increase the > value of $SLEEP_TIME in t/007_TaskForest.t > > # Failed test ' After first cycle, J9 ran successfully' > # at t/022_Debug.t line 69. > t/022_Debug............................NOK 7/7# Try to increase the > value of $SLEEP_TIME in t/007_TaskForest.t > # Looks like you failed 5 tests of 7. > t/022_Debug............................dubious > Test returned status 5 (wstat 1280, 0x500) > DIED. FAILED tests 3-7 > Failed 5/7 tests, 28.57% okay > t/023_Tokens...........................ok 1/5# Loop # 5: missing the > following files: > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J1.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J3.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J4.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J7.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J8.0 > # Loop # 10: missing the following files: > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J1.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J3.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J4.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J7.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J8.0 > > # Failed test 'After first cycle, jobs J1, J3, J4, J7, J8 ran > successfully' > # at t/023_Tokens.t line 51. > t/023_Tokens...........................NOK 3/5# Loop # 5: missing the > following files: > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J2.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J5.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J9.0 > # Loop # 10: missing the following files: > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J2.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J5.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J9.0 > > # Failed test 'After second cycle, jobs J2, J5 and J9 ran successfully' > # at t/023_Tokens.t line 62. > t/023_Tokens...........................NOK 4/5# Loop # 5: missing the > following files: > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J6.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J10.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J11.0 > # Loop # 10: missing the following files: > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J6.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J10.0 > # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J11.0 > > # Failed test 'After third cycle, jobs J6, J10 and J11 ran successully' > # at t/023_Tokens.t line 71. > t/023_Tokens...........................NOK 5/5# Looks like you failed > 3 tests of 5. > t/023_Tokens...........................dubious > Test returned status 3 (wstat 768, 0x300) > DIED. FAILED tests 3-5 > Failed 3/5 tests, 40.00% okay > t/024_Midnight.........................ok > t/025_Midnight_Ready...................ok 1/19 > > > ------------------------------------------------------------------------------ > The Planet: dedicated and managed hosting, cloud storage, colocation > Stay online with enterprise data centers and the best network in the > business > Choose flexible plans and management services without long-term contracts > Personal 24x7 support from experience hosting pros just a phone call away. > http://p.sf.net/sfu/theplanet-com > _______________________________________________ > Taskforest-discuss mailing list > Tas...@li... > https://lists.sourceforge.net/lists/listinfo/taskforest-discuss > |
|
From: Aijaz A. <en...@us...> - 2010-02-02 17:19:35
|
Excellent detective work. Thanks, Steve. My comments are interspersed below. On Tue, Feb 2, 2010 at 10:55 AM, Steve Hulet <st...@id...> wrote: ... > > Now for the news: running the exact same commands this morning (`perl > Makefile.PL`, `make`, & `make test`) does *not* result in the same > failures experienced last night. That is, all of the tests pass, > *until* t/025_Midnight_Ready, which still hangs. > I think the test case has a bug. The code uses its own function (called LocalTime, I think) that returns the current time. That way during testing I can change the 'current time' so that I can test how the program would behave in the future. I think t/025_Midnight_Ready may not be using that function, and therefore must be confused about what day it is. I can test that out pretty easily. > > Interestingly, between 10pm and midnight PST and CST are on different > days. Could this have caused the other `make test` failures? > Yes. I'm pretty sure that's it. I think (hope) it's just a bug in the test code and not in the actual program itself. I'll verify that over the next day or two. It will take me a few days to get the new release out officially, but I'll let you know what I find as I go along. > > Looking inside t/families/MIDNIGHT I see: > > > start => '00:00', tz=>'GMT', days => 'Mon,Tue,Wed,Thu,Fri,Sat,Sun' > > J_005( start=>'04:59') # 11:59pm Chicago time April 19 2009 > > > Based on the comment this test appears to make the assumption that > 4:59 GMT is 11:59pm Chicago time. While this was true April 19, 2009 > it is not true today due to Daylight Savings Time not being in effect. > > 2009-04-09: > http://timeanddate.com/worldclock/converted.html?month=4&day=19&year=2009&hour=4&min=55&sec=0&p1=0&p2=64 > 2010-02-02<http://timeanddate.com/worldclock/converted.html?month=4&day=19&year=2009&hour=4&min=55&sec=0&p1=0&p2=64%0A2010-02-02>: > > http://timeanddate.com/worldclock/converted.html?month=2&day=2&year=2010&hour=4&min=55&sec=0&p1=0&p2=64 > > Could this have caused t/025_Midnight_Ready to hang? > I think this is ok. I don't have the code in front of me, but if I remember correctly, the test case sets the 'current time' to some time on 4/19/2009 (Chicago time). I'll double check this. You've given me all the information I need to be able to duplicate this at home and determine whether there's a bug in the test program or in the taskforest code itself, or both. As for the failure of the taskforestd daemon test, that's probably not related, but I'll gladly help you get that running once these other issues are resolved. Thanks again, Aijaz. > > Thanks, > steve > > > > On Tue, Feb 2, 2010 at 6:17 AM, Aijaz Ansari > <en...@us...> wrote: > > Steve: > > > > Thanks for your detailed email. There may be a problem with the test > case > > or something unexpected in your setup. I'll try to duplicate this on my > > end. Could you please answer these questions for me? It might help > > identify the problem. > > > > What time zone is your computer set at, and at what time were you running > > these tests? It really shouldn't matter, but it may help identify the > > problem. > > > > Were you attempting to install via CPAN or did you download the package > and > > and manually type in perl Makefile.PL, make and make test? > > > > Did the linux user you were logged in as have write permission to the > > current directory you were in? > > > > Thanks, > > > > Aijaz. > > > > > > On Tue, Feb 2, 2010 at 1:23 AM, Steve Hulet <st...@id...> > wrote: > >> > >> Hi, > >> > >> I just downloaded Task Forest 1.33 with high hopes and great > >> expectations--I've been looking for a simple yet powerful > >> text-file-based job scheduler of this sort for a long time. > >> > >> I ran `perl Makefile.PL`, `make`, and `make test` but many of the unit > >> test fail and t/025_Midnight_Ready hangs after printing 1/19. Is this > >> expected? Did I miss a setup or configuration step? > >> > >> I'm running perl v5.10.0 on Linux version 2.6.31-17-generic (Ubuntu > >> 9.10). `make test` output included below. > >> > >> Thanks, > >> steve > >> > > > > <snip> > |
|
From: Steve H. <st...@id...> - 2010-02-02 16:55:51
|
Aijaz, Thanks for the prompt reply. I think you asked the right questions. Here is what I have learned this morning. First the easy questions: I saw the exact same set of failures last night installing both through CPAN and directly from the tarball. Yes, all of the relevant directories are writeable. Now for the news: running the exact same commands this morning (`perl Makefile.PL`, `make`, & `make test`) does *not* result in the same failures experienced last night. That is, all of the tests pass, *until* t/025_Midnight_Ready, which still hangs. When I was installing TaskForest last night it was between 10pm and midnight, PST. My system time zone was set to 'America/Los_Angeles'. If I change my system time zone to 'America/Chicago' then t/025_Midnight_Ready does not hang, and all test pass with the exception of t/018TaskForestDaemon, which seems to have errors either way (after starting the daemon with the suggested `perl -T -I lib ./blib/script/taskforestd --config_file=./taskforestd.test.cfg`). Interestingly, between 10pm and midnight PST and CST are on different days. Could this have caused the other `make test` failures? Looking inside t/families/MIDNIGHT I see: start => '00:00', tz=>'GMT', days => 'Mon,Tue,Wed,Thu,Fri,Sat,Sun' J_005( start=>'04:59') # 11:59pm Chicago time April 19 2009 Based on the comment this test appears to make the assumption that 4:59 GMT is 11:59pm Chicago time. While this was true April 19, 2009 it is not true today due to Daylight Savings Time not being in effect. 2009-04-09: http://timeanddate.com/worldclock/converted.html?month=4&day=19&year=2009&hour=4&min=55&sec=0&p1=0&p2=64 2010-02-02: http://timeanddate.com/worldclock/converted.html?month=2&day=2&year=2010&hour=4&min=55&sec=0&p1=0&p2=64 Could this have caused t/025_Midnight_Ready to hang? Thanks, steve On Tue, Feb 2, 2010 at 6:17 AM, Aijaz Ansari <en...@us...> wrote: > Steve: > > Thanks for your detailed email. There may be a problem with the test case > or something unexpected in your setup. I'll try to duplicate this on my > end. Could you please answer these questions for me? It might help > identify the problem. > > What time zone is your computer set at, and at what time were you running > these tests? It really shouldn't matter, but it may help identify the > problem. > > Were you attempting to install via CPAN or did you download the package and > and manually type in perl Makefile.PL, make and make test? > > Did the linux user you were logged in as have write permission to the > current directory you were in? > > Thanks, > > Aijaz. > > > On Tue, Feb 2, 2010 at 1:23 AM, Steve Hulet <st...@id...> wrote: >> >> Hi, >> >> I just downloaded Task Forest 1.33 with high hopes and great >> expectations--I've been looking for a simple yet powerful >> text-file-based job scheduler of this sort for a long time. >> >> I ran `perl Makefile.PL`, `make`, and `make test` but many of the unit >> test fail and t/025_Midnight_Ready hangs after printing 1/19. Is this >> expected? Did I miss a setup or configuration step? >> >> I'm running perl v5.10.0 on Linux version 2.6.31-17-generic (Ubuntu >> 9.10). `make test` output included below. >> >> Thanks, >> steve >> > > <snip> |
|
From: Aijaz A. <en...@us...> - 2010-02-02 14:17:45
|
Steve: Thanks for your detailed email. There may be a problem with the test case or something unexpected in your setup. I'll try to duplicate this on my end. Could you please answer these questions for me? It might help identify the problem. What time zone is your computer set at, and at what time were you running these tests? It really shouldn't matter, but it may help identify the problem. Were you attempting to install via CPAN or did you download the package and and manually type in perl Makefile.PL, make and make test? Did the linux user you were logged in as have write permission to the current directory you were in? Thanks, Aijaz. On Tue, Feb 2, 2010 at 1:23 AM, Steve Hulet <st...@id...> wrote: > Hi, > > I just downloaded Task Forest 1.33 with high hopes and great > expectations--I've been looking for a simple yet powerful > text-file-based job scheduler of this sort for a long time. > > I ran `perl Makefile.PL`, `make`, and `make test` but many of the unit > test fail and t/025_Midnight_Ready hangs after printing 1/19. Is this > expected? Did I miss a setup or configuration step? > > I'm running perl v5.10.0 on Linux version 2.6.31-17-generic (Ubuntu > 9.10). `make test` output included below. > > Thanks, > steve > > <snip> |
|
From: Steve H. <st...@id...> - 2010-02-02 07:23:54
|
Hi, I just downloaded Task Forest 1.33 with high hopes and great expectations--I've been looking for a simple yet powerful text-file-based job scheduler of this sort for a long time. I ran `perl Makefile.PL`, `make`, and `make test` but many of the unit test fail and t/025_Midnight_Ready hangs after printing 1/19. Is this expected? Did I miss a setup or configuration step? I'm running perl v5.10.0 on Linux version 2.6.31-17-generic (Ubuntu 9.10). `make test` output included below. Thanks, steve PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/001_DateTime.........................ok t/002_TimeDependency...................ok t/003_Options..........................ok t/004_LogDir...........................ok t/005_Job..............................ok t/0065_StringHandle....................ok 1/16 Ignore this line: Hello, world! Back to stderr - Ignore this line Ignore this line: This is the first line Ignore this line: This is the second line t/0065_StringHandle....................ok t/006_Family...........................ok 1/74 # Failed test ' J1 is now ready' t/006_Family...........................NOK 62/74# at t/006_Family.t line 136. # got: 'Waiting' # expected: 'Ready' # Failed test ' J5 is now ready' # at t/006_Family.t line 136. t/006_Family...........................NOK 63/74# got: 'Waiting' # expected: 'Ready' # Failed test ' J10 is now ready' # at t/006_Family.t line 136. # got: 'Waiting' # expected: 'Ready' # Failed test ' J2 is has succeeded' # at t/006_Family.t line 142. # got: 'Ready' # expected: 'Success' # Failed test ' J3 is has succeeded' # at t/006_Family.t line 142. # got: 'Ready' # expected: 'Success' # Failed test ' J6 is has succeeded' # at t/006_Family.t line 142. # got: 'Ready' # expected: 'Success' # Failed test ' J7 is has succeeded' # at t/006_Family.t line 142. # got: 'Ready' # expected: 'Success' # Failed test ' J9 is has succeeded' # at t/006_Family.t line 142. # got: 'Ready' # expected: 'Success' # Looks like you failed 8 tests of 74. t/006_Family...........................dubious Test returned status 8 (wstat 2048, 0x800) DIED. FAILED tests 62-64, 70-74 Failed 8/74 tests, 89.19% okay t/007_TaskForest.......................ok 1/7# Haven't found job log files on try 1 of 10. Sleeping another 2 seconds # Haven't found job log files on try 2 of 10. Sleeping another 2 seconds # Haven't found job log files on try 3 of 10. Sleeping another 2 seconds # Haven't found job log files on try 4 of 10. Sleeping another 2 seconds # Haven't found job log files on try 5 of 10. Sleeping another 2 seconds # Haven't found job log files on try 6 of 10. Sleeping another 2 seconds # Haven't found job log files on try 7 of 10. Sleeping another 2 seconds # Haven't found job log files on try 8 of 10. Sleeping another 2 seconds # Haven't found job log files on try 9 of 10. Sleeping another 2 seconds # Haven't found job log files on try 10 of 10. Sleeping another 2 seconds # Failed test ' After first cycle, J2 ran successfully' # at t/007_TaskForest.t line 65. # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t t/007_TaskForest.......................NOK 3/7 # Failed test ' After first cycle, J3 ran successfully' # at t/007_TaskForest.t line 66. t/007_TaskForest.......................NOK 4/7# Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t # Failed test ' After first cycle, J6 ran successfully' # at t/007_TaskForest.t line 67. t/007_TaskForest.......................NOK 5/7# Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t # Failed test ' After first cycle, J7 ran successfully' # at t/007_TaskForest.t line 68. t/007_TaskForest.......................NOK 6/7# Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t # Failed test ' After first cycle, J9 ran successfully' # at t/007_TaskForest.t line 69. t/007_TaskForest.......................NOK 7/7# Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t # Looks like you failed 5 tests of 7. t/007_TaskForest.......................dubious Test returned status 5 (wstat 1280, 0x500) DIED. FAILED tests 3-7 Failed 5/7 tests, 28.57% okay t/008_Repeat...........................ok t/009_Display..........................ok t/010_LongRunningJobs..................ok 1/6 # Failed test 'Got Line LONG_RUNNING::JLongRunning Ready - America/Chicago 00:00 --:-- --:--' # at /var/tmp/TaskForest-1.33/blib/lib/TaskForest/Test.pm line 28. # 'LONG_RUNNING::JLongRunning Ready - America/Chicago 00:00 --:-- --:--' # doesn't match '(?-xism:LONG_RUNNING::JLongRunning +Running +- +America/Chicago +00:00 +\d\d:\d\d +--:--)' # Looks like you failed 1 test of 6. t/010_LongRunningJobs..................dubious Test returned status 1 (wstat 256, 0x100) DIED. FAILED test 6 Failed 1/6 tests, 83.33% okay t/011_NotEnoughTime....................ok t/012_Collapse.........................ok t/013_Rerun............................ok t/014_Mark.............................ok t/015_Syntax...........................ok 1/18Odd number of elements in hash assignment at /var/tmp/TaskForest-1.33/blib/lib/TaskForest/Family.pm line 1155. Use of uninitialized value $args in list assignment at /var/tmp/TaskForest-1.33/blib/lib/TaskForest/Family.pm line 1155. t/015_Syntax...........................ok t/016_Cascade..........................ok t/017_IgnoreRegex......................ok 1/12# Haven't found job log files on try 1 of 10. Sleeping another 2 seconds # Haven't found job log files on try 2 of 10. Sleeping another 2 seconds # Haven't found job log files on try 3 of 10. Sleeping another 2 seconds # Haven't found job log files on try 4 of 10. Sleeping another 2 seconds # Haven't found job log files on try 5 of 10. Sleeping another 2 seconds # Haven't found job log files on try 6 of 10. Sleeping another 2 seconds # Haven't found job log files on try 7 of 10. Sleeping another 2 seconds # Haven't found job log files on try 8 of 10. Sleeping another 2 seconds # Haven't found job log files on try 9 of 10. Sleeping another 2 seconds # Haven't found job log files on try 10 of 10. Sleeping another 2 seconds # Failed test ' After first cycle, J1 ran successfully' # at t/017_IgnoreRegex.t line 67. # Try to increase the value of $SLEEP_TIME in t/017_IgnoreRegex.t t/017_IgnoreRegex......................NOK 3/12 # Failed test ' After first cycle, J2 ran successfully' # at t/017_IgnoreRegex.t line 68. t/017_IgnoreRegex......................NOK 4/12# Try to increase the value of $SLEEP_TIME in t/017_IgnoreRegex.t t/017_IgnoreRegex......................ok 5/12# Haven't found job log files on try 1 of 10. Sleeping another 2 seconds # Haven't found job log files on try 2 of 10. Sleeping another 2 seconds # Haven't found job log files on try 3 of 10. Sleeping another 2 seconds # Haven't found job log files on try 4 of 10. Sleeping another 2 seconds # Haven't found job log files on try 5 of 10. Sleeping another 2 seconds # Haven't found job log files on try 6 of 10. Sleeping another 2 seconds # Haven't found job log files on try 7 of 10. Sleeping another 2 seconds # Haven't found job log files on try 8 of 10. Sleeping another 2 seconds # Haven't found job log files on try 9 of 10. Sleeping another 2 seconds # Haven't found job log files on try 10 of 10. Sleeping another 2 seconds # Failed test ' After second cycle, J1 ran successfully' # at t/017_IgnoreRegex.t line 106. # Try to increase the value of $SLEEP_TIME in t/017_IgnoreRegex.t t/017_IgnoreRegex......................ok 9/12# Looks like you failed 3 tests of 12. t/017_IgnoreRegex......................dubious Test returned status 3 (wstat 768, 0x300) DIED. FAILED tests 3-4, 8 Failed 3/12 tests, 75.00% okay t/018TaskForestDaemon..................# # ********************************************************************** # # No web server was found at port 1111. Skipping the rest of # these tests. If you want, you can start the web server and # rerun 'make test'. # # To start the web server enter: # # perl -T -I lib ./blib/script/taskforestd --config_file=./taskforestd.test.cfg # # For more help, see http://www.taskforest.com # # ********************************************************************** # t/018TaskForestDaemon..................ok 362/362 skipped: various reasons t/019_FamilyRemoval....................ok t/020_Run_With_Log.....................ok 1/7# Haven't found job log files on try 1 of 10. Sleeping another 2 seconds # Haven't found job log files on try 2 of 10. Sleeping another 2 seconds # Haven't found job log files on try 3 of 10. Sleeping another 2 seconds # Haven't found job log files on try 4 of 10. Sleeping another 2 seconds # Haven't found job log files on try 5 of 10. Sleeping another 2 seconds # Haven't found job log files on try 6 of 10. Sleeping another 2 seconds # Haven't found job log files on try 7 of 10. Sleeping another 2 seconds # Haven't found job log files on try 8 of 10. Sleeping another 2 seconds # Haven't found job log files on try 9 of 10. Sleeping another 2 seconds # Haven't found job log files on try 10 of 10. Sleeping another 2 seconds # Failed test ' After first cycle, J2 ran successfully' # at t/020_Run_With_Log.t line 65. # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t # Failed test ' After first cycle, J3 ran successfully' # at t/020_Run_With_Log.t line 66. # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t # Failed test ' After first cycle, J6 ran successfully' # at t/020_Run_With_Log.t line 67. # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t # Failed test ' After first cycle, J7 ran successfully' # at t/020_Run_With_Log.t line 68. # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t # Failed test ' After first cycle, J9 ran successfully' # at t/020_Run_With_Log.t line 69. # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t # Looks like you failed 5 tests of 7. t/020_Run_With_Log.....................dubious Test returned status 5 (wstat 1280, 0x500) DIED. FAILED tests 3-7 Failed 5/7 tests, 28.57% okay t/021_Release..........................ok t/022_Debug............................ok 1/7# Haven't found job log files on try 1 of 10. Sleeping another 2 seconds # Haven't found job log files on try 2 of 10. Sleeping another 2 seconds # Haven't found job log files on try 3 of 10. Sleeping another 2 seconds # Haven't found job log files on try 4 of 10. Sleeping another 2 seconds # Haven't found job log files on try 5 of 10. Sleeping another 2 seconds # Haven't found job log files on try 6 of 10. Sleeping another 2 seconds # Haven't found job log files on try 7 of 10. Sleeping another 2 seconds # Haven't found job log files on try 8 of 10. Sleeping another 2 seconds # Haven't found job log files on try 9 of 10. Sleeping another 2 seconds # Haven't found job log files on try 10 of 10. Sleeping another 2 seconds # Failed test ' After first cycle, J2 ran successfully' # at t/022_Debug.t line 65. # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t t/022_Debug............................NOK 3/7 # Failed test ' After first cycle, J3 ran successfully' # at t/022_Debug.t line 66. # Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t t/022_Debug............................NOK 4/7 # Failed test ' After first cycle, J6 ran successfully' # at t/022_Debug.t line 67. t/022_Debug............................NOK 5/7# Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t # Failed test ' After first cycle, J7 ran successfully' # at t/022_Debug.t line 68. t/022_Debug............................NOK 6/7# Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t # Failed test ' After first cycle, J9 ran successfully' # at t/022_Debug.t line 69. t/022_Debug............................NOK 7/7# Try to increase the value of $SLEEP_TIME in t/007_TaskForest.t # Looks like you failed 5 tests of 7. t/022_Debug............................dubious Test returned status 5 (wstat 1280, 0x500) DIED. FAILED tests 3-7 Failed 5/7 tests, 28.57% okay t/023_Tokens...........................ok 1/5# Loop # 5: missing the following files: # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J1.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J3.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J4.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J7.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J8.0 # Loop # 10: missing the following files: # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J1.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J3.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J4.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J7.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J8.0 # Failed test 'After first cycle, jobs J1, J3, J4, J7, J8 ran successfully' # at t/023_Tokens.t line 51. t/023_Tokens...........................NOK 3/5# Loop # 5: missing the following files: # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J2.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J5.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J9.0 # Loop # 10: missing the following files: # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J2.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J5.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J9.0 # Failed test 'After second cycle, jobs J2, J5 and J9 ran successfully' # at t/023_Tokens.t line 62. t/023_Tokens...........................NOK 4/5# Loop # 5: missing the following files: # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J6.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J10.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J11.0 # Loop # 10: missing the following files: # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J6.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J10.0 # /var/tmp/TaskForest-1.33/t/logs/20100201/TOKENS.J11.0 # Failed test 'After third cycle, jobs J6, J10 and J11 ran successully' # at t/023_Tokens.t line 71. t/023_Tokens...........................NOK 5/5# Looks like you failed 3 tests of 5. t/023_Tokens...........................dubious Test returned status 3 (wstat 768, 0x300) DIED. FAILED tests 3-5 Failed 3/5 tests, 40.00% okay t/024_Midnight.........................ok t/025_Midnight_Ready...................ok 1/19 |
|
From: Aijaz A. <en...@us...> - 2009-09-20 03:51:18
|
Hello again. You are correct. J3 should not wait for both J2 and J1 to finish. It should run after J2 is done. It could be a bug, or it could be a configuration issue. Could you please tell me which version of taskforest you are using? Also, could you please send me a copy of your config file, especially the value of the 'wait_time' and 'once_only' configuration variables? Thank you, Aijaz. On Sat, Sep 19, 2009 at 10:14 PM, 数字熊 <yan...@gm...> wrote: > hi，Aijaz Ansari > > > I have many job to run everyday, > > i use tokens <http://www.taskforest.com/docs/tokens.html> to limit how > many jobs of a class may run simultaneously. > > > but the performance is poor 。 > > why？ > > > this is the configure file: > +------------------------------------------------------- > 01 |start => '00:00', tz => 'GMT', days => 'Mon,Wed,Fri' > > > 02 | > 03 | J1( token => 'T') J2 ( token => 'T' ) J3(token => 'T') > 04 | > 05 |------------------------------------------------------- > 06 | > 07 | > 08 | > > > 09 | > +------------------------------------------------------- > > <token T> > number = 2 > </token> > > > > the J1 run about 1 hour. and J2 run about 10 min. > the J3 must be wait until the J1 run finish. > > > when it has many job. very job will wait. > > I think J3 can run when J2 is finish. not wait until J1 and J2 finish. > > |
|
From: Aijaz A. <en...@us...> - 2009-09-20 03:42:41
|
Hello.
Yes, waiting for a file to arrive is a very common pre-requisite for a job.
In my experience I have found that it is best to have a separate script
check for the file, and not have the job scheduler like taskforest wait for
the file. There are many things to consider, like:
a) Is the file on a remotely mounted network drive that may not be visible
by taskforest
b) Will the file name change every day?
c) What if the file name is dependent on an external counter, so we don't
know if the file has arrived until we do something like check the file
extension, or even check the contents of the file.
d) Does the user account that runs taskforest have read permission on the
file (or execute permission on the directory)
I have seen all these situations in the recent past, and I am sure there are
more things to consider. It is probably simplest to write a simple perl
script that looks like this pseudocode:
my $file = "filename.txt"
while (current_time < "2300") {
if (file exists) {
exit 0;
}
sleep for 1 minute;
}
exit 1;
Then you could make this small script into its own job, and make that job a
dependency for your real job.
I think this is the safest and most reliable way to do file dependencies.
Would something like this work for you?
Putting this functionality in taskforest in a way that will work across all
environments could make taskforest too complicated.
Aijaz
On Sat, Sep 19, 2009 at 9:05 PM, 数字熊 <yan...@gm...> wrote:
> Hello Aijaz,
>
> We are currently using Taskforest . in some situation。 File arriving is a
> event to run a job.
>
> this Feature is very important for me.
>
>
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry® Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9-12, 2009. Register now!
> http://p.sf.net/sfu/devconf
> _______________________________________________
> Taskforest-discuss mailing list
> Tas...@li...
> https://lists.sourceforge.net/lists/listinfo/taskforest-discuss
>
>
|
|
From: 数字熊 <yan...@gm...> - 2009-09-20 03:14:52
|
hi，Aijaz Ansari I have many job to run everyday, i use tokens <http://www.taskforest.com/docs/tokens.html> to limit how many jobs of a class may run simultaneously. but the performance is poor 。 why？ this is the configure file: +------------------------------------------------------- 01 |start => '00:00', tz => 'GMT', days => 'Mon,Wed,Fri' 02 | 03 | J1( token => 'T') J2 ( token => 'T' ) J3(token => 'T') 04 | 05 |------------------------------------------------------- 06 | 07 | 08 | 09 | +------------------------------------------------------- <token T> number = 2 </token> the J1 run about 1 hour. and J2 run about 10 min. the J3 must be wait until the J1 run finish. when it has many job. very job will wait. I think J3 can run when J2 is finish. not wait until J1 and J2 finish. |
|
From: 数字熊 <yan...@gm...> - 2009-09-20 02:14:04
|
Hello Aijaz, We are currently using Taskforest . in some situation。 File arriving is a event to run a job. this Feature is very important for me. |
|
From: Aijaz A. <en...@us...> - 2009-07-15 21:15:26
|
Teresia: I checked on the dev box, and the issue is fixed in 1.34, not 1.33. 1.34 is still in testing. I found one small change that I need to put in (in new functionality) and I expect that to go to prod in a couple of weeks. I might place a test version on the web site in case anyone wants to test before it's officially released. Thanks, Aijaz. ... > > On Tue, Jul 14, 2009 at 9:08 AM, Teresia Arthur <Ter...@sa...> > wrote: > > I was wondering if the fix is in 1.33 for my previous posting titled > 'failure after rerun in taskforestd'? > > > |
|
From: Teresia A. <Ter...@sa...> - 2009-07-14 14:12:36
|
Thank you so much Aijaz. Teresia Arthur Software Developer 5 Software Quality Tools and Infrastructure Planning, Operations and Strategy SAS Institute From: aij...@gm... [mailto:aij...@gm...] On Behalf Of Aijaz Ansari Sent: Tuesday, July 14, 2009 10:11 AM To: Teresia Arthur; tas...@li... Subject: Re: [Taskforest-discuss] does 1.33 have fix for 'failure after rerun in taskforestd' I'll check and get back to you tonight. I know I have fixed it, but it might be in the version that's still in test. Aijaz. On Tue, Jul 14, 2009 at 9:08 AM, Teresia Arthur <Ter...@sa...<mailto:Ter...@sa...>> wrote: I was wondering if the fix is in 1.33 for my previous posting titled 'failure after rerun in taskforestd'? |
|
From: Aijaz A. <en...@us...> - 2009-07-14 14:11:50
|
I'll check and get back to you tonight. I know I have fixed it, but it might be in the version that's still in test. Aijaz. On Tue, Jul 14, 2009 at 9:08 AM, Teresia Arthur <Ter...@sa...>wrote: > I was wondering if the fix is in 1.33 for my previous posting titled > 'failure after rerun in taskforestd'? > |
|
From: Teresia A. <Ter...@sa...> - 2009-07-14 14:08:59
|
I was wondering if the fix is in 1.33 for my previous posting titled 'failure after rerun in taskforestd'? Thanks, Teresia Arthur Software Developer 5 Software Quality Tools and Infrastructure Planning, Operations and Strategy SAS Institute |
|
From: Aijaz A. <en...@us...> - 2009-06-11 19:28:06
|
Please see my responses below: On Thu, Jun 11, 2009 at 1:56 PM, Rosco Rouse <Ros...@sa...> wrote: > ... > How 'bout the "Rerun" and "Mark Success/Mark Failure" buttons become > checkboxes. Once I've got all of them checked, then click on > "Submit/Go/DoIt" button to cause them all to be executed in one cycle. > Makes sense. I was considering using YUI to make a right-click menu, but I like this method more, because it doesn't rely on Javascript and should even work in (gasp) lynx. One thing to keep in mind is that there probably can't be, and shouldn't be, any guarantee on the order in which the Reruns/Marks happen. Version 1.34 also adds the ability to Hold a job, which would also be in the list, along with "Release". > > Second feature request. How 'bout some buttons that'll filter things based > on status. Click "Failures" to show only those jobs with current status of > "Failure"...that is, exclude the "...Orig_1" failures and show me only the > jobs that failed when they were rerun or those that failed and have *not* > been rerun. Perhaps there are other filter situations that make sense, too; > "Waiting", "Running". Perhaps filter the return code based on some > conditional expression like "> 0" or "255" or "< 8". And why not ask for > the moon...how 'bout a regex that could be applied to the family name and > another regex that could be applied to the job name. These sound good, too. Let me think about how to implement them and allowing for flexibility and ease of use. I'll probably put them in v1.35 and not delay releasing 1.34 because 1.34 has a minor bug fix. If you're willing/able to beta-test 1.35 when I have it ready, please let me know. Thanks, Aijaz. |
|
From: Aijaz A. <en...@us...> - 2009-06-11 19:17:00
|
Rosco: Thanks for the input. Could you please clarify: Is it taskforestd (the web daemon) that goes down or taskforest (the actual engine)? I'm gonna try duplicating this on my dev box. When you rerun the job, can you say for sure that you haven't removed the job that you're trying to rerun from the Family file? I know in all versions out there right now, when you rerun a job that doesn't exist any more, the system doesn't behave gracefully. This has been fixed in version 1.34 (been in testing for about a week). Another user (Teresia) brought this to my attention in a previous email to this list. Is the log directory NFS mounted? There may be a condition where a file may not be visible immediately on an NFS mount. Do you see any errors in your log files (either the taskforest log file or the taskforestd stdout or stderr log files) when this happens? If you do, please forward those errors to me. I'll try to duplicate this on my dev box this evening and get back to you after that. Thanks, Aijaz. On Thu, Jun 11, 2009 at 1:45 PM, Rosco Rouse <Ros...@sa...> wrote: > ... to rerun a job. In the olden days, I'd do this with the batch > "./bin/rerun..." command. Nowadays, I'm clicking the web client "Rerun" > button. All seems well as you issue the command. Then after several > minutes the job doesn't appear to finish when I think it should and I begin > snooping around. I then discover that the ./bin/taskforest process itself > has terminated. > > Sometimes a single, isolated rerun command will cause this...but not very > often. Sometimes, I find it necessary to fire-off several rerun commands > from a ksh...or lately, with the web client I'll click several jobs as quick > as the web page can be rebuilt. The more rerun commands I fire at the > daemon in a short time, the greater the chance that it'll go down. > > Generally, we have very little trouble with the daemon. It performs > reliably in all but this circumstance. > ... > |
|
From: Rosco R. <Ros...@sa...> - 2009-06-11 18:56:23
|
As I get more and more experience with the web client, I've come to feel the need for a feature extension. When I have more than a small handful of jobs that need to be individually rerun...that is, not via "cascade" or "dependents-only"; I have to click on them one at a time. Thus, for each job I have to wait for the command to be accepted, the acceptance page displayed, then click on "Status" and wait for the page to be rebuilt before I can click on the next job. How 'bout the "Rerun" and "Mark Success/Mark Failure" buttons become checkboxes. Once I've got all of them checked, then click on "Submit/Go/DoIt" button to cause them all to be executed in one cycle. Second feature request. How 'bout some buttons that'll filter things based on status. Click "Failures" to show only those jobs with current status of "Failure"...that is, exclude the "...Orig_1" failures and show me only the jobs that failed when they were rerun or those that failed and have *not* been rerun. Perhaps there are other filter situations that make sense, too; "Waiting", "Running". Perhaps filter the return code based on some conditional expression like "> 0" or "255" or "< 8". And why not ask for the moon...how 'bout a regex that could be applied to the family name and another regex that could be applied to the job name. Rosco |
|
From: Rosco R. <Ros...@sa...> - 2009-06-11 18:45:25
|
We're currently running TF 1.26; but I've been noticing this problem since we first began experimenting back at around 1.10 or so. Until recently, I was not convinced that it truly was a bug...thinking it probably something incorrect about our local configuration. Here's the rub. Things will be humming along nicely; our nightly processing will finish with everything OK. Then for some application reason, we'll need to rerun a job. In the olden days, I'd do this with the batch "./bin/rerun..." command. Nowadays, I'm clicking the web client "Rerun" button. All seems well as you issue the command. Then after several minutes the job doesn't appear to finish when I think it should and I begin snooping around. I then discover that the ./bin/taskforest process itself has terminated. Sometimes a single, isolated rerun command will cause this...but not very often. Sometimes, I find it necessary to fire-off several rerun commands from a ksh...or lately, with the web client I'll click several jobs as quick as the web page can be rebuilt. The more rerun commands I fire at the daemon in a short time, the greater the chance that it'll go down. Generally, we have very little trouble with the daemon. It performs reliably in all but this circumstance. Rosco |
|
From: Aijaz A. <en...@us...> - 2009-06-01 17:27:45
|
Lana: FYI - the version that was released today supports this functionality (dependencies between families). Aijaz. On Sat, Apr 18, 2009 at 4:45 PM, Aijaz Ansari <en...@us...>wrote: > Lana: > > Some more comments about calendars and Family dependencies: > > ... > Now for dependencies between families: > > I think I can do what you described by including some syntax that refers to > jobs in other families. If Family F1 has job J1, and family F2 has job J2, > we could make J2 depend on J1 using syntax similar to this in F2: > > F1::J1() > J2() # J2 will only run after J1 in F1 runs. > > This is not supported yet, but it should be pretty easy to implement. It's > a good idea and I'll let you know what I find out in a day or two. > > Thanks, > > Aijaz. > |
|
From: Aijaz A. <en...@us...> - 2009-06-01 03:42:26
|
Hello: Version 1.33 of TaskForest is now available. The changes in this release are: - *Added support for external dependencies (jobs depending on jobs that are in other families)*. - Removed the VERSION string from all the .pm files, and moved it to Makefile.PL. Thils will mean less diffs between versions. - Got rid of unused code. - Improved some test cases, to reduce the risk of false test failures. - Fixed a bug where the last entry of a recurring job was sometimes not run. - Modified taskforestd and taskforestdssl to accept the 'chained' option in the config file. Thanks, Aijaz |
|
From: Aijaz A. <aij...@gm...> - 2009-05-26 19:23:19
|
Thanks for bringing this up, Teresia. I meant to address this several releases ago, but I think this fell through the cracks. The solution here is to support a 'default time zone' setting in the config file. This will be the time zone in which job times are displayed when TaskForest does not know what timezone to use. In the case of logs (as opposed to status), TaskForest does *not* know what time zone to use, because it doesn't necessarily have the version of the Family file as it was when the job ran. For example, if my family's time zone is America/New_York, and I'm viewing logs from a month ago, there is no reason to believe that a month ago the family's time zone was still America/New_York. It could have been GMT. There's just no way to know, because all we know is what the time zone is right now. If we had a default time zone, we could use that when using the 'logs' page. In your case, I think that config variable should be set to "America/New_York." In other words, in the logs page the time zone is used for display purposes only, whereas in the status page the timezone is part of the configuration derived from the header of the family file, which also has an impact on scheduling and deciding what the date is at the current moment in time. I'll make this change and release it in the next code release (tentatively scheduled for June 5th). Thanks, Aijaz. On Tue, May 26, 2009 at 10:19 AM, Teresia Arthur <Ter...@sa...>wrote: > My status page appropriately shows a time zone of America/New_York but the > logs page shows America/Chicago and the times are also off by an hour. It > seems the logs.html page is ignoring the time zone setting. > > Teresia Arthur > Software Developer 5 > Software Quality Tools and Infrastructure > Planning, Operations and Strategy > SAS Institute > > |
|
From: Aijaz A. <en...@us...> - 2009-05-26 19:06:56
|
Let's try out the easy solution first... In your config file, which run wrapper are you using? ".../run" or ".../run_with_log"? You would need the latter to generate the log files that the status page would link to. If you are using run_with_log, could you please confirm that the log files actually exist in the log directory? The log file name would be something like F_GMT.J5.22175.1243316176.stdout for job J5 in family F_GMT Please let me know what you find... Thanks, Aijaz. On Tue, May 26, 2009 at 10:17 AM, Teresia Arthur <Ter...@sa...>wrote: > On my jobs and status pages the status tag never has a link to the log > files for the jobs. It seems the <ifo c=has_output_file> tag is not finding > the log files. I have appropriately set the log_dir in my cfg file. > > Teresia Arthur > Software Developer 5 > Software Quality Tools and Infrastructure > Planning, Operations and Strategy > SAS Institute > > > |
|
From: Teresia A. <Ter...@sa...> - 2009-05-26 15:20:02
|
My status page appropriately shows a time zone of America/New_York but the logs page shows America/Chicago and the times are also off by an hour. It seems the logs.html page is ignoring the time zone setting. Teresia Arthur Software Developer 5 Software Quality Tools and Infrastructure Planning, Operations and Strategy SAS Institute |
