[Taskforest-discuss] Fwd: Looking for TaskForest USERS!!!

[Aijaz A. <en...@us...>]

Forwarded to the Taskforest-discuss mailing list with the original author's
permission:

---------- Forwarded message ----------
From: Aijaz Ansari <en...@us...>
Date: Fri, Apr 17, 2009 at 1:48 PM
Subject: Re: [Taskforest-discuss] Looking for TaskForest USERS!!!
To: Svetlana Lemeshov <lem...@mi...>


Lana:

Thanks for your email.  These are excellent points.  I'll give you a quick
response now (as I'll be leaving my desk soon), and a more detailed one
tomorrow.

In a nutshell -

a) You're right about the midnight issue.  I had some ideas about it, and
was discussing it with a user, but haven't heard back from him.  I'd love to
discuss this with you. I'll send you a more detailed response about this
tonight.

b) This brings us back to another of your questions.  Yes, the system
changes quite often, but every effort has been taken to keep version n+1
backward compatible with version n.  The only exception I can think of is
the --collapse option, which determines how status is displayed.   I try to
ensure that new development does not break any of the old test cases.

c) Arbitrary job calendars is in the works.  I know how it's going to be
implemented and am waiting to complete the current version that deals with
tokens before working on calendars.  The main feature timetable is as
follows: Tokens (currently in dev), Calendars, and then support for multiple
machines.   The multiple machine support might be version 2.0.

d) Dependencies between families - honestly, I hadn't even thought of that.
Your examples make sense, but there are some important design decisions that
will need to be made - where each option has it's own merits.  Because of
issues like this, I prefer not making a decision before speaking to users
about it.  Their needs may cause one option to outweigh the other.

e) I'm taking the development a little slower than I'd like because I really
want to speak to users before making major design decisions.  I use
TaskForest in Production every day on multiple machines, and it meets my
needs.  I won't use some of the new features as much as I use the older
ones, so I consider the input from users such as yourself (who rely on these
features) as having more weight than my own input.  Unfortunately, even
though TaskForest is apparently being used by a lot of people, you are only
the third person to email me about it in the last 12 months.   So getting a
dialog with users is kinda tough :)

I'll send you some a more detailed email tomorrow morning.  In the meantime,
is it OK if I forward your email, and this response and future responses to
the mailing list?  It might help other users decide whether or not
TaskForest is appropriate for them.  If not, that's cool too.

Thanks,

Aijaz.



On Fri, Apr 17, 2009 at 12:45 PM, Svetlana Lemeshov <lem...@mi...>wrote:

> Hello Aijaz,
>
> Thank you very much for promptly responding to my inquiry!
>
> I would love to hear feedback from your users.
>
> Meanwhile, if you don't mind, I have a couple of questions for you
> since I couldn't find any forum where I can post my questions and have them
> answered by other users. In addition, I thought you might be interested
> to hear the feedback from the potential users of your product.
>
> Here is some background information:
> I am a Data Warehouse developer/analyst designated to explore and analyze
> various scheduling products available on the market today with the goal of
> migrating our jobs to a more flexible scheduler. I've installed TaskForest
> and spent some time exploring its features trying to figure out how well
> it can handle various DW specific needs.
>
> MIT Data warehouse consists of data coming from the various source systems:
> Hr, Financial, Libraries, Student Systems, Facilities, etc.
>
> Our data gets refreshed nightly, starting after 10:00PM and finishing
> around
> 7:30 AM. The refresh consists of numerous extraction and load processes and
> has
> many inter--dependencies between them.
>
> For instance, many financial loads depend on HR loads, etc.
>
> While exploring TaskForest I stumbled on the following obvious limitations:
>
> - Handling a 'midnight' point;
> I restart taskforest at 00:01 from cron file, but since our loads run all
> night
> long, we need to come up with some custom scripts to handle scripts that
> run
> 'across' midnight point.
> Specifically, the situation when a job that runs after midnight depends on
> the
> status of the one that finished before the midnight.
>
> -  TaskForest Schedule based only on the days of the week;
> Our current cron utility allows for various scheduling options (by months,
> day of the month, etc.) Since TaskForest is only Days-of-the-Week driven,
> and some of our jobs happen to run only on certain days of the month
> (first of each month, or once in a quarter), we will have to handle this
> with the custom scripts.
>
> - Absence of dependencies between the Families;
> There is no mechanism that would allow to have a Family within a Family to
> handle Jobs that obviously belong to the same family, but have slightly
> different execution schedules.
>
> Sometimes, we can have a situation when jobs that obviously belong to the
> same
> family (same subject area, lots of dependencies between them) run on a
> different days of the week (Ex. Job A runs everyday, but it depends on the
> successful completion of Job B which runs only on weekdays).
>
> If I understand it correctly, in current TaskForest setting, this scenario
> calls
> for 2 different families. However, it seems that there is no mechanism to
> establish dependencies between 2 families. So, it should be handled through
> custom scripts like many other things (handling Holidays, etc)
>
> Overall, it seems that our application will require a lot of custom scripts
> to
> help TaskForest handle our jobs.
>
> One more observation. It seems that the product is still evolving and every
> time
> I check there is a new version of it. Since I started working
> with the tool a few months ago, I've already upgraded it once, but I am
> still
> about 2 versions behind. I am concerned about the on-going maintenance and
> support of the product and possible frequent modifications to the scheduler
> once it is implemented in production (example: if/when a new feature like
> calendar is implemented, it might require some modifications on our site.)
>
> Aijaz, I am sorry for my lengthy rambling, but these are some of the issues
> I am
> running into. I am not sure whether it is due to the fact that I just fail
> to
> use the product correctly, or these are real limitations.
>
> I was hoping to discuss these issues with other users, but, on the other
> hand,
> who can address these points better than the author of TaskForest?
>
> I am giving a TaskForest presentation to my team on Wednesday, April 22
> and will be covering many of my 'findings'.
>
> Aijaz, it would be really very helpful if you could clarify some of the
> points
> mentioned above! I appreciate your time and attention!
>
> Thank you very much!
>
> Lana Lemeshov
>
>
>
>
>
> Quoting Aijaz Ansari <en...@us...>:
>
>  Svetlana:
>>
>> I'm the author of TaskForest.  Since I'm obviously biased, I won't give
>> you
>> my unsolicited opinion :), but feel free to ask me any questions - I'll be
>> glad to answer them.  I'm also taking feature requests.  Many of the
>> recent
>> features are a result of user emails.
>>
>> ...
>>
>> Thanks,
>>
>> Aijaz.
>>
>> On Thu, Apr 16, 2009 at 3:53 PM, Svetlana Lemeshov <lem...@mi...>
>> wrote:
>>
>>  Hello,
>>>
>>> My name is Lana Lemeshov, I work for MIT DataWarehouse.
>>> We are currently using UNIX cron utility for job scheduling, however
>>> we would like to switch to a more flexible product.
>>>
>>> We are looking into a couple of schedulers and TaskForest is one of
>>> the strong contenders (with Oracle 10g Scheduler as an alternative.)
>>>
>>> I am trying to find someone who has any experience using TaskForest and
>>> wouldn't mind sharing the feedback us.
>>> MIT Data Warehouse would love to hear honest reviews of the product
>>> from the user's point of view: specific problems, limitations, support
>>> issues,
>>> etc.
>>>
>>> We are interested to compare how TaskForest stands against the Oracle
>>> Sheduler.
>>>
>>> ANY feedback is greatly appreciated.
>>>
>>> Thank you very much!
>>>
>>> Lana
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Stay on top of everything new and different, both inside and
>>> around Java (TM) technology - register by April 22, and save
>>> $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
>>> 300 plus technical and hands-on sessions. Register today.
>>> Use priority code J9JMT32. http://p.sf.net/sfu/p
>>> _______________________________________________
>>> Taskforest-discuss mailing list
>>> Tas...@li...
>>> https://lists.sourceforge.net/lists/listinfo/taskforest-discuss
>>>
>>>
>>
>
>