Re: [Clockwork-developers] Architecture of job scheduler

[Shawn D. <sd...@cf...>]

I agree about NTP.  No sense re-inventing the wheel.  The folks who devised
NTP were undoubtedly pretty bright so we won't make any substantial
improvement, and the environment for our scheduler will likely already have
NTP configured.

If we decentralize the schedule (which I do favor) then we have a couple of
choices in terms of providing a central view.  $Universe does use a central
master server that keeps track of jobs on all systems, but if I remember
correctly it is passive;  it waits to be notified by the clients so little
or no polling occurs.  In this model you won't have an up-to-the-second view
of the state of the schedule, though you could make it pretty close by
adjusting how often updates are sent to the master.  Another way would be to
actively poll all the servers from the master, but this will probably cause
substantial processor and network overhead.  The polling wouldn't have to be
continual, only occurring when someone is viewing the status through the
GUI, but many shops will probably want the GUI open all of the time, so
you'd be taking the hit all the time.

If we go to a centralized schedule then we have easy access to the current
state.  However, AutoSys illustrates the trade-offs with sort of
configuration, and I don't know if we want to accept the heavy compute
requirements and single-point failure possibilities that this entails.  If
we do go with a centralized design then we need to include elements in the
design to ensure that the scheduler will cleanly failover or, better yet,
support some sort of hot-standby.

Shawn


----- Original Message -----
From: "Joel Loudermilk" <jo...@lo...>
To: <clo...@li...>
Sent: Saturday, January 12, 2002 10:53 AM
Subject: Re: [Clockwork-developers] Architecture of job scheduler


>
> +- On Friday (1/11/2002 14:4) "Charles Brian Hill" <br...@do...>
Wrote-
> | A valid question may be whether our tool should be responsible for time
> | synchronization.  The users could always use NTP to keep the clocks on
their
> | systems synchronized (to a certain degree).
>
> Anything we would build to syhcnronize time would probably not be as good
> as NTP, because we've never tried that before and because it's just one
> component of our software.
>
> When I think of the intended users of this software, I see people whose
> scheduling needs have outgrown cron, either because they have too many
> systems or too complicated a schedule, or both. In a medium to large
network
> like that, I think it's safe to assume that the administrators have
already
> taken care of syhcnronizing the time.
>
> | If we choose to use
> | a full SQL database, we have two possible routes:  either choose a
database
> | that everyone will have to do, or decide to support multiple database
> | platforms.
>
> I'm very much in favor of supporting multiple platforms. Even AutoSys can
> support at least Sybase and Oracle (and I don't know how much more).
>
> | If I wanted to use a full-fledged database, I would want to take
advantage
> | of some of the more advanced features of the database engine (let's say
we
> | had one that supports transactions, replication, triggers, etc).  To use
> | those features we'd need to go with a single database platform.  Trying
to
> | use, for example, JDBC, and let the user choose the database platform
would
> | mean that we wouldn't be able to use features that aren't commonly
> | supported.
>
> JDBC does support transactions, and aren't replication and triggers things
> that happen "behind the scenes" that we wouldn't be controlling through
> the database's API?
>
> If so, what about supporting any database for which the user can find a
> JDBC driver and that supports transactions and replication (and whatever
> other features we need to use). To set up the database, the user might
> have to do some database-specific stuff, since defining replication
probably
> varies across database platforms, but that's just one-time setup, and we
> could probably even include scripts for the most popular databases.
>
> But I suppose all this is irrelevant if we decide to decentralize the
> scheduler, since we would want to distribute the database as well. In that
> design, does anybody have an idea how a GUI monitoring tool would be able
> to see the current state of the schedule? Wouldn't it have to poll lots
> and lots of systems?
>
> --
> Joel
>
> _______________________________________________
> Clockwork-developers mailing list
> Clo...@li...
> https://lists.sourceforge.net/lists/listinfo/clockwork-developers