Re: [Queue-developers] new design details

[wernerkrebs <wer...@ya...>]

--- Koni <mh...@co...> wrote:

> On Wed, 2005-05-11 at 22:01 -0700, wernerkrebs
> wrote:
> > Progamming is partially an artform as much as a
> > science, and it's usually best to try to use the
> most
> > modern techniques that everyone else is using.
> This
> > way, you can leverage off what other people in the
> > community are doing, and develop synergy with
> other
> > projects. GQ was modern for its time, but lots has
> > happened since it came out.
> > 
> 
> I agree here in principle, but some things that
> might be called modern
> techniques are really just trendy to me. 
> 
> I think we (we as in all interested parties on this
> list) first need to
> sort out a couple critical items:
> 
> First, do we wish or intend to subscribe firmly to
> the RMS point of view
> of free software for this project? If GNU Queue
> remains "GNU", then I
> think this is most relevant albeit restrictive. A
> firm commitment to
> that would limit our discussions about mysql or high
> performance
> commercial database backends and use of Java or
> C#/Mono. 

Up to you. >;->

> Second, I think we should define a limited scope of
> problems that GQ
> will purport to solve and do that well. I am
> confused by the idea of
> meta-clustering. It seems like it won't be much work
> on us to have some
> meta-controller interface with GQ's controller, but
> why do people do
> this? 

Lack of centralized administrative control of the
different clusters, which are spread out at different
institutions. 

Computational biology (specifically simulation and
modelling of protein molecules for rational drug
design) in support of the multi-billion dollar
pharmaceutical R&D budget is the current "killer app"
for "the grid", in part because the ageing population
of affulent countries is seen as creating the business
model to sell the next generation of supercomputers.
(GQ was originally written to help in econometrics
problems, but I apparently moved in comp. bio. with
the rest of the supercomputer market as I underwent my
doctoral studies.)

"The Grid" is the current buzzword that describes what
GQ does (aka "clustering", but on a wide-area,
multi-institution scale). You can read books on
Amazon.com on it; I know some of the authors
personally. It's a word that didn't even exist when GQ
was first released.

The idea is that multiple institutions in different
countries across the globe would loan their clusters
to form a meta-cluster, connected by a dedicated
TeraBit backbone network currently being built. Time
on this meta cluster might then be rented to
pharmaceutical companies. There is very serious
government money in this area. (It's also an area that
your institution, Cornell, has been heavily involved
in, and has it's dedicated own Linux clusters. The
person at Cornell is Prof. Harold Scheraga, whom I've
had the privilege of meeting.)

Your analysis may be dead on: it might not be the
thing for GQ, which maybe should focus on small
clusters and easy installation.

Still, it's something to keep in mind. If you want to
write a GQ manual someday, it will sell a lot more
copies if you advertise GQ as facilitating "The Grid"
or "The TeraGrid." Similarily, the GQ website would
pick up a lot more traffic if we added "Grid
Applications" or "TeraGrid Applications" somewhere on
the webpage. "Clustering" is the old-school term, I'm
afraid. >:->

> I can think of a few reasons, but are the
> needs of these
> applications fitting within the problem space we
> want to address?

Don't know; your call.

> I see a growing market of small dedicated
> homogeneous systems with
> simple distribution needs, driven by cheap computer
> prices. In time up
> to present or perhaps into the future some more,
> interest heterogeneous
> and non-dedicated systems I think is coming from
> interest in tapping the
> resource of idle desktop CPUs. With that comes
> substantial complexity
> that other projects like condor have embraced and
> seem to be doing
> well. 
> 
> I don't think its beneficial to GQ to try to tackle
> some of the same
> problems if condor already fills that niche. I also
> think many groups
> with new interest in this area will have small
> dedicated systems and be
> looking for something simple and easy. Thus, this
> seems like an
> expanding niche that would be well filled by
> something that takes 2
> minutes to setup and start using. GQ can do this,
> especially if it is
> stand-alone with no backend database and no library
> dependencies.  
> 
> I think the ease of setup and use will make a big
> difference in the
> adoption of GQ as a solution in the situations where
> it is applicable. 

Sounds good. >:->

> 
> 
> 
> 
>
-------------------------------------------------------
> This SF.Net email is sponsored by Oracle Space
> Sweepstakes
> Want to be the first software developer in space?
> Enter now for the Oracle Space Sweepstakes!
>
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
> _______________________________________________
> Queue-developers mailing list
> Que...@li...
> To unsubscribe, subscribe, or set options:
>
https://lists.sourceforge.net/lists/listinfo/queue-developers
>