Re: [Queue-developers] Feedback requested on detailed plans and code for contrib project

[Tavis B. <ta...@ma...>]

The code has got some neat features, I just have once oncern.  As it 
stands, the machine running queue_manager cannot be taken down for 
service.  With the old implementation, machines attempting to submit a 
job would eventually give up on a dead machine, so no machine was 
essential. The old code certainly wasn't perfect; if a machine was down 
or even busy, it could take quite a long time for a job to get 
submitted.  But it seemed like it could be improved upon by allowing the 
queue client package to fail out more quickly when trying to reach an 
unresponsive host.  (I understand thsi was in the works for the new 
version; I haven't had a chance to try it yet since it doesn't seem to 
work on Digital Unix, maybe if I have tme I'll try to do some 
debugging.)  

With the new implementation, as I understand it you can't take the machine 
running queue_manager out of service without reconfiguring the whole 
system.  This might be correctable by allowing for failover on the 
queue_manager daemon (e.g., sharing the process database, having 
queue_manager store a lock flag every second that is valid for ten seconds, 
and having queued on a secondary machine to grab the lock and spawn its 
own queue_manager if the primary machine stops renewing its own locks or 
something).  I don't know if this would cause problems interacting with 
license managers.

Anyway, I don't want to disparage what looks overall like some great 
work, I'm just trying to suggest how it might be improved if I understand 
it correctly.

Cheers,
Tavis


On Tue, 12 Sep 2000, W. G. Krebs wrote:

> 
> Although I haven't yet had a chance to look over the code, this seems very
> impressive.
> 
> She includes C/C++ source code, patch files, a detailed plan, and about 12
> slides, which she submitted to the list for our consideration.
> 
> Unfortunately, the listserver does not allow 1 Mb submissions (for obvious
> reasons --- sending 1 Mb email messages to 100s of people is generally not a
> good idea.)
> 
> The document and the slides were in StarOffice format
> (http://www.staroffice.com) which I know not everyone has installed, so I used
> StarOffice to reformat the document in HTML and text. I included versions of
> the slides in HTML and JPG, so everyone can now view them with just a web
> browser. (However, it makes the archive even larger, but this is offset by the
> use of Gzip compression.)
> 
> I've put the document up on our anonymous FTP site for downloading by those of
> you who are interested in viewing the slides or the source code.
> 
> The text file is short enough to be sent out to the list, so I'm attaching it
> here in text format for your convenience (some of the formatting was lost from
> the original StarOffice file; it is better viewed in HTML.)
> 
> The URL for the complete submission is (~600Kb, compressed):
> ftp://queue.sourceforge.net/pub/queue/lau_gnuqueue_submission.tar.gz
> 
> Please send comments to the list.
> 
> Monica Lau wrote:
> 
> >
> >
> >   ------------------------------------------------------------------------
> >
> > Subject: Queue Development Project
> > Date: Mon, 11 Sep 2000 17:24:08 -0700
> > From: Monica Lau <ml...@al...>
> > To: que...@li..., ml...@al...
> >
> > Mr. Krebs,
> >
> > One requested feature of the Queue software has been to keep track of
> > statistics and resource usage on running jobs by using a database as a
> > central repository of job information.  I am a student intern working on
> > a Queue development project that is very similar to this feature.  I
> > have been working on this project for a few months now, and I have coded
> > the basic structure.  We have many other features that we would like to
> > incorporate, such as using a GNU SQL database as mentioned above.  This
> > feature would not be hard to incorporate in our project.
> >
> > Attached is a tar file that includes the Queue programs that I've
> > modified, the programs that I've written to incorporate in the Queue
> > package, and the documentation about our project (which includes
> > diagrams and text, both done in StarOffice).  All of my programs are
> > written in C++ (we can modify the code if the programs must strictly be
> > written in C).
> >
> > We would really appreciate it if you can give us feedback on this.  When
> > you have time, please read the documentation (just the "overview"
> > section will be fine) and look over the diagrams to get an idea of the
> > architecture that we have in mind.
> >
> > Thank you for your time.
> >
> > Regards,
> > Monica Lau
> >
> >   ------------------------------------------------------------------------
> >                                Name: queue_patches.tar
> >    queue_patches.tar           Type: Unix Tape Archive (application/x-tar)
> >                            Encoding: base64
> >                     Download Status: Not downloaded with message
>