[Queue-developers] Re: Gnu queue
Brought to you by:
wkrebs
From: W. G. K. <wer...@ya...> - 2000-10-09 16:00:15
|
Hi Ole, Quoting Ole W Saastad <ole...@ch...>: > > Hi, > first of all I would like to thank you for a good program. > > I have installed it on a RH 6.2 box that will run quantum chemical > calculation. At such a machine it is important to have a kind of batch > processing system. Queue does this nicely. > > However, > there are no commands to list the queues and no commands to > remove jobs from the queues. > I have written a simple script that cats the queuestat file and counts > the number of files in the CFDIR in order to print at least some info > about the queue. Please contribute it to que...@li.... If it a longer script (more than, say, 10 or 20 lines) make it copyright yourself and say that it is GPL'd somewhere in the comments. Thanks! > I would like to have some tools for manipulation > of the queues. Like lq (list queue) and rmq (remove from queue). > lq -d Gaussian which should produce some information about the > queue named Gaussian. And rmq -d Gaussian JobID which deletes > the JobID from the queue, the option -f would terminate Job ID if > it were running. Otherwise rmq -d Gaussian JobID will not stop > currently running jobs. I think Monica Lau is supposedly working on this. She submitted a set of C/C++ patches for GNU Queue that sets up a central server and may include additional tools. The patches and other stuff she submitted are available for downloading from Sourceforge; see the archives of queue-developers on www.gnuqueue.org to find out where this is. I would like to incorporate her patches into the next release of GNU Queue, or at least put them on the CVS archive so that people can begin hacking on them and the resulting parallelization will speed up development. I'm trying to get her to either release the patches under her company's copyright (as GPL'd --- this is the fastest way) but I made the mistake of also suggesting the standard GNU/FSF spiel and suggested she consider transfering the copyright to the FSF. Of course she did this, and now presumably the whole thing is tied up in her company's legal department. (It may also be the FSF got the forms from her company and hasn't told me yet.) Right now, the program is in copyright limbo; she's distributed it to the net, but it's not clear who has copyright or whether it is GPL'd. And so it goes.... In any event, as someone suggested a few weeks ago in developers, Queue's tables could be put into SQL database, which could then be used as the central server for the system. The scripts to manipulate Queue would then simply be things like PHP3 or Perl DBI scripts that would interact with the database and control variables in table there. Counting the number of jobs running globally would then be a simple SQL statement. I've been reluctant to move in this direction because it would make GNU Queue less reliable; if the database fails, the whole system goes down. Also, the Internet RFC people want the protocol to be Internet-wide scalable (or, we have to explictly advise people that it is not scalable and indicate what circumstances it would be appropriate to use GNU Queue.) If we stipulate central server, then protocol is inherently not scalable. (Currently, it can easily be made scalable with slight modifications to the host searching protocol, although you'd want cryptographic cookies before doing this.) I'll probably leave the current mechanism in tact; that will take care of Internet-wide scalability and also the folks that don't want to go through the trouble of setting up a SQL database. But, I'll probably make Queue try to look for a SQL database (at least during configuration) and use that if it's available. The tools would then include PHP3 scripts to manipulate the SQL database (probably MySQL.) That would leave the issue of including a bunch of RPMS for configuring phplib to run with MySQL --- the IMP package currently includes a bunch of RPMs like this (http://www.horde.org.), and it can be a pain to set up, but goes relatively quickly if you are running the latest version of RedHat. > I would also like to know the syntax of the options in the profile > file. How do I specify a maximum of 15 min CPU time. A list of > examples should be included in the manual. Apart from this the manual > was good. You put an "rlimitcpu time" as a line in the profile for the batch queue, where time is the time in minutes. This uses the rlimit() kernel facility. The other way to do this originally was to place a "setlimit cpu time" statement in a csh/tcsh script; this set the kernel rlimit(), which then got passed to Queue. However, I think the code that does this is commented out now because it caused problems on Solaris and some other systems. All kernels have such anr limit() facility, but not all of them use it --- HP-UX, for example, ignores this limit for cpu. You can set it, but the kernel never stops the program. There's a package copyright Uwaterloo that's not GPL that's available for download that fixes this problem under HPUX. This is the #ifdef HAVE_UWATERLOO directive in queued.c . Hope this helps. > > > Best regards > Ole > > > > -- > Ole W Saastad, Dr.Scient. Tel. 22067852 > SINTEF Kjemi. PB 124 Blindern > 0314 Oslo > > |