queue-developers

Re: [Queue-developers] Feedback requested on detailed plans and code for contrib project

[Monica L. <ml...@al...>]

Thank you for the feedback!  Currently, if the queue_manager crashes,
then all the jobs that have been submitted will be lost because the
queue_manager keeps track of this information within its internal data
structures.  However, the queue daemons will be unaware that the
queue_manager has crashed and will continue to run the jobs they are
currently running; they simply won't be able to connect to the
queue_manager.  When the queue_manager starts back up on the same
machine again, then everything resumes its usual course (users would
have to resubmit their jobs).  On the other hand, if you want to start
the queue_manager on another machine, then you would have to reconfigure
the whole system.  We can fix this problem by putting the server name
(that runs the queue_manager) in a file, and right before the queue
daemons connect to the queue_manager, they will read from this file and
connect to this particular server.  

Someone also brought up the idea of a back-up queue_manager.  There
would be a master queue_manager and a slave queue_manager.  The slave
queue_manager has all the information that the master queue_manager has,
so if the slave ever detects that the master has failed, it will take
over the master's role until the master starts back up again.  Is this
similar to what you have in mind?

Monica


Tavis Barr wrote:
> 
> The code has got some neat features, I just have once oncern.  As it
> stands, the machine running queue_manager cannot be taken down for
> service.  With the old implementation, machines attempting to submit a
> job would eventually give up on a dead machine, so no machine was
> essential. The old code certainly wasn't perfect; if a machine was down
> or even busy, it could take quite a long time for a job to get
> submitted.  But it seemed like it could be improved upon by allowing the
> queue client package to fail out more quickly when trying to reach an
> unresponsive host.  (I understand thsi was in the works for the new
> version; I haven't had a chance to try it yet since it doesn't seem to
> work on Digital Unix, maybe if I have tme I'll try to do some
> debugging.)
> 
> With the new implementation, as I understand it you can't take the machine
> running queue_manager out of service without reconfiguring the whole
> system.  This might be correctable by allowing for failover on the
> queue_manager daemon (e.g., sharing the process database, having
> queue_manager store a lock flag every second that is valid for ten seconds,
> and having queued on a secondary machine to grab the lock and spawn its
> own queue_manager if the primary machine stops renewing its own locks or
> something).  I don't know if this would cause problems interacting with
> license managers.
> 
> Anyway, I don't want to disparage what looks overall like some great
> work, I'm just trying to suggest how it might be improved if I understand
> it correctly.
> 
> Cheers,
> Tavis

Re: [Queue-developers] Feedback requested on detailed plans and code for contrib project

[Tavis B. <ta...@ma...>]

The code has got some neat features, I just have once oncern.  As it 
stands, the machine running queue_manager cannot be taken down for 
service.  With the old implementation, machines attempting to submit a 
job would eventually give up on a dead machine, so no machine was 
essential. The old code certainly wasn't perfect; if a machine was down 
or even busy, it could take quite a long time for a job to get 
submitted.  But it seemed like it could be improved upon by allowing the 
queue client package to fail out more quickly when trying to reach an 
unresponsive host.  (I understand thsi was in the works for the new 
version; I haven't had a chance to try it yet since it doesn't seem to 
work on Digital Unix, maybe if I have tme I'll try to do some 
debugging.)  

With the new implementation, as I understand it you can't take the machine 
running queue_manager out of service without reconfiguring the whole 
system.  This might be correctable by allowing for failover on the 
queue_manager daemon (e.g., sharing the process database, having 
queue_manager store a lock flag every second that is valid for ten seconds, 
and having queued on a secondary machine to grab the lock and spawn its 
own queue_manager if the primary machine stops renewing its own locks or 
something).  I don't know if this would cause problems interacting with 
license managers.

Anyway, I don't want to disparage what looks overall like some great 
work, I'm just trying to suggest how it might be improved if I understand 
it correctly.

Cheers,
Tavis


On Tue, 12 Sep 2000, W. G. Krebs wrote:

> 
> Although I haven't yet had a chance to look over the code, this seems very
> impressive.
> 
> She includes C/C++ source code, patch files, a detailed plan, and about 12
> slides, which she submitted to the list for our consideration.
> 
> Unfortunately, the listserver does not allow 1 Mb submissions (for obvious
> reasons --- sending 1 Mb email messages to 100s of people is generally not a
> good idea.)
> 
> The document and the slides were in StarOffice format
> (http://www.staroffice.com) which I know not everyone has installed, so I used
> StarOffice to reformat the document in HTML and text. I included versions of
> the slides in HTML and JPG, so everyone can now view them with just a web
> browser. (However, it makes the archive even larger, but this is offset by the
> use of Gzip compression.)
> 
> I've put the document up on our anonymous FTP site for downloading by those of
> you who are interested in viewing the slides or the source code.
> 
> The text file is short enough to be sent out to the list, so I'm attaching it
> here in text format for your convenience (some of the formatting was lost from
> the original StarOffice file; it is better viewed in HTML.)
> 
> The URL for the complete submission is (~600Kb, compressed):
> ftp://queue.sourceforge.net/pub/queue/lau_gnuqueue_submission.tar.gz
> 
> Please send comments to the list.
> 
> Monica Lau wrote:
> 
> >
> >
> >   ------------------------------------------------------------------------
> >
> > Subject: Queue Development Project
> > Date: Mon, 11 Sep 2000 17:24:08 -0700
> > From: Monica Lau <ml...@al...>
> > To: que...@li..., ml...@al...
> >
> > Mr. Krebs,
> >
> > One requested feature of the Queue software has been to keep track of
> > statistics and resource usage on running jobs by using a database as a
> > central repository of job information.  I am a student intern working on
> > a Queue development project that is very similar to this feature.  I
> > have been working on this project for a few months now, and I have coded
> > the basic structure.  We have many other features that we would like to
> > incorporate, such as using a GNU SQL database as mentioned above.  This
> > feature would not be hard to incorporate in our project.
> >
> > Attached is a tar file that includes the Queue programs that I've
> > modified, the programs that I've written to incorporate in the Queue
> > package, and the documentation about our project (which includes
> > diagrams and text, both done in StarOffice).  All of my programs are
> > written in C++ (we can modify the code if the programs must strictly be
> > written in C).
> >
> > We would really appreciate it if you can give us feedback on this.  When
> > you have time, please read the documentation (just the "overview"
> > section will be fine) and look over the diagrams to get an idea of the
> > architecture that we have in mind.
> >
> > Thank you for your time.
> >
> > Regards,
> > Monica Lau
> >
> >   ------------------------------------------------------------------------
> >                                Name: queue_patches.tar
> >    queue_patches.tar           Type: Unix Tape Archive (application/x-tar)
> >                            Encoding: base64
> >                     Download Status: Not downloaded with message
> 

[Queue-developers] Feedback requested on detailed plans and code for contrib project

[W. G. K. <wer...@ya...>]

Although I haven't yet had a chance to look over the code, this seems very
impressive.

She includes C/C++ source code, patch files, a detailed plan, and about 12
slides, which she submitted to the list for our consideration.

Unfortunately, the listserver does not allow 1 Mb submissions (for obvious
reasons --- sending 1 Mb email messages to 100s of people is generally not a
good idea.)

The document and the slides were in StarOffice format
(http://www.staroffice.com) which I know not everyone has installed, so I used
StarOffice to reformat the document in HTML and text. I included versions of
the slides in HTML and JPG, so everyone can now view them with just a web
browser. (However, it makes the archive even larger, but this is offset by the
use of Gzip compression.)

I've put the document up on our anonymous FTP site for downloading by those of
you who are interested in viewing the slides or the source code.

The text file is short enough to be sent out to the list, so I'm attaching it
here in text format for your convenience (some of the formatting was lost from
the original StarOffice file; it is better viewed in HTML.)

The URL for the complete submission is (~600Kb, compressed):
ftp://queue.sourceforge.net/pub/queue/lau_gnuqueue_submission.tar.gz

Please send comments to the list.

Monica Lau wrote:

>
>
>   ------------------------------------------------------------------------
>
> Subject: Queue Development Project
> Date: Mon, 11 Sep 2000 17:24:08 -0700
> From: Monica Lau <ml...@al...>
> To: que...@li..., ml...@al...
>
> Mr. Krebs,
>
> One requested feature of the Queue software has been to keep track of
> statistics and resource usage on running jobs by using a database as a
> central repository of job information.  I am a student intern working on
> a Queue development project that is very similar to this feature.  I
> have been working on this project for a few months now, and I have coded
> the basic structure.  We have many other features that we would like to
> incorporate, such as using a GNU SQL database as mentioned above.  This
> feature would not be hard to incorporate in our project.
>
> Attached is a tar file that includes the Queue programs that I've
> modified, the programs that I've written to incorporate in the Queue
> package, and the documentation about our project (which includes
> diagrams and text, both done in StarOffice).  All of my programs are
> written in C++ (we can modify the code if the programs must strictly be
> written in C).
>
> We would really appreciate it if you can give us feedback on this.  When
> you have time, please read the documentation (just the "overview"
> section will be fine) and look over the diagrams to get an idea of the
> architecture that we have in mind.
>
> Thank you for your time.
>
> Regards,
> Monica Lau
>
>   ------------------------------------------------------------------------
>                                Name: queue_patches.tar
>    queue_patches.tar           Type: Unix Tape Archive (application/x-tar)
>                            Encoding: base64
>                     Download Status: Not downloaded with message

[Queue-developers] Queue 1.20.2 released.

[Werner G. K. <wer...@ya...>]

Numerous bug fixes, including:

                                  - fix for deadlock bug
                                  - preliminary support for hetergenous
clusters, as we move closer to draft protocol.

Re: [Queue-developers] IRIX distress

[W. G. K. <wer...@ya...>]

Sorry for the delay; I've been out of town and away from email.

I've compiled earlier versions of Queue under IRIX.

First off, IRIX/Linux will not work (yet) because we need to make things
internet draft compliant. (You could download stuff off the cvs repository
(see instructions in an earlier post) to obtain CVS stuff, which is much
closer to internet draft complaince and allows mixed Sun/Linux environment.
It may also allow a mixed IRIX/Linux environment. But 1.20.1 alone will not.

It looks like IRIX libxnet and libnsl are incompatible with each other. I've
seen this before working on another project on IRIX. Include only one
library in the compilation. (./configure is probably selecting both for you
automatically; you may need to edit the Makefile and engaging in other
gynamistics to forcibly ensure only one of the libraries is included). Also
try forcing compilation in 64 bit mode and see what happens (libxnet and
libnsl in 64 bit mode compilation may well be compatible. This assumes, of
course, that all of your machines are 64 bit ready, which they may not be.)

Buckley Collum wrote:

> Hello all.
>
> I am trying to get Queue working in a mixed IRIX/Linux network and am
> looking for help making the IRIX build.
> I followed the instructions, but it is having too many warnings/errors,
> and fails.
>
> I tried both '--enable-root' and single user with comparable results.
>
> IRIX's 'make' complains:
> make: file `Makefile' line 336: Syntax error
> and line 336 reads:
> DEPS_MAGIC := $(shell mkdir .deps > /dev/null 2>&1 || :)
>
> But it was noted in the INSTALL that 'make' might not work, so it was
> obvious to use 'gmake'.  But...:
> <atlas:buckley> gmake
> gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c queue.c
> queue.c: In function `main':
> queue.c:883: warning: assignment makes pointer from integer without a
> cast
> queue.c:923: warning: assignment makes pointer from integer without a
> cast
> queue.c:1266: warning: comparison is always 0 due to limited range of
> data type
> gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c wakeup.c
> wakeup.c: In function `con_daemon':
> wakeup.c:179: warning: assignment makes pointer from integer without a
> cast
> wakeup.c:221: warning: assignment makes pointer from integer without a
> cast
> wakeup.c: In function `wakeup':
> wakeup.c:347: warning: passing arg 4 of `qsort' from incompatible
> pointer type
> gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c ident.c
> gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c qlib.c
> gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c sha1.c
> In file included from util.h:23,
>                  from sha1.c:41:
> types.h:52: warning: redefinition of `ushort'
> /usr/include/sys/bsd_types.h:31: warning: `ushort' previously declared
> here
> types.h:58: warning: redefinition of `ulong'
> /usr/include/sys/bsd_types.h:35: warning: `ulong' previously declared
> here
> gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c getloadavg.c
> gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c setenv.c
> gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c getopt_long.c
> gcc  -g -O2  -o queue  queue.o wakeup.o ident.o qlib.o sha1.o
> getloadavg.o setenv.o getopt_long.o -lelf -lcrypt -lxnet -lnsl -lsocket
> -lelf -lrpcsvc
> ld32: WARNING 84: /usr/lib32/libelf.a is not used for resolving any
> symbol.
> ld32: WARNING 84: /usr/lib32/libcrypt.so is not used for resolving any
> symbol.
> ld32: WARNING 84: /usr/lib32/libxnet.so is not used for resolving any
> symbol.
> ld32: WARNING 85: definition of _getkey in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of getkey in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of cs_connect in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _cs_connect in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of read_status in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of cs_perror in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _cs_perror in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of write_dialrequest in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of cbc_crypt in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of ecb_crypt in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of des_setparity in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of __des_crypt in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of netdir_free in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of netdir_sperror in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of netdir_perror in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _netdir_perror in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of setnetconfig in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of endnetconfig in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of getnetconfig in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of getnetconfigent in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of freenetconfigent in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of setnetpath in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of endnetpath in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of getnetpath in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of nc_sperror in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _nc_perror in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of nc_perror in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_accept in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_bind in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_close in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_connect in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_error in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _t_error in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_getinfo in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_getname in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_getstate in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_listen in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_look in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_open in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _t_optmgmt in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_rcv in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _t_rcvconnect in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_rcvconnect in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_rcvdis in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _t_rcvrel in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_rcvrel in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_rcvudata in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: Giving up after printing 50 warnings.  Use -wall to print all
> warnings.
> gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c queued.c
> queued.c: In function `transmitjob':
> queued.c:3053: warning: comparison is always 1 due to limited range of
> data type
> queued.c: In function `check_query':
> queued.c:4268: warning: assignment makes pointer from integer without a
> cast
> queued.c:4300: warning: assignment makes pointer from integer without a
> cast
> gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c lex.c
> gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c handle.c
> handle.c: In function `chldsigh':
> handle.c:119: warning: comparison is always 0 due to limited range of
> data type
> handle.c: In function `handle':
> handle.c:656: warning: assignment makes pointer from integer without a
> cast
> handle.c:688: warning: assignment makes pointer from integer without a
> cast
> handle.c:896: warning: passing arg 1 of `open' makes pointer from
> integer without a cast
> gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c pty.c
> pty.c: In function `allocpty':
> pty.c:64: warning: assignment makes pointer from integer without a cast
> gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c mrestart.c
> gcc  -g -O2  -o queued  queued.o lex.o handle.o ident.o pty.o qlib.o
> mrestart.o wakeup.o sha1.o getloadavg.o setenv.o getopt_long.o -lelf
> -lcrypt -lxnet -lnsl -lsocket -lelf -lrpcsvc
> ld32: WARNING 84: /usr/lib32/libelf.a is not used for resolving any
> symbol.
> ld32: WARNING 84: /usr/lib32/libcrypt.so is not used for resolving any
> symbol.
> ld32: WARNING 84: /usr/lib32/libxnet.so is not used for resolving any
> symbol.
> ld32: WARNING 85: definition of _getkey in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of getkey in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of cs_connect in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _cs_connect in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of read_status in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of cs_perror in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _cs_perror in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of write_dialrequest in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of cbc_crypt in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of ecb_crypt in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of des_setparity in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of __des_crypt in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of netdir_free in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of netdir_sperror in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of netdir_perror in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _netdir_perror in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of setnetconfig in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of endnetconfig in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of getnetconfig in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of getnetconfigent in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of freenetconfigent in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of setnetpath in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of endnetpath in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of getnetpath in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of nc_sperror in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _nc_perror in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of nc_perror in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_accept in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_bind in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_close in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_connect in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_error in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _t_error in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_getinfo in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_getname in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_getstate in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_listen in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_look in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_open in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _t_optmgmt in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_rcv in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _t_rcvconnect in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_rcvconnect in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_rcvdis in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 85: definition of _t_rcvrel in /usr/lib32/libxnet.so
> preempts that definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_rcvrel in /usr/lib32/libxnet.so
> preempts that weak definition in /usr/lib32/libnsl.so.
> ld32: WARNING 134: weak definition of t_rcvudata in
> /usr/lib32/libxnet.so preempts that weak definition in
> /usr/lib32/libnsl.so.
> ld32: Giving up after printing 50 warnings.  Use -wall to print all
> warnings.
>
> Also, 'make install' has some problems, too.
> Specifically:
> <atlas:root> gmake install
> gmake[1]: Entering directory
> `/mnt/users/buckley/public_html/q/queue-1.20.1'
> /bin/sh ./mkinstalldirs /usr/local/bin
>   ./install-sh -c  queue /usr/local/bin/queue
> /bin/sh ./mkinstalldirs /usr/local/sbin
>   ./install-sh -c  queued /usr/local/sbin/queued
> if test ! -x random ; then gcc -o random random.c qlib.o; fi
> random.c:5: parse error before `getuid'
> random.c:5: warning: data definition has no type or storage class
> gmake[1]: *** [install-local-stuff] Error 1
> gmake[1]: Leaving directory
> `/mnt/users/buckley/public_html/q/queue-1.20.1'
> gmake: *** [install-am] Error 2
>
> Anybody got time to help a queue newbie?
>
> Thanks in advance.
>
>     -Buckley
>
> _______________________________________________
> Queue-developers mailing list Que...@li...
> To unsubscribe, subscribe, or set options:
> http://lists.sourceforge.net/mailman/listinfo/queue-developers

[Queue-developers] Re: GNU queue 1.20.1 fails to start job on remote host

[W. G. K. <wer...@ya...>]

Sorry for the long delay; I've been out of town and away from email.

QingLong wrote:

>         Hello!
>
>    I have a problem which probably is due to lack of up to date docs
>  for `queue', so I am trying to get help online.
>  IMHO, making a bug mailing list (actually `tips' list certainly is such one)
>  closed (`subscribers only') is really impolite.

This is unfortunately necessary to keep out robo-spam. It's actually kept out quite a bit of spam that would
otherwise have gone out to the list. It's easy to get on and off, and of course there are also the discussion
forums for bugs on SourceForge.

The new queue-support list (where these sorts of messages will begin to go in the future) is not spam-proofed
in this way, but it is intended to have a lower signal-to-noise ratio anyway.

>
>    I try to start a job on an other host using queue command like:
>
>         queue --queue --spooldir wait --wait --no-pty -- echo X
>
>  it just hangs forever. While
>
>         queue --immediate --wait --no-pty -- echo X
>
>  works fine (I have also tried running heavier jobs).
>  When I start job in `wait' queue (configured in a way that makes
>  jobs always start on remote end (using pfactor)) I get in supervisorlog
>  on remote end:
>
> Aug 30 01:44:23 khvanchkara.bunch.ihep.su wait: cfm448981877: START (delayed 0 min): qinglong (qinglong)
> Aug 30 01:57:31 khvanchkara.bunch.ihep.su wait: cfm448981877: END: cpu 0.0s signal 0 exit 2 (02)
>
>  but NOTHING happens (no output), and job (shell job) is still there
>  and do not return control to shell.

What platform is this on.

Try running queued -D &.



>
>
>    I have also tried using NFS-shared queue spool dir, jobs seemed to be
>  starting and processing just fine although not on the remote end,

1.20.1 doesn't use NFS, so the queue spool dir should not be NFS shared (although it will probably work fine if
it is; I don't recommend it.).

>
>  but on local host. Here are messages from supervisorlog on local host:
>
> Aug 29 17:30:54 alexandreuli.bunch.ihep.su wait: cfm448507134: START (delayed 1 min): qinglong (qinglong)
> Aug 29 17:30:55 alexandreuli.bunch.ihep.su wait: cfm448507134: END: cpu 1.6s signal 0 exit 0 (00)
> Aug 29 17:32:56 alexandreuli.bunch.ihep.su wait: cfm448508334: START (delayed 2 min): qinglong (qinglong)
> Aug 29 17:32:57 alexandreuli.bunch.ihep.su wait: cfm448508334: END: cpu 1.6s signal 0 exit 0 (00)
>
>  And queued on remote end was dying with messages like:
>
> Aug 29 17:29:56 khvanchkara.bunch.ihep.su wait: cfm448507134: START (delayed 0 min): qinglong (qinglong)
> Aug 29 17:31:11 khvanchkara.bunch.ihep.su Trouble aborting vanished job cfm448507134
> Aug 29 17:31:11 khvanchkara.bunch.ihep.su wait: cfm448508334: START (delayed 0 min): qinglong (qinglong)
> Aug 29 17:33:11 khvanchkara.bunch.ihep.su Trouble aborting vanished job cfm448508334
> Aug 29 18:10:37 khvanchkara.bunch.ihep.su wait: enabled: maxexec=48 loadsched=36 loadstop=66 nice=1 cpu inf
>
>    This looked like remote queued got confused by local queued control files
>  in shared spool dir, so I decided that new `queue' do not use NFS-shared
>  spool dir, in contrary to what the `queue' documentation states.
>
>    To say the truth, first I tried to use 1.12.8. No success.
>  Too many segfaults and other weirdness.
>
>    So, what I am doing wrong? How should I create batch queue,
>  which executes all jobs on remote end (local host should be `submit only').

You set profile on the submit only host to have a maxexec of zero.


>
>
>    BTW, is it possible to limit pty emulation level for the queue?
>  Say, I would like to insist on `--no-pty' (and `--batch') mode
>  for all jobs in `wait' queue.
>
>    I have also had to hack `queue's Makefile.am (heavily) and configure.in
>  and profile.in to make the build/installation process relocatable
>  (required for src.rpm and other source packages).

build/installation should certainly be relocatable as written (via ./configure); send me the patches so that I
can see what you felt necessary to change.

>
>  If you are interested, I would be happy to send you the patch and RPM spec.
>
>    BR,
>
>       QingLong.

[Queue-developers] IRIX distress

[Buckley C. <Bu...@Me...>]

Hello all.

I am trying to get Queue working in a mixed IRIX/Linux network and am
looking for help making the IRIX build.
I followed the instructions, but it is having too many warnings/errors,
and fails.

I tried both '--enable-root' and single user with comparable results.

IRIX's 'make' complains:
make: file `Makefile' line 336: Syntax error
and line 336 reads:
DEPS_MAGIC := $(shell mkdir .deps > /dev/null 2>&1 || :)

But it was noted in the INSTALL that 'make' might not work, so it was
obvious to use 'gmake'.  But...:
<atlas:buckley> gmake
gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c queue.c
queue.c: In function `main':
queue.c:883: warning: assignment makes pointer from integer without a
cast
queue.c:923: warning: assignment makes pointer from integer without a
cast
queue.c:1266: warning: comparison is always 0 due to limited range of
data type
gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c wakeup.c
wakeup.c: In function `con_daemon':
wakeup.c:179: warning: assignment makes pointer from integer without a
cast
wakeup.c:221: warning: assignment makes pointer from integer without a
cast
wakeup.c: In function `wakeup':
wakeup.c:347: warning: passing arg 4 of `qsort' from incompatible
pointer type
gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c ident.c
gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c qlib.c
gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c sha1.c
In file included from util.h:23,
                 from sha1.c:41:
types.h:52: warning: redefinition of `ushort'
/usr/include/sys/bsd_types.h:31: warning: `ushort' previously declared
here
types.h:58: warning: redefinition of `ulong'
/usr/include/sys/bsd_types.h:35: warning: `ulong' previously declared
here
gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c getloadavg.c
gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c setenv.c
gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c getopt_long.c
gcc  -g -O2  -o queue  queue.o wakeup.o ident.o qlib.o sha1.o
getloadavg.o setenv.o getopt_long.o -lelf -lcrypt -lxnet -lnsl -lsocket
-lelf -lrpcsvc
ld32: WARNING 84: /usr/lib32/libelf.a is not used for resolving any
symbol.
ld32: WARNING 84: /usr/lib32/libcrypt.so is not used for resolving any
symbol.
ld32: WARNING 84: /usr/lib32/libxnet.so is not used for resolving any
symbol.
ld32: WARNING 85: definition of _getkey in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of getkey in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of cs_connect in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _cs_connect in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of read_status in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of cs_perror in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _cs_perror in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of write_dialrequest in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of cbc_crypt in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of ecb_crypt in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of des_setparity in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 85: definition of __des_crypt in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of netdir_free in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of netdir_sperror in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of netdir_perror in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _netdir_perror in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of setnetconfig in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of endnetconfig in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of getnetconfig in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of getnetconfigent in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of freenetconfigent in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of setnetpath in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of endnetpath in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of getnetpath in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of nc_sperror in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _nc_perror in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of nc_perror in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_accept in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_bind in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_close in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_connect in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_error in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _t_error in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_getinfo in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_getname in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_getstate in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_listen in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_look in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_open in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _t_optmgmt in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_rcv in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _t_rcvconnect in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_rcvconnect in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_rcvdis in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _t_rcvrel in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_rcvrel in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_rcvudata in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: Giving up after printing 50 warnings.  Use -wall to print all
warnings.
gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c queued.c
queued.c: In function `transmitjob':
queued.c:3053: warning: comparison is always 1 due to limited range of
data type
queued.c: In function `check_query':
queued.c:4268: warning: assignment makes pointer from integer without a
cast
queued.c:4300: warning: assignment makes pointer from integer without a
cast
gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c lex.c
gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c handle.c
handle.c: In function `chldsigh':
handle.c:119: warning: comparison is always 0 due to limited range of
data type
handle.c: In function `handle':
handle.c:656: warning: assignment makes pointer from integer without a
cast
handle.c:688: warning: assignment makes pointer from integer without a
cast
handle.c:896: warning: passing arg 1 of `open' makes pointer from
integer without a cast
gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c pty.c
pty.c: In function `allocpty':
pty.c:64: warning: assignment makes pointer from integer without a cast
gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c mrestart.c
gcc  -g -O2  -o queued  queued.o lex.o handle.o ident.o pty.o qlib.o
mrestart.o wakeup.o sha1.o getloadavg.o setenv.o getopt_long.o -lelf
-lcrypt -lxnet -lnsl -lsocket -lelf -lrpcsvc
ld32: WARNING 84: /usr/lib32/libelf.a is not used for resolving any
symbol.
ld32: WARNING 84: /usr/lib32/libcrypt.so is not used for resolving any
symbol.
ld32: WARNING 84: /usr/lib32/libxnet.so is not used for resolving any
symbol.
ld32: WARNING 85: definition of _getkey in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of getkey in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of cs_connect in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _cs_connect in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of read_status in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of cs_perror in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _cs_perror in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of write_dialrequest in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of cbc_crypt in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of ecb_crypt in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of des_setparity in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 85: definition of __des_crypt in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of netdir_free in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of netdir_sperror in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of netdir_perror in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _netdir_perror in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of setnetconfig in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of endnetconfig in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of getnetconfig in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of getnetconfigent in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of freenetconfigent in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of setnetpath in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of endnetpath in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of getnetpath in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of nc_sperror in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _nc_perror in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of nc_perror in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_accept in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_bind in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_close in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_connect in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_error in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _t_error in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_getinfo in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_getname in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_getstate in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_listen in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_look in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_open in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _t_optmgmt in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_rcv in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _t_rcvconnect in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_rcvconnect in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_rcvdis in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 85: definition of _t_rcvrel in /usr/lib32/libxnet.so
preempts that definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_rcvrel in /usr/lib32/libxnet.so
preempts that weak definition in /usr/lib32/libnsl.so.
ld32: WARNING 134: weak definition of t_rcvudata in
/usr/lib32/libxnet.so preempts that weak definition in
/usr/lib32/libnsl.so.
ld32: Giving up after printing 50 warnings.  Use -wall to print all
warnings.


Also, 'make install' has some problems, too.
Specifically:
<atlas:root> gmake install
gmake[1]: Entering directory
`/mnt/users/buckley/public_html/q/queue-1.20.1'
/bin/sh ./mkinstalldirs /usr/local/bin
  ./install-sh -c  queue /usr/local/bin/queue
/bin/sh ./mkinstalldirs /usr/local/sbin
  ./install-sh -c  queued /usr/local/sbin/queued
if test ! -x random ; then gcc -o random random.c qlib.o; fi
random.c:5: parse error before `getuid'
random.c:5: warning: data definition has no type or storage class
gmake[1]: *** [install-local-stuff] Error 1
gmake[1]: Leaving directory
`/mnt/users/buckley/public_html/q/queue-1.20.1'
gmake: *** [install-am] Error 2


Anybody got time to help a queue newbie?

Thanks in advance.

    -Buckley

Re: [Queue-developers] Cross-platform termios handling...

[Mike C. <da...@ix...>]

On Sun, Aug 20, 2000 at 01:15:23PM -0400, W. G. Krebs wrote:
> One of the early drafts of the internet protocol actually included the telnet 
> WILL/WONT mechanism for negotiating various things.

In general, the telnet protocol, especially the negotiation, is show as an
example of how NOT to do things.  You really do not want to follow that
example.

Shoot... just thought of stuff for rlogin, and already delete the
references but...

One thing to remember with the r commands is that you either run a command
non-interactively, or you run an interactive shell that probably gets
initialized with something equivalent to "stty sane", regardless of what
the originating terminal was set for.  You can't remotely run a command
that requires a terminal.

Now, on the otherhand, there is ssh.  How does ssh solve this?  Or OpenSSH?
Or (what's the gnu one? lsh?)?  They may be better canidates for examples.

> For simplicity, however, we could write a translator function that translates 
> Linux termios to the simplified rlogin structure (portable) and then use code 
> from rlogin to translate the simplified structure to machine termios on 
> non-Linux systems. This might save us some porting effort.

Use XDR.  It will save a lot of headaches in the long run.  Of course, you
can't just pass the binary values of the structures across.  Where one
system will have:

#define FOO 0
#define BAR 1

the other one will have:

#define FOO 1
#define BAR 2

So you still have to come up with things like:

enum {
QUEUE_FOO,
QUEUE_BAR
}

And do mappings from each local system to a neutral format, pass that
across, and convert back.  But for the passing of the neutral format, you
probably should look at XDR.  

mrc
-- 
       Mike Castle       Life is like a clock:  You can work constantly
  da...@ix...  and be right all the time, or not work at all
www.netcom.com/~dalgoda/ and be right at least twice a day.  -- mrc
    We are all of us living in the shadow of Manhattan.  -- Watchmen

Re: [Queue-developers] Cross-platform termios handling...

[Mike C. <da...@ix...>]

On Sat, Aug 19, 2000 at 10:26:22PM -0400, W. G. Krebs wrote:
> Expect's would probably be a good replacement for some of the stuff that goes 
> on in pty.c, however.

My original reply was directed towards this end, actually.  I hadn't read
the original post well enough, and my response didn't actually answer the
question being asked.

I agree that pty.c could be replaced with much of the stuff from expect.

Sorry for the confusion.

mrc
-- 
       Mike Castle       Life is like a clock:  You can work constantly
  da...@ix...  and be right all the time, or not work at all
www.netcom.com/~dalgoda/ and be right at least twice a day.  -- mrc
    We are all of us living in the shadow of Manhattan.  -- Watchmen

Re: [Queue-developers] Cross-platform termios handling...

[Andrew P. <at...@pe...>]

 Hi,
 
 Well, my (limited) experience of implementing the telnet protocol
 in a proprietary package is that you either write almost all of it,
 or none at all. From that point of view rlogin is cheaper and 
 easier. 
 
 Perhaps you could consider looking at OpenSSH as well, as that seems
 to be becoming widely accepted as an rlogin replacement. Besides 
 which Last time I checked the sources only unix-style systems were
 supported.
 
  Cheers, 
     Andy

> On reading the RFCs for several telnets, I discovered that it was very 
> complicated, however. One of the later RFCs complains that the original RFC for 
> telnet describes a protocol that doesn't work. A big problem with this option 
> negotiation is ensuring that the negotiation terminates.


-- 
Dr Andy Phillips                                 at...@pe... 
Pergamentum Solutions                              at...@co...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Re: [Queue-developers] Cross-platform termios handling...

[W. G. K. <wer...@ya...>]

One of the early drafts of the internet protocol actually included the telnet 
WILL/WONT mechanism for negotiating various things.

On reading the RFCs for several telnets, I discovered that it was very 
complicated, however. One of the later RFCs complains that the original RFC for 
telnet describes a protocol that doesn't work. A big problem with this option 
negotiation is ensuring that the negotiation terminates.

I felt it would be difficult to implement this properly. Moreover, it was to far 
away from the way Queue currently did things, so I removed this from the 
protocol. The final draft of the submitted draft protocol is very close to the 
way Queue actually does things, and you might say that 1.20.1 is a draft 
protocol-compliant implementation.

Of course, the draft protocol is just a draft, and if there's strong feeling in 
favor of this mechanism (e.g., someone supplies code that actually implements a 
telnet-style WILL-WONT mechanism in Queue) then we can go along with this.

However, I don't think we need option negotiation. We just need to pass a 
superset of all supported options, and then let the other side support whatever 
it can support or refuse to provide service.

The rlogin protocol is quite different from telnet. It doesn't use option 
negotiation AFAIK. It passes the lowest common denominator of terminal 
information, i.e., a subset of termios, to the other side. This works, and 
rlogin has been implemented on many non-UNIX platforms as well, so it can be 
done transparantly.

We don't have to follow this approach either. We can pass the Linux termios 
structure (a superset of termios on many systems) and then let each system pick 
and choose want it wants to support.

For simplicity, however, we could write a translator function that translates 
Linux termios to the simplified rlogin structure (portable) and then use code 
from rlogin to translate the simplified structure to machine termios on 
non-Linux systems. This might save us some porting effort.

In the end, though, we should do this in simpliest possible way, and if people 
have ideas they should speak up (or provide code. :) )


Quoting Andrew Phillips <at...@pe...>:

> Hi,
> 
> > Based on this description, it's unlikely that Expect sends terminal
> information 
> > over the network. (On the other hand, rlogin and telnet do send terminal 
> > information over the network, so perhaps we should look at the code base for 
> > these two programs. Ultimately, however, they will have to translate
> information 
> > to termios structures as well -- there's no way to completely get around
> this. 
>     
>     The telnet RFC plus the EDIT/Linemode extension RFC contains a pretty
>     complete description of the way this is done.
>     briefly, telnet defines an 'escape' character, plus a whole of modifiers,
>     and during the course of a telnet connection conversations like
>     
>     (telnet)  DO ECHO
>     (telnetd) WILL ECHO
>     turns remote echo on.
>     (telnet)  DONT ECHO
>     (telnetd) WONT ECHO
>     turns remote echo off.
>     
>     (These actually occur in binary, but these are the mnemonics used in 
>      the RFC.)
>     
>     Likewise for edit. The client and server negotiate capabilities, 
>     and settle on a combination of line editing in the client and 
>     on the remote host. They dont actually transfer termios structures
>     or similar across the network, as it was intended as a platform 
>     independent virtual terminal, with servers running on all sorts 
>     of systems ranging from VAX/VMS, MVS, and UNIX, only one of which
>     understood termios at that time. 
>     
>     Another extension allows the transfer of a users unix environment
>     including the TERM environment variable across the connection. 
>     
>         Hope this helps.
> 	   Andy
> 
> -- 
> Dr Andy Phillips                                 at...@pe... 
> Pergamentum Solutions                              at...@co...
> 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> 
> _______________________________________________
> Queue-developers mailing list
> Que...@li...
> http://lists.sourceforge.net/mailman/listinfo/queue-developers
> 

Re: [Queue-developers] Cross-platform termios handling...

[Andrew P. <at...@pe...>]

Hi,

> Based on this description, it's unlikely that Expect sends terminal information 
> over the network. (On the other hand, rlogin and telnet do send terminal 
> information over the network, so perhaps we should look at the code base for 
> these two programs. Ultimately, however, they will have to translate information 
> to termios structures as well -- there's no way to completely get around this. 
    
    The telnet RFC plus the EDIT/Linemode extension RFC contains a pretty
    complete description of the way this is done.
    briefly, telnet defines an 'escape' character, plus a whole of modifiers,
    and during the course of a telnet connection conversations like
    
    (telnet)  DO ECHO
    (telnetd) WILL ECHO
    turns remote echo on.
    (telnet)  DONT ECHO
    (telnetd) WONT ECHO
    turns remote echo off.
    
    (These actually occur in binary, but these are the mnemonics used in 
     the RFC.)
    
    Likewise for edit. The client and server negotiate capabilities, 
    and settle on a combination of line editing in the client and 
    on the remote host. They dont actually transfer termios structures
    or similar across the network, as it was intended as a platform 
    independent virtual terminal, with servers running on all sorts 
    of systems ranging from VAX/VMS, MVS, and UNIX, only one of which
    understood termios at that time. 
    
    Another extension allows the transfer of a users unix environment
    including the TERM environment variable across the connection. 
    
        Hope this helps.
	   Andy

-- 
Dr Andy Phillips                                 at...@pe... 
Pergamentum Solutions                              at...@co...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Re: [Queue-developers] Cross-platform termios handling...

[W. G. K. <wer...@ya...>]

Quoting Eric Deal <eri...@co...>:

> 
> The particular problem I'm trying to address is the passing of 
> terminal settings when queueing a job from one platform to another.
> One basic problem is that the job control file passes a binary
> structure from the queue client to the queued server that 
> conveys the current state of the terminal at the client.
> 
> When the platform is the same for both the client and server,
> this works fine.  Even if queue's terminal handling is not good,
> the mechanism does work.
> 
> Where this falls apart is when the client and server machines
> are different platforms and the native termios structure differs.
> In this case, major problems occur because the termios structures can
> be arranged differently and have different lengths, which causes
> the job control parsing to fail.
> 
> So it seems that there are two underlying problems to be solved.
> The first is how to abstract the terminal settings between platforms to 
> allow the passing of terminal settings from client to server, and the 
> second is whether queue should improve its general terminal handling.  
> 
> I'm fairly new to queue (cross-platform capability is a must before 
> it even becomes useful in my environment), so I don't have any opinion
> on how well queue handles terminal settings.  I'm also not familiar
> with expect and how it handles terminals.
> 
> So I'll repose the original question:  
> 
>   1) Does Expect have a modular way of handling terminal IO that could be 
>      applied in Queue that would improve the Queue codebase?

I'm not that familar with expect (Mike Castle's post was the first I'd heard of 
it), but it is a public domain scripting language for controlling interactive 
tasks developed by the U.S. government (NIST) for Unix and Windows. The URL is: 
http://expect.nist.gov/ . The examples directory lists expect scripts that use 
rlogin and telnet to perform an FTP-like function, run the "passwd" program 
non-interactively to set passwords for the system administrator, etc.

Based on this description, it's unlikely that Expect sends terminal information 
over the network. (On the other hand, rlogin and telnet do send terminal 
information over the network, so perhaps we should look at the code base for 
these two programs. Ultimately, however, they will have to translate information 
to termios structures as well -- there's no way to completely get around this. 
Telnet and rlogin don't have to worry about their remotes being resumed or their 
local end being suspended by unexpected signals like tty in and then having to 
figure out how the terminal has changed, so they are solving a much simpler 
problem than Queue.)

Expect's would probably be a good replacement for some of the stuff that goes 
on in pty.c, however.


>   2) Does such a modular architecture change the method that terminal 
>      settings would be exchanged between client/server machines such that
>      implementing a new terminal-handling scheme would make passing
>      settings easier than it currently can be done?
> 
> I'd like to get other's input on this before coding up something that
> might end up as a band-aid instead of doing this the right way.
> 
> Eric
> 
> >On Wed, Aug 16, 2000 at 11:02:54AM -0500, Eric Deal wrote:
> >> I have a request for people running queue on various platforms.
> >> Please send me a copy of the termios.h header file for platforms
> >> that you are using.  
> >
> >Of all the programs that I know that use terminals like this (queue,
> >screen, expect), I've only seen one ever get it right.  Expect.
> >
> >The last time I looked at queue and it's terminal stuff, because it wasn't
> >working on my Linux box, I thought it was pretty much of a mess.
> >
> >I'd probably suggest that it all be ripped out and steal the stuff from
> >expect, and try to track it as a reference source.  No since reinventing
> >the wheel when there is a perfectly good one you already use.
> >
> >mrc
> >-- 
> >       Mike Castle       Life is like a clock:  You can work constantly
> >  da...@ix...  and be right all the time, or not work at all
> >www.netcom.com/~dalgoda/ and be right at least twice a day.  -- mrc
> >    We are all of us living in the shadow of Manhattan.  -- Watchmen
> >
> >_______________________________________________
> >Queue-developers mailing list
> >Que...@li...
> >http://lists.sourceforge.net/mailman/listinfo/queue-developers
> >
> 
> 
> 
> _______________________________________________
> Queue-developers mailing list
> Que...@li...
> http://lists.sourceforge.net/mailman/listinfo/queue-developers
> 

Re: [Queue-developers] Cross-platform termios handling...

[W. G. K. <wer...@ya...>]

On Wed, Aug 16, 2000 at 11:02:54AM -0500, Eric Deal wrote:
> I have a request for people running queue on various platforms.
> Please send me a copy of the termios.h header file for platforms
> that you are using.  

Of all the programs that I know that use terminals like this (queue,
screen, expect), I've only seen one ever get it right.  Expect.

The last time I looked at queue and it's terminal stuff, because it wasn't
working on my Linux box, I thought it was pretty much of a mess.

I'd probably suggest that it all be ripped out and steal the stuff from
expect, and try to track it as a reference source.  No since reinventing
the wheel when there is a perfectly good one you already use.

mrc
-- 
       Mike Castle       Life is like a clock:  You can work constantly
  da...@ix...  and be right all the time, or not work at all
www.netcom.com/~dalgoda/ and be right at least twice a day.  -- mrc
    We are all of us living in the shadow of Manhattan.  -- Watchmen

_______________________________________________
Queue-developers mailing list
Que...@li...
http://lists.sourceforge.net/mailman/listinfo/queue-developers

[Queue-developers] Re: questions about Queue

[W. G. K. <wer...@ya...>]

Probably the best way to get answers to questions like these is to write to
the Queue developers at que...@li... (you must
subscribe first because that's how the spam-proofing works) or the new
(currently much smaller) support list for the anticipated higher volume
support traffic at que...@li... .

Boris Lorbeer wrote:

> Hi,
> my name is Boris Lorbeer and I am working for a company which is looking
> for a good load sharing tool
> which is open source and not as expensive as LSF.
> So we came across the Queue program and after spending some time with it
> I have a couple of
> questions.
>

Again, Open Source software works because the costs (often chiefly time) for
developing and maintaining useful programs are distributed over many
organizations via the Internet.

Hence, if you have thoughtful questions or complaints (asside from security
bugs, if there are any) it is important that these be posted to one of the
lists so that maximal resources can be brought to bear on the problem. A key
issue is dividing up the lists (currently, there are three, along with the
discussions forums on Sourceforge) so that the signal-to-noise ratio of each
list is minimized and the right people are getting the right list traffic.

First of all, when I installed Queue as root, it did not acknowledge my

> settings for the option
> --sharedstatedir to the configure program, but it always chose the
> default which was
> /usr/local/var/queue (not to /usr/local/com/queue which is what the docu
> was claiming).

The ./configure tool is generated by autoconf and should work; see
./configure --help for usage.
--sharedstatedir and the other directories should be recognized.

I think the docs are out of date, however. "--sharedstatedir" is indeed
"com/queue", but after 1.20.1 there no longer is a sharedstatedir, so stuff
gets stored in "var/queue" which is "--localstatedir"

The manuals should be updated. They are currently FAQ-O-MATIC HTMLs
http://bioinfo.mbb.yale.edu/fom/cache/1.html , so can be updated by anyone,
but eventually I should put them on Sourceforge so that our developers can
update them.

>
> Next, when I changed the entries in the profile file and wanted the tool
> to read it, the way proposed by
> the docu (kill -HUP) did not work; this signal was always terminating
> the program. The docu says:
> "queued will also periodically check for modifications to these files".
> What means 'periodically'?

Sign. Again, the docs are out-of-date, especially after 1.20 came out.

"Periodically" used to be every 120 seconds, although this has changed. The
program checks for modifications more frequently when there is activity in
the batch queue.

kill -ALRM to queued should work, however.

If it doesn't see your changes within 120 seconds or after "kill -ALRM"
something is wrong.

>
> For testing I chose 3 linux machines. The first setting for maxexec was
> 2 which caused the daemons and
> the client queue to hang whenever I started more than say 5 programs at
> once.

3*2 = 6, so 6 jobs throughout the cluster. If you run five, there's only one
free spot in the cluster.

You're probably using 1.20.1 and seeing the deadlock bug; upgrade to the
version in the CVS archive (unpack the installation, cd the top-level
directory, and type "cvs update" with CVS installed or get cvs off of
ftp.gnu.org.)

Sounds like it is time for 1.20.2 to come out.

>
> When I increased maxexec to 20 these problems were gone, even when I was
> starting 40 processes at once:
> repeat 40 queue -n -q -d test_queue -r -- /usr/local/bin/perl -e
> '$SIG{ALRM} = sub {print "finished\n"; exit;}; alarm(20); while(1){};'

20*3 = 60 job slots, only 40 jobs, 20 spare jobs; 20/3 suggests high
probability of find a job on any machine. Deadlock bug not triggered.

>
> But now I got other problems: First I should say that I have closed one
> host in this queue so that there
> were only 2 left. For these 40 jobs I received 80 mails : each of these
> 40 jobs appeared twice. One mail
> was send from running the job on the one host and one mail was send from
> running the same job
> (or rather a job with the same job id) on the
> other host. One of the mails had a proper looking message but the other
> one always complained:
> Can't stat output file test_queue/ofm433421925
> (probably because his twin finished first).
>
> Furthermore, the documentation is not very clear about the variables
> vmaxexec, nexec, maxexec.
> One time it was:
> 1-min load average/ (( max(0, vmaxexec - nexec) + 1)*pfactor)
> and in another piece of the docu I read:
> 1-min load average/ (( max(0, vmaxexec - maxexec) + 1)*pfactor).
> Is maxexec the maximum number of jobs sent to one host from this queue
> or is it the maximum number
> of the jobs in this queue (i.e. summed up over all the hosts)?

Time to update and upgrade the documentation (Volunteers? It's possible to
set up the docs so that can be edited by developers on sourceforge.
Currently, they can be edited by anyone on the FAQ-O-MATIC page,
http://bioinfo.mbb.yale.edu/fom/cache/1.html . If you see something you
think should be changed, feel free to go ahead and change it without asking.
I get emails of all changes that are made, anyway.)

nexec is the "C" language variable used in the program, maxexec is the same
variable as named in "profile" files. The documentation should use "maxexec"
for consistancy and clarity.

Maxexec and vmaxexec are per node. It has been suggested that GNU Queue use
a central database to establish variables over the entire network, but no
one has implemented this, so far.

>
>
> The docu also says:
> 'These options, if present, will only override the user's values (via
> queue) for these
> limits if they are lower than what the user has set (or larger in the
> case of nice).'
> (this was related to options in the profile file like 'maxfree' and so
> on)
> How can I specify these options via the client 'queue'?

These limits involve operating system resource limits (such as "nice", which
specifies the priority a program should be run at.) These can be specified
both by the user (via the "limit" command in csh scripts, e.g., limit the
size of coredumps to zero to turn off coredumping) and in "profile" as well.
The lower value is chosen, so an administrator could turn off coredumping
for a specific batch queue even if the user allows coredumping.

maxexec and vmaxexec cannot be set by the user in current versions of GNU
Queue.

>
>
> I saw in queue.c an option 'j'. What is it used for?

You can attach an optional comment to your jobs. This may be used in email
output, although it is really a vestige of very old days.

>
> I would really appreciate it if you could help me with these problems.
> Thank you very much.
> Boris Lorbeer.

Re: [Queue-developers] Cross-platform termios handling...

[Eric D. <eri...@co...>]

The particular problem I'm trying to address is the passing of 
terminal settings when queueing a job from one platform to another.
One basic problem is that the job control file passes a binary
structure from the queue client to the queued server that 
conveys the current state of the terminal at the client.

When the platform is the same for both the client and server,
this works fine.  Even if queue's terminal handling is not good,
the mechanism does work.

Where this falls apart is when the client and server machines
are different platforms and the native termios structure differs.
In this case, major problems occur because the termios structures can
be arranged differently and have different lengths, which causes
the job control parsing to fail.

So it seems that there are two underlying problems to be solved.
The first is how to abstract the terminal settings between platforms to 
allow the passing of terminal settings from client to server, and the 
second is whether queue should improve its general terminal handling.  

I'm fairly new to queue (cross-platform capability is a must before 
it even becomes useful in my environment), so I don't have any opinion
on how well queue handles terminal settings.  I'm also not familiar
with expect and how it handles terminals.

So I'll repose the original question:  

  1) Does Expect have a modular way of handling terminal IO that could be 
     applied in Queue that would improve the Queue codebase?

  2) Does such a modular architecture change the method that terminal 
     settings would be exchanged between client/server machines such that
     implementing a new terminal-handling scheme would make passing
     settings easier than it currently can be done?

I'd like to get other's input on this before coding up something that
might end up as a band-aid instead of doing this the right way.

Eric

>On Wed, Aug 16, 2000 at 11:02:54AM -0500, Eric Deal wrote:
>> I have a request for people running queue on various platforms.
>> Please send me a copy of the termios.h header file for platforms
>> that you are using.  
>
>Of all the programs that I know that use terminals like this (queue,
>screen, expect), I've only seen one ever get it right.  Expect.
>
>The last time I looked at queue and it's terminal stuff, because it wasn't
>working on my Linux box, I thought it was pretty much of a mess.
>
>I'd probably suggest that it all be ripped out and steal the stuff from
>expect, and try to track it as a reference source.  No since reinventing
>the wheel when there is a perfectly good one you already use.
>
>mrc
>-- 
>       Mike Castle       Life is like a clock:  You can work constantly
>  da...@ix...  and be right all the time, or not work at all
>www.netcom.com/~dalgoda/ and be right at least twice a day.  -- mrc
>    We are all of us living in the shadow of Manhattan.  -- Watchmen
>
>_______________________________________________
>Queue-developers mailing list
>Que...@li...
>http://lists.sourceforge.net/mailman/listinfo/queue-developers
>

Re: [Queue-developers] Cross-platform termios handling...

[Mike C. <da...@ix...>]

On Wed, Aug 16, 2000 at 11:02:54AM -0500, Eric Deal wrote:
> I have a request for people running queue on various platforms.
> Please send me a copy of the termios.h header file for platforms
> that you are using.  

Of all the programs that I know that use terminals like this (queue,
screen, expect), I've only seen one ever get it right.  Expect.

The last time I looked at queue and it's terminal stuff, because it wasn't
working on my Linux box, I thought it was pretty much of a mess.

I'd probably suggest that it all be ripped out and steal the stuff from
expect, and try to track it as a reference source.  No since reinventing
the wheel when there is a perfectly good one you already use.

mrc
-- 
       Mike Castle       Life is like a clock:  You can work constantly
  da...@ix...  and be right all the time, or not work at all
www.netcom.com/~dalgoda/ and be right at least twice a day.  -- mrc
    We are all of us living in the shadow of Manhattan.  -- Watchmen

[Queue-developers] Cross-platform termios handling...

[Eric D. <eri...@co...>]

I have a request for people running queue on various platforms.
Please send me a copy of the termios.h header file for platforms
that you are using.  

I have access to Linux and Solaris machines, so I don't need these.

Please send the headers to eri...@co... and I ask that
you identify which headers come from which systems.  Also, please
check to see if /usr/include/termios.h includes something like
<bits/termios.h> (from Linux) which has the real bit and structure
definitions.  I really need these definitions, not the top-level termios.h.

Thanks... Eric Deal

[Queue-developers] Fwd: Re: load averages...

[W. G. K. <wer...@ya...>]

Looks great Eric.

This changes involve replacing fwrite and fread with netfwrite and netfread, 
which write bytes on little endian machines in network byte order.

Eric's changes go some way towards allowing communication between Linux clients 
and Solaris servers (I think he indicated he was about 80% of the way there, 
jobs start on the other architecture, although there may be some problem getting 
data back by way of the terminal.)

Queue users can download the changes by unpacking the 1.20.1 distribution and 
typing "cvs update" from inside the directory. This assumes that cvs is 
installed on your system; it is installed by default under most distributions. 
You can also get it off ftp.gnu.org.

If you fix the remaing problems, be sure to either submit a patch or apply for 
write access to the CVS repository so that you can bounce code between yourself 
and other developers.


----- Forwarded message from Eric Deal <eri...@co...> -----
Date: Mon, 14 Aug 2000 16:57:38 -0500
From: Eric Deal <eri...@co...>
Reply-To: eri...@co...
Subject: Re: load averages...
To: "W. G. Krebs" <wer...@ya...>


Werner,

I just checked in my initial changes for cross-platform operation.
The relevant changes are in the following files:

  queue.h        (included queue_endian.h)
  queue_endian.h (new file)
  sha1.c         (included queue_endian.h)
  queue.c        (netfread/netfwrite)
  queued.c       (netfread/netfwrite)
  wakeup.c       (netfread/netfwrite)
  handle.c       (netfread/netfwrite)
  qlib.c         (added netfread/netfwrite functions)

[snip.]

[Queue-developers] Bugfixes/updates in CVS repository.

[Werner G. K. <wer...@ya...>]

I've collected the various bugfixes and put them in the CVS repository.

You should be able to obtain these changes by doing the following:

- unpacking the queue distribution.
- running the command "cvs update" from inside the distribution
directory.

This assumes you have cvs installed. Many linux distributions install
this by default; you can get it from ftp.gnu.org . There is also a web
browser of the repository on Sourceforge, but this is much harder to use
if you just want to update your files.

Hopefully, I can encourage more developers to become familar with CVS so
that they can upload changes themselves, and these can then be
distributed much more quickly between developers and testers. With write
access, you make your changes and then run "cvs ci"; you're prompted for
your password and a ChangeLog entry. This change is permanently recorded
with your SourceForge ID and log entry, but can be easily undone by
another developer if it doesn't work out as expected. All other
developers can then see your change by running "cvs update", presumably
in response to a message on one of the lists.

The main fixes are elimination of the strange commentation in handle.c
pointed out by Eric Deal and elimination of the potential deadlock
condition pointed out by eh...@so...

Here is the complete ChangeLog:

*August 11, 2000      W. G. Krebs <wk...@gn...>

        In Makefile.am :

        "rd...@io..."<rd...@io...> pointed out that random must be
        installed by Makefile.am (rather than in install-local-stuff)
        because it needs to link gethostbyname under Solaris. So, it
        was registered with automake as a noinst program. See
Makefile.am


*August 11, 2000      W. G. Krebs <wk...@gn...>

        Eric Deal <eri...@co...> pointed out strange
commentation
        are setrlimit source in handle.c that affected the socket setup.

        This deed indeed look strange. In fact, it probably caused a new
bug
        in the handling of X-windows applications (which close stdin and

        stdout) that I know wasn't present in old versions of Queue.

        So, I've fixed handle.c to just comment out the setrlimit code.
This
        may also have been commented out because of Solaris. If so, it
        should be put back in with an "#ifndef solaris" instead of a
comment.

*August 11, 2000      W. G. Krebs <wk...@gn...>

        Two bugs:

        1.

        Bug spotted by eh...@so....
        QueueD could deadlock if both are trying to transmit at same
time to
        each other. So, always fork by setting TRANSMIT_DEBUG in
queued.c

        2.

        Eric Deal <eri...@co...> pointed out that 1260-1264 in

        queued.c causes a hang Solaris, so I added an "ifndef solaris"
to this
        section of the code to prevent it from being compiled into
Solaris.


*August 11, 2000      W. G. Krebs <wk...@gn...>

        Made some changes to pty.c for linux. #ifdef linux causes
        utmp to be processed slightly differently under linux. Also,
        added check to make sure /dev/tty doesn't get chmod'ded on
        all platforms.

[Queue-developers] re: bug queue 1.20.1 on solaris 2.5.1

[Eric D. <eri...@co...>]

I found the problem when running 1.20.1 on Solaris.
I don't know the proper fix, but commenting out lines 2860-2864
in queued.c takes care of the problems that I was seeing:

/* ------ EJD this breaks Solaris
  for( i=0; i<RLIM_NLIMITS; i++ ){
    register struct rlimit *rlp = &(qp->q_rlimit[i]);
    if( rlp->rlim_cur >= 0 && rlp->rlim_max >= 0 )
      (void) setrlimit( itorl(i), rlp );
  }
-----------*/

Eric

>
>--------------FEBE046F0715C0346C7AF156
>Content-Transfer-Encoding: 7bit
>Content-Type: text/plain; charset=us-ascii
>
>Not sure what to tell you.
>
>This looks like a problem communicating between handle.c and queue.c
>(job control file parsing problem) under the Solaris
>
>Some people seem to have gotten it under Solaris, which is why I'm
>posting it here.
>
>The problem we've had with Solaris in the past is that quite a lot
>changes from one release to the next, and it is difficult getting it to
>work with all releases.
>
>However, you're right -- random should be compiled by the Makefile so
>that ./configure throws gethostbyname into the compile command for
>random.c
>
>--------------FEBE046F0715C0346C7AF156
>Content-Transfer-Encoding: 7bit
>Content-Type: message/rfc822
>Content-Disposition: inline
>
>Content-Type: text/plain
>Content-Transfer-Encoding: quoted-printable
>
>Hi.=20
>   =20
>I've compiled queue-1.20.1 on Solaris 2.5.1 workstation and I've found
>the
>following problems:
>   =20
>1) I run ./configure --enable-root. Then I compile with make, and i've
>attached the warnings produced by gcc. When i use 'make install' it
>gives me
>the following error:
>
>  /bin/sh ./mkinstalldirs /tmp/sbin
>  ./install-sh -c  queued /tmp/sbin/queued
>  if test ! -x random ; then gcc -o random random.c qlib.o; fi
>  Undefined                       first referenced
>   symbol                             in file
>   gethostbyname                       qlib.o
>   ld: fatal: Symbol referencing errors. No output written to random
>
>I think that random should be compiled during "make" and not during make
>install. It's quite annoying having a root owned file in the sources
>directory, and probably also fix the linking problem.
>
>BTW: i solved the problem manually linking the 'random' file.
>
>2) after installing it, I've launched queued:
>
># queued -D
>
>on other window i've launched queue as normal user:
>
>$ queue -i -w -n -- hostname
> =20
>and the process keep on running, until I stop it with ^C.
> =20
>On the 'queued' window i see the following output:
>  =20
> sbin/queued -D
> SENDMAIL: To 'diego' from 'queued': Subject: batch queue=5Fb on
> ic9ws41.settimo.italtel.it: now/CFDIR/cfm420644940: Job is starting
>now.
>
> now/CFDIR/cfm420644940: Job is starting now.
>
>The 'ps -ef | grep hostname' output is
>    root  6573   860  0 11:46:44 pts/4    0:00 queue -i -w -n --
>hostname
>   diego  6580  6579  1 11:55:48 pts/1    0:00 sh -c (ps -ef | grep
>hostname)
>2>&1 < /dev/null
>   diego  6581  6580  1 11:55:48 pts/1    0:00 grep hostname
>
>I can't figure what is the problem.=20
>
>Thanks in advance.
> =20
>  Diego Roversi.
>
>
>--------------FEBE046F0715C0346C7AF156--
>

[Queue-developers] [Fwd: queued bug]

[W. G. K. <wer...@ya...>]

This is a problem on Debian systems when run with root privileges
(--enable-root in the ./configure).

I've applied what I think will fix the problem to pty.c in CVS for Linux
systems. I've also added a check to make sure /dev/tty doesn't get
changed on any platforms.

pty.c might need some work to make the utmp handling code a bit more
portable.

Re: [Queue-developers] [Fwd: bug queue 1.20.1 on solaris 2.5.1]

[Eric D. <eri...@co...>]

I'm having the same problem with Solaris 5.5.1 (and presumably 5.7).
Something appears to go wrong soon after queued forks off the 
child job and calls the handle() routine.  At this point, I'm not even
sure that handle() gets called.

The only additional info I have so far is that the child is terminated
with signal 31 (SIGXFSZ) and a core dump.  

I'll report back to the mailing list when I know more.

Eric

>--------------FEBE046F0715C0346C7AF156
>Content-Transfer-Encoding: 7bit
>Content-Type: text/plain; charset=us-ascii
>
>Not sure what to tell you.
>
>This looks like a problem communicating between handle.c and queue.c
>(job control file parsing problem) under the Solaris
>
>Some people seem to have gotten it under Solaris, which is why I'm
>posting it here.
>
>The problem we've had with Solaris in the past is that quite a lot
>changes from one release to the next, and it is difficult getting it to
>work with all releases.
>
>However, you're right -- random should be compiled by the Makefile so
>that ./configure throws gethostbyname into the compile command for
>random.c
>
>--------------FEBE046F0715C0346C7AF156
>Content-Transfer-Encoding: 7bit
>Content-Type: message/rfc822
>Content-Disposition: inline
>
>Content-Type: text/plain
>Content-Transfer-Encoding: quoted-printable
>
>Hi.=20
>   =20
>I've compiled queue-1.20.1 on Solaris 2.5.1 workstation and I've found
>the
>following problems:
>   =20
>1) I run ./configure --enable-root. Then I compile with make, and i've
>attached the warnings produced by gcc. When i use 'make install' it
>gives me
>the following error:
>
>  /bin/sh ./mkinstalldirs /tmp/sbin
>  ./install-sh -c  queued /tmp/sbin/queued
>  if test ! -x random ; then gcc -o random random.c qlib.o; fi
>  Undefined                       first referenced
>   symbol                             in file
>   gethostbyname                       qlib.o
>   ld: fatal: Symbol referencing errors. No output written to random
>
>I think that random should be compiled during "make" and not during make
>install. It's quite annoying having a root owned file in the sources
>directory, and probably also fix the linking problem.
>
>BTW: i solved the problem manually linking the 'random' file.
>
>2) after installing it, I've launched queued:
>
># queued -D
>
>on other window i've launched queue as normal user:
>
>$ queue -i -w -n -- hostname
> =20
>and the process keep on running, until I stop it with ^C.
> =20
>On the 'queued' window i see the following output:
>  =20
> sbin/queued -D
> SENDMAIL: To 'diego' from 'queued': Subject: batch queue=5Fb on
> ic9ws41.settimo.italtel.it: now/CFDIR/cfm420644940: Job is starting
>now.
>
> now/CFDIR/cfm420644940: Job is starting now.
>
>The 'ps -ef | grep hostname' output is
>    root  6573   860  0 11:46:44 pts/4    0:00 queue -i -w -n --
>hostname
>   diego  6580  6579  1 11:55:48 pts/1    0:00 sh -c (ps -ef | grep
>hostname)
>2>&1 < /dev/null
>   diego  6581  6580  1 11:55:48 pts/1    0:00 grep hostname
>
>I can't figure what is the problem.=20
>
>Thanks in advance.
> =20
>  Diego Roversi.
>
>
>--------------FEBE046F0715C0346C7AF156--
>

[Queue-developers] [Fwd: bug queue 1.20.1 on solaris 2.5.1]

[W. G. K. <wer...@ya...>]

Not sure what to tell you.

This looks like a problem communicating between handle.c and queue.c
(job control file parsing problem) under the Solaris

Some people seem to have gotten it under Solaris, which is why I'm
posting it here.

The problem we've had with Solaris in the past is that quite a lot
changes from one release to the next, and it is difficult getting it to
work with all releases.

However, you're right -- random should be compiled by the Makefile so
that ./configure throws gethostbyname into the compile command for
random.c

Re: [Queue-developers] Fwd: Queue Question

[W. G. K. <wer...@ya...>]

My own personal take on this is that anything that makes Open Source software 
more useful (without creating legal headaches for the software) is a good thing. 
I'm in the Eric Raymond camp that believes commercial software will never 
totally go away for various economic reasons.

Some influential members of the Open Source community automatically oppose 
anything that "would encourage the use of commercial software" on strong 
ideological grounds. They don't even want the names of commercial software 
products mention in program documentation (asside from the names of operating 
systems). So, I'm not supposed to mention the existance of "flexlm" in program 
documentation.

So, I suppose if your modifications merely grudingly tolerate commercial 
software rather than encourage its proliferation, you're OK with everyone. :)

Your modication might be aimed more at controlling the total number of copies 
that can run simultaneously in a cluster for organization global resource 
allocation purposes. (This has even been a requested feature; it has been 
suggested that there would be a central MySQL server that would keep track of 
statistics and resource usage on running jobs.) So, perhaps you might want to 
add hooks for a MySQL (or Perl DBI) based GNU "license manager" that would check 
with the central database and would limit global resource usage. This would make 
it more palatiable to some in the GNU community.

Note that write access to the CVS repository is available for developers.

Quoting Lee Bradshaw <le...@se...>:

> On Fri, Aug 04, 2000 at 09:06:04PM -0400, W. G. Krebs wrote:
> > It would be nice to know what sort of functionality you are adding to GNU
> Queue. 
> > This helps reduce duplication of effort if someone else is making similar 
> > changes. OpenSource works best if developers contribute their modifications
> to 
> > the general project. You can submit your modifications as patches to the
> Patch 
> > archive on SourceForge (or just post them here if they are small.)
> 
> Thanks for answering Monica's questions. She's trying to add support
> for tools that require software licenses (such as those managed by the
> flexlm license manager). We've talked about sending a description of
> her plans to you or the mailing list, but she hasn't had time to write
> anything yet. We'd like to make sure her modifications (or something
> similar) are acceptable to the queue community so that they will be
> included in future versions of the software.
> 
> -- 
> Lee Bradshaw                 le...@se... (preferred)
> Alantro Communications       le...@al...
> 
> _______________________________________________
> Queue-developers mailing list
> Que...@li...
> http://lists.sourceforge.net/mailman/listinfo/queue-developers
>