[Nelsonlab-analysis] SGE Quotas and Host prioritization (follow up from lab meeting)

[Jordan M. <jme...@uc...>]

Hi all,

I think I resolved two issues, so please keep an eye on the cluster and your
jobs to make sure nothing breaks:

1) I added a quota to the external users, so that each can not use more than
64 slots at a single time. Each machine has 8 slots, so this equates to 8
full machines.

2) I resolved the issues of host prioritization, without having to add a
queue. When a user submits a job, they are scheduled as follows:
    - By default, the job will use a low memory node
    - If all low memory nodes are full, or the job requires more than 8GB of
RAM, it will use one of the faster high memory nodes (compute-3-x) so that
it ends ASAP.
    - If all compute-3-x nodes are full and the job needs more memory, then
it will be scheduled to a slower high memory node (compute-2-x).

In my testing, everything seems to work properly, but please let me know if
you see anything weird happening. If you want to know the gory
implementation details, I added them to
https://secure.genome.ucla.edu/index.php/Sun_Grid_Engine

Cordially,
Jordan