Menu

#649 ResPool service: sub-pools / entry-attributes

open
nobody
5
2010-07-29
2010-05-20
Amir Gonnen
No

We have a pool of machines that perform "tests". Some of these machines can also perform a "build". But a machine can only perform one task ("test" or "build") simultaneously.
When I submit a test I request a machine from the machine pool (any machine). When I submit a build I would like to request a machine from the same pool, but only a machine that can perform a "build". At any time no more than a single test/build could be performed on the same machine.
If I created a separate pool of "build machines" then these machines could not be used to perform tests and remain idle when there are no active builds.
Another reason to have a single pool is the fact that we use priorities when we request a machine and we would like to apply them also between "tests" and "builds".

My suggestion is to have "subpools" within a pool (perhaps any hierarchy of subpools).
Suppose pool A contains subpool B. If I request a machine from A I would get any available machine from A+B. If I request a machine from B I'll get only an available machine from B. This should take into consideration the priority of each request from either A and B.
Another way to achieve the same thing is the ability to specify a list of entries on a request (and not only a single entry). Such way, when I need to submit a "build", I could request a machine from my machine pool from a specific list of machines which are capable of performing a "build".

We can generalize this idea with "pool entry attributes". We could specify a list of attributes on each pool-entry when adding it the pool (attributes such as "build-machine" or "test-machine") and then when requesting an entry from the pool we optionally specify a list of attributes the entry must have to satisfy the request.

Since we need this capability now we are thinking of "emulating" it by requesting each build-machine from machines pool in parallel, and once any of these requests is fulfilled, cancel all other request. We consider this an inconvenient workaround to the problem, which could be elegantly solved with sub-pools or entry-attributes.

Discussion

  • Sharon Lucas

    Sharon Lucas - 2010-05-20

    We have a similar type of environment with test machines and build machines. We a mutex semaphore to make sure that only one build/test is active at a time on a machine, etc. The STAF SEM (Semaphore) service allows you to manipulate and manage mutex semaphores (as well as event semaphores). A mutex semaphore allows you to synchronize access to a particular resource. You may request exclusive access of the semaphore, release exclusive access of the semaphore, query the semaphore, and delete the semaphore. See section "8.15 Semaphore (SEM) Service" in the STAF User's Guide at http://staf.sourceforge.net/current/STAFUG.htm#HDRSEMSRV for more information. I think you'll see that this is exactly what you need. Post again to let us know if this works for you (or if you need more information on using mutex semaphores).

     
  • Amir Gonnen

    Amir Gonnen - 2010-05-23

    Let's say I have a mutex per machine and two resource pools, one for test machines and one for build machines. (some machines appear on both pools).
    Suppose all build machines are in use by "build" tasks and now the job submits a "test" task, and requests a "test" machine from the "test machines" pool.
    Since the build machines also appear in the "test machines" pool, there's a chance a machine that is already working on a build task is selected. In such case the test task will wait on the mutex until that machine finishes the build task, although there are other idle machines in the "test machines" pool that could perform the test right away.
    I think that mutex is not enough to solve the problem. It ensures only one task is performed on a machine, but as a side effect some tasks could be waiting on a mutex instead of working on some available machines, resulting in inefficient resource usage.
    Not to mention the fact that we need priorities between all tasks (tests and builds) as I mentioned above, so I guess they must wait in a single queue anyway.
    I don't see how mutex solves these problem, please explain.

     
  • Sharon Lucas

    Sharon Lucas - 2010-05-24

    Ok. It doesn't sound like using mutex semaphores in conjunction with a resource pool is going to do what you want.

    We've tried to keep the existing RESPOOL service fairly simple. But, we understand that some people need a more complex Resource Pool service, such as providing the ability to specify attributes for pool entries. For example, if a resource entry could be specified with attributes such as an operating system, an os version, whether certain applications were installed on it, etc., then someone could request the first available machine that is running Windows 7 with applications A and B installed, etc. However, once attributes are allowed to be specified, then the Resource Pool may require an underlying database to allow quick searches for entries that match the specified attributes. We haven't had the time yet to work on something like this yet.

    For now, you could try to emulate what you need as you mentioned. Or, you could create your own version of the ResPool service that provides the ability to support specifying attributes for entries (and contribute it back to the STAF open source project as a Patch to be incorporated in a future STAF version). As this could get pretty complex, it might be easier to add support to specify a list of entries on a request (instead of a single entry), though this is a less-elegant solution as you mentioned.

     
  • Sharon Lucas

    Sharon Lucas - 2010-07-29
    • summary: sub-pools / entry-attributes --> ResPool service: sub-pools / entry-attributes
     
  • sfresher

    sfresher - 2010-09-23

    Hi Sharon, we have the same/similar need. Indeed, it will be especially helpful if you can implement "support to specify a list of entries on a request (instead of a single entry)". So that we can utilize our database to find the matching entries, and request multiple of them in one staf command. And whichever first available will be assigned. e.g. poolA has entry machine1, machine2, and machine3. I can use the command "RESPOOL poolA REQUEST ENTRY machine2 machine3" to request for one of these two, which ever is available first. Can you please consider elevate this request?

     

Log in to post a comment.