Re: [Lse-tech] Re: NUMA: Simple binding API, rev 0.x

[Niels C. <nc...@us...>]

Michael Hohnbaum wrote:
-----------------------

= Sounds like a dangerous printout.  I would recommend destroying it.

Done!

= > All binding calls are to be issued by the task to be bound.
= > Ergo, you must have an interface to the task to tell it to
= > bind.  The application, whose task you want to bind, must have
= > code to accept a signal (or whatever) and code to execute
= > the binding functions.

= I suspect that the model you are envisioning is more of a
= workload management model, where an outside entity (workload
= manager) determines that the system would benefit from moving
= processes to other resources and binding them there.  That
= is an entirely different beast from what we are working on.

You bet.  I sort of dislike the idea that only applications
that are members of a fraternity can make efficient use of NUMA.

It is my understanding that the primary goal for improving Lunix
NUMA support is to make it work well for applications who don't
even know about the fraternity.

= Binding of a process to resources needs to happen early in
= the life of a process, before memory for the process is
= populated on the node.  If the process is not to have knowledge
= of its resource bindings, then launch policies can be used.
= If the process knows what type of resource binding it needs,
= these APIs provide this.  This is not something that most
= casual processes would use.  However, large, long running
= processes benefit from this type of binding.

Sure, which is why I suggested the API functions that would
allow you to make suggestions rather than issue orders and to
do it so the affinity is relative.  Wouldn't work very nicely
with cpumemsets, which doesn't acknowledge the idea of nodes,
would it?  But, frankly, that doesn't bother me.  I look with
a lot of suspicion on the combined chain saw, jackhammer and
surgical instrument that would be required to make efficient
(read.: intelligent) use of cpumemsets.

= It sounds to me like that how you envision binding to work is
= for an outside process to decide, at some arbitrary time, that
= a process should be bound to a set of resources.
= Moving a process from one set of resources to another at random
= times, except for extreme load balancing situations, does not
= typically gain much, and can be detrimental to system performance.

Yes and no.  Yes to relative binding and launch binding.  Yes to
the ability to issue directives to already running tasks to
allow a task group to consolidate its work on a node-centric
set of processors when it can do so without exceeding some spending
limit.  Yes to re-bind your fraternity database tasks when there
is a need to do other work at the same time.

I do not expect to see all this in your first API but I do expect
it to be ready for it.  Hence the need for a task group argument.

Paul Jackson wrote:
-------------------

= > some common understanding is reached as to what distance is,
= > it does not make sense to define higher-level APIs based upon it.

= Note Neils what Michael said here -- he is assuming
= we will define higher-level APIs, when the time is ripe.

I hear you but I think the API as presented is too narrow.  All belt
and no pants.  Not that it matters what I think.

= > ... t may be a lot of fun to work with actual processor numbers
= > (whether physical or logical) and actual node numbers but it
= > is not a way to run an efficient shop.

= Then don't use the "actual processor number" API ...
= Wrong tool for the job.

Which tool should I use then, pj?

= > Seems to me that it would make more sense to add a task-ID
= > or maybe task-group-ID (thinking of NGPT) to all the bind
= > functions ...
=
= Hmmm ... are you trying to make a chain saw out of a hand
= saw again, Neils?

It is Niels, not Neils.  If "nc" is easier, use that.  That'll
be what I use in the future.  No, pj, I'm just trying to look
beyond the tip of my nose, which isn't very far, actually.

-------------

I didn't really want to but I felt I had to respond.  Now it is
done and I'll jump back on my lily pad and watch you guys splash
in the community pool, skinny legs and all :-)

-nc-