[Lse-tech] Re: NUMA: Simple binding API, rev 0.3

[Paul J. <pj...@en...>]

Michael wrote:
|> >
|> > 7) You state:
|> >
|> >     ... the binding in effect for the process or thread that
|> >     executes the first page fault on a given page of memory
|> >     determines the binding for that page ... see the rationale
|> >
|> >    I don't see any "rationale".  And I'd think it would be
|> >    more stable to have the creator of the memory region determine
|> >    its memory policy, not the first faulter.
|> 
|> The "rationale" is a link.  Click on it and it will take you to
|> a separate document that has, amongst other things, a section
|> titled "Conflicting Memory Bindings".  It also has quite a bit
|> of discussion and examples, of the processor/node numbering
|> scheme.

Ok - now that I read this rationale, my question still stands.

You list two possible approaches therein to specifying the
memory policy on a shared region:
  1. a global policy, where the last binding wins
  2. a local policy, where the first page fault wins.

Then you recommend choice 2.

I see a third possible choice:
  3. a per vm area policy that can differ per faulting cpu

This results in each vm area linking to a (usually shared)
structure (list of struct of list) that specifies, for any
fault on any given cpu, for that given vm area, which nodes
to search for memory, in what order.

This is what cpumemsets has.  It determines the policy
deteministically up front, rather than based on some
random accident such as the location of the first fault.

I'd recommend choice 3 (big surprise).

-- 
                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <pj...@sg...> 1.650.933.1373