Re: [Discontig-devel] NUMA aware __alloc_pages function (3)

[Samuel O. <so...@db...>]

On Fri, 8 Feb 2002, Kimio Suganuma wrote:

> Hi Samuel,
>
> numa ratio is nothing to do with number of nodes.
This I understood. My first objective with that constant was having some
value that would define some memory shortage limit. When you pass this
limit, that means you have too many out of memory nodes around you, and
it's time for you to launch the swap daemons on those nodes.
I agree with you when you say that this is not the definition of the numa
ratio. However, I think it is strongly related with it, because the bigger
your numa ratio is, the closer to you your pages have to be. So the bigger
your numa ratio is, the bigger the limit represented by NUMA_RATIO is.
So, you're right, NUMA_RATIO is an inapropriate name. Maybe I should have
used something like SWAP_LAUNCH_POLICY. What do you propose ?

> So, it should be a config parameter or something.
Do you mean a boot-time parameter ?


> I think such the topic about NUMA aware memory scheduling
> was discussed in LSE before, so I feel we should ask
> LSE guys also. Any idea?
>
> Thanks,
> Kimi
>
> On Thu, 7 Feb 2002 18:44:23 -0800 (PST)
> Samuel Ortiz <so...@db...> wrote:
>
> > Hi Jack,
> >
> > I have been trying to think of a way to fix the problems that Jack and
> > Kimio have been pointing out the last couple days.
> >
> > Basically, the problem is :
> >  - A kswapd daemon is waken up as soon as its node is short on memory.
> >    As Kimio noticed, this behaviour shouldn't be hardcoded, since it
> >    really depends on the machine's numa-ratio
> >
> > As a consequence, We need a way to tune the memory allocation routine
> > behaviour according to the machine numa-ratio.
> > I have then introduced a constant called NUMA_RATIO.
> >
> > What is NUMA_RATIO ?
> > --------------------
> > NUMA_RATIO is a number that has to be between 1 and the number of nodes of
> > the machine. The bigger NUMA_RATIO is, the less you care about having your
> > memory away from the local node. The smaller NUMA_RATIO is, the more you
> > want pages to be close to your local memory.
> > For example, on a 4 nodes machine, if
> >     * NUMA_RATIO = 1, then as soon as a node is short of memory, its swap
> >       daemon wakes up and starts to work on the node.
> >       That means that you want to try hard to get pages on your local
> >       node. Thanks to the swap daemon, when a process looks for
> >       pages on a node, he finds some of them.
> >       This is the original behaviour of my patch.
> >     * NUMA_RATIO = 4, then all the swap daemons will wake up at the same
> >       time, when all the node are getting out of memory.
> >       Since nodes can get emptied, we don't really care about processes
> >       having to look far from their local node for pages.
> >       This is the old behaviour.
> >     * NUMA_RATIO = 2, then 2 swap daemons will wake up as soon as an
> >       allocation path finds at least 2 nodes short on memory.
> >       According to our numa-ratio, we can not afford processes to look for
> >       pages more than 2 nodes away.
> >
> > The modifications are made in __alloc_pages and have been tested on a
> > bigsur and a NUMA machie.
> >
> > Please let me know your thoughts about such a solution.
> > As usual, test, bugs, comments are more than welcome.
> >
> > Thanks in advance,
> > Samuel.
>
>

-- 
Thanks,
Samuel.