Re: [Lse-tech] RE: node aware slab allocator for cc-numa systems

[Manfred S. <ma...@co...>]

Thanks for your data.
The local/foreign ratio varies quite a bit:
The friendliest cache is the names cache (used to hold the file name for 
all syscalls that take a filename as a parameter):  less than 1% foreign.
Some are quite bad - the journal head cache has a foreign rate of 43%, 
nearly the 50% for completely random on a 2 node system.

                         :  :  : alloc_hit alloc_miss free_hit free_miss 
foreign.
journal_head      :  :  : 123109   1294  69474    416  53643
sigqueue             : : :   3456     52   2139      0   1364
names_cache      : : : 303906    388 301862      3   2385
buffer_head        : : : 158503   3748  51207    358  16226
(foreign rate is forein/(forein+free_hit+free_miss))

New question:
Should the "numa object return" be enabled on all caches, or only on some?
I bet that journal heads or dentries are touched by all cpus in the 
system, they are long-term objects. Is it the right thing to return them 
to the home node? Returning objects means spinlock operations for the 
return, additional spinlock operations to refill the arrays, etc. And it 
wont help, we'll trash cachelines anyway.

--
    Manfred