Re: [Lse-tech] Many CPUs

[Paul M. <Pau...@us...>]

> Hi,
>
>          Anton alerted me to the many CPUs thread.  In fact, the
> hotplug CPU patch abstracts the bitmask (although it's still a bitmask
> on all architectures), so the correct way to iterate through CPUS is:
>
>          for (i = 0; i < NR_CPUS; i++) {
>                    if (!cpu_online(i)) continue;
>
>                    ...
>          }
>
>          If an archtecture had really sparse CPU numbers for some
> valid reason, we write a macro for every architecture:
>
>          for_each_cpu(i) {
>                    ...
>          }
>
> Of course, you *don't want* to do this as people still use NR_CPUS to
> allocate arrays...  You avoid sparse CPU numbers unless you can come
> up with a brilliantly good reason...

Yep!  The reasons thus far are (1) when a CPU has been removed from
service and (2) when you have a NUMA system where additional CPUs
might be added to the existing NUMA nodes.  For #2, one could argue
that it is better to completely fill out the nodes, since you get
better memory latencies with completely filled-out nodes.

But, if someone does have a NUMA system with partially full nodes,
gaps should be left in the numbering scheme so that you don't end
up with the CPU numbers all mixed together.

Does this sound reasonable?

                    Thanx, Paul

> Anyway, example for most archs (cpu_online_mask needs to be inverted,
> so 1 means offline, 0 means online):
>
> #define for_each_cpu(i) \
>    for (i = find_first_zero_bit(&cpu_online_mask,
sizeof(cpu_online_mask); \
>          i < NR_CPUS; \
>          i = find_next_zero_bit
(&cpu_online_mask,vsizeof(cpu_online_mask), i+1))