RE: [Lse-tech] topology diagram

[Patrick M. <mo...@os...>]

Hi there.

> Paul Dorwin's topology work clearly has stated that it is focused on
> describing the CPU and memory topology of the system, and plans
> to follow on with support to then connect the I/O busses into the
> processor/memory topology.  There is no plan to expand this into
> a topology description for the I/O subsystem, capturing all of the
> controllers, devices, etc.  There are already multiple projects
> covering that space.  It seems like much of the discussion,
> especially from Niels Christiansen and Patrick Mochel, is focused
> on topology description of the I/O subsystem.
>
> As Paul Dorwin has stated, the CPU and memory topology does not
> readily map into the devfs or driverfs as those mechanisms are
> driver oriented.  There is no concept of a CPU driver or a memory
> driver (and I would prefer to not go there).  However, if Paul
> Dorwin's topology description goes down to the I/O bus level, then
> there is, to a certain extent, a connection to driverfs.  Does it
> make more sense to extend driverfs upwards to incorporate the
> processors and memory?  Or should these topology descriptions,
> since they, for the most part serve very distinct purposes, be
> kept as two separate entities?  Or possibly a hybrid where the
> in-kernel description/structures are separate from driverfs, but
> the user-level interface is via driverfs?

Yes, devfs and driverfs are driver-oriented. But, as I mentioned in a
previous email, driverfs attempts to address the data/control of
non-standard device semantics.

CPUs and memory are not "devices" in the sense that you can open(2),
lseek(2), and write(2) them. But, there is data to be extracted from them,
and there is some amount of control that you may want to do at runtime
(like taking a CPU or memory bank offline).

That is the kind of stuff that procfs has been (ab)used to do, and the
exact thing that driverfs was written for. An effective "ioctl"
replacement on the device, without having to implment a special /dev node.

So, in this sense, I think there should be a "driver" for CPUs and memory
banks. Even if you don't call it a "driver", there will still be something
like it that exports the data to userspace. I'm just suggesting to follow
the semantics in the driver model and put in the filesystem. :)

> ...
> > > Finally, just to show that this is not Numa specific, this could
> > > easily be applied to a standard SMP system. In this case,
> > > the Node 0 Container would become the only container in the
> > > system. The system container would then contain the object ids
> > > of the 4 cpus and single memory block. This is shown in figure 3 of
> > > the diagrams.
> >
> > I would prefer to see the good old SMP just like I see the NUMA box,
> > except that the system container has only one node container to
> > point to (and a fuel tank, of course).
>
> The person coding up the architecture representation has the flexibility
> to make that choice.  It seems to me creating a one node container to
> represent an old fashioned SMP system seems like it is forcing a NUMA
> concept where it is not necessary.  However, either approach is valid.

Make it implicit with the compile time support. If it is a NUMA box,
create tree nodes for each quad in the system, then build off of that. If
it's not, don't create the quad nodes (node nodes?).

	-pat