[kbuild-devel] [RFC][LONG] Kbuild changes to support early userspace packages

[Kevin P. F. <kpf...@co...>]

I am working with the klibc people and Greg KH on early userspace stuff for 
kernel 2.5.X. One of the issues I'm trying to address is how end users are going 
to incorporate "off the shelf" packages of tools into their initramfs. If the 
kbuild system doesn't provide some support for that, then the whole 
initramfs/early userspace system will be dependent on tools being rewritten and 
shipped in the kernel tarball, and many tools will never have that done.

To this end, I think I have a solution, but I'd like some input from you kbuild 
masters :-)

OK, here goes:

All the early userspace stuff will happen in the "usr" subtree of the kernel 
source tree. As it stands, usr contains a Makefile but no Kconfig as there is 
nothing configurable in there. The klibc patch for 2.5.64 adds a usr/root 
directory (with a sample program) and some other bits to make building an 
initramfs work reasonably well.

I am proposing that packages the user wants to incorporate into their source 
tree will have to be specially prepared; the package will have to include a 
directory called "kbuild" at the top level of its tarball. This directory would 
contain a Kconfig (if the package has any configuration options), a Makefile, 
and a Makefile fragment to be included into usr/Makefile (so that subdirs:= can 
be set properly). Obviously, for an "off the shelf" package, i.e. mdadm, the 
Makefile in the kbuild directory would be completely independent of any 
Makefile/configure/etc. process the package would normally use. Hopefully these 
files will be simple enough that they can be maintained by the package's normal 
maintainer, and thus end users can just download the tarball and unpack it into 
usr/pkgs in the source tree.

Note that I specifically did not mention the package installation including any 
patching of the kernel's source tree; I don't think that's a viable solution, 
because inclusion of two (or three, or more) packages would make patching very 
difficult.

What I would like to accomplish is to have the kernel's config and build process 
automatically find the package that has been put there and include it. Here's 
how I see that working:

arch/${ARCH}/Kconfig does a "source usr/Kconfig" (probably at the end)

usr/Kconfig does a "source usr/Kconfig.pkgs"

usr/Kconfig.pkgs is a target in usr/Makefile, dependent on usr/pkgs and 
$(wildcard usr/pkgs/*/kbuild/Kconfig), with a command list that rebuilds 
usr/Kconfig.pkgs to have lines of the form "source <foo>" for each source object 
that make found

usr/Makefile does an "include $(wildcard usr/pkgs/*/kbuild/Makefile.subdir)" or 
something similar, so the resulting combined usr/Makefile has the appropriate 
subdir specifications for kbuild to descend into the packages

So, on the first run with a clean tree, make will try to make usr/Kconfig.pkgs 
since it doesn't exist. It will be created, but empty if the user hasn't 
installed any optional packages into the usr subtree.

On a subsequent run, the kconf dependency file generated during the first run 
will include all the Kconfig files found in usr/pkgs/*/kbuild/*, so if any of 
those are changed the right thing will happen. If any are added, the make target 
rule for usr/Kconfig.pkgs will notice that usr/pkgs has changed and rebuild 
usr/Kconfig.pkgs, which is also in the kconf dependency list. If any are 
removed, again usr/pkgs will have changed and usr/Kconfig.pkgs will get 
regenerated. Obviously, if the user actually adds or removes files directly in 
usr/pkgs/*/kbuild the process can break, but they shouldn't have any reason to 
do that.

I have a rough version of this working now, but it required modifying the 
top-level Makefile to make usr/Kconfig.pkgs a dependency of something (I chose 
the menuconfig target for testing), otherwise make will never cause 
usr/Kconfig.pkgs to be made.

I'm looking for criticism and suggestions, especially if you think this is 
unworkable for some reason. I think it should have little to no downside if 
someone never puts anything into usr/pkgs, but if they do it would be very nice 
to have it automatically picked up and not require patching of anything that 
came in the kernel tarball (meaning subsequent kernel patches could be applied 
without causing unnecessary rejects).