#118 Determine VMemPool base at run-time
The repository's VMemPool subsystem has a configuration
setting ([Repository]VMemPool_base) which specifies the
base address at which a memory region will be created
(with mmap(2)) to hold the packed data structures which
represent directories and attributes in the repository.
This setting is problematic because picking a value
works depends on knowing something about the memory
layout of a process on the host OS. So far we've
gotten by with one setting for Alpha on both Tru64 and
Linux (0x26600000000) one for Linux on 32-bit
architectures (0x60000000) and one for Linux on x86-64
(0x6600000000). It would be much better if this value
could be automatically chosen.
It's worth noting that recent security-related OS
innovations are going to make it harder to do this.
OpenBSD recently started randomizing the placement of
mapped regions:
http://kerneltrap.org/node/5584
Another problem with this has to do with expanding the
VMemPool region. When the repository needs more space
(because of newly created directory structure or
attributes), it simply expands the VMemPool region by
mmap-ing more space directly above the current
VMemPool. If this fails (for instance because
something else is mapped there, such as a shared
library or the stack of a thread) the repository will
die with an assertion failure. It would be better for
it to instead try to relocate the entire VMemPool heap
to a new location. We would probably have to deal with
this at the same time as the above, as is we allow the
OS to place the VMemPool heap arbitrarily then
expanding it will probably be even less likely to succeed.
Implementing relocation of the VMemPool heap would
require some care, as currently threads in the
repository freely translate VMemPool "short pointers"
(32-bit quantities) to real addresses when they need to
access the packed data structures. We would need some
new mechanism to (a) wait until all threads are done
using real pointers into the VMemPool heap and (b)
prevent new threads from creating them. We could do
this with a ReadersWritersLock for the VMemPool base
address, but that would require adding calls to acquire
and release read locks on that in many places.
