James McDonald <mcdonald@...> writes:
> I'm trying to use SBCL under a 32-bit (Red Hat) Linux for some large
> but seem to be limited to about 880 MBytes for the heap, apparently
> because some C
> code is deposited at 0x40000000, thus limiting the heap to space on
> either side of that.
> (1) Is there a straightforward way to build SBCL with the C code
> relocated elsewhere,
> or is there something about Linux that insists on using that
> prime real estate for
> program startup?
> [If it must be there to start SBCL, would it be feasible to start
> SBCL, dynamically
> relocate that code elsewhere, then allocate the heap?]
That piece of real estate is normally not in use. Can you try turning
off prelinking? (http://article.gmane.org/gmane.lisp.steel-bank.devel/8033)
> (2) Is there anything intrinscally preventing me from having a heap
> that spans,
> for example, from 0x10000000 to 0xFFFFFFFF?
> E.g., are there problems with heap addresses exceeding 0x80000000?
> If so, is this a localized issue or are there assumptions
> scattered all through the code?
> [I noticed there is a 64-bit version, so am hopeful large
> addresses are possible even
> in a 32-bit version.]
Normally Linux the highest 1GB of address space will be completely
unavailable to userspace processes, so getting a 4GB heap will be
completely impossible. You can recompile the kernel with different
splits, but some of the address space will always be lost.
Additionally there are indeed various assumptions in 32-bit sbcl that
you'll run into trouble with. The one thing I remember is relocation
information / fixups for compiled code: trying to move or load a code
object into the >2GB space causes problems. Possibly eq hash-tables
will run into trouble due to objects in high memory having a negative
> (3) If there is no straightforward solution, can anyone estimate how
> long it would take
> for an experienced lisp programmer (ages ago I wrote much of Lucid
> Common Lisp,
> including LAP, interrupt handlers, etc.) with minimal exposure to
> Linux and SBCL
> internals to make this possible? [days vs. weeks vs. months is
> close enough]
I'd expect days, but it depends on just what kind of muck turns up.
> (4) Assuming this was solved for Linux, would there be substantial
> additional work
> to solve it for Darwin and Windows as well?
While Linux doesn't formally make any guarantees about which bits of
the address space are free, it's in practice pretty deterministic
except for address space randomization (disabled by sbcl) and
prelinking. On Windows libraries will be mapped at completely
unpredictable locations, so some support for a disjoint heap would be
needed. (Which is theoretically easy enough: just mark some ranges as
unavailable in the gc page table. The tricky bit will be dumping and
Also note that SBCL uses a parly-copying GC. So even if you get a 3GB
heap, you can't really use all of it. Some space (the amount of which
would depend on your small/large object mix) needs to be left free, so
that the GC has room to copy object to. So you might really want to