[Sbcl-devel] Memory maps (freebsd and openbsd)

[Raymond W. <Ray...@fa...>]

	A couple of days back I noticed that SBCL core-dumped when I
tried to run it on one of my "secondary" machines. After a bit of
investigation, it turned out that the reason for this, was that the
machine had a larger maximum data segment size (set by the MAXDSIZ
preprocessor symbol in the kernel config file).

	Further analysis led me to the kernel source file
sys/vm/vm_mmap.c, which contains the snippet

	/*
	 * XXX for non-fixed mappings where no hint is provided or
	 * the hint would fall in the potential heap space,
	 * place it after the end of the largest possible heap.
	 *
	 * There should really be a pmap call to determine a reasonable
	 * location.
	 */
	else if (addr == 0 ||
	    (addr >= round_page((vm_offset_t)vms->vm_taddr) &&
	     addr < round_page((vm_offset_t)vms->vm_daddr + MAXDSIZ)))
		addr = round_page((vm_offset_t)vms->vm_daddr + MAXDSIZ);

--- I take this to mean that any addresses specified (to mmap()) that
lie between the start of the "text" segment (where you'll normally find
the executable code) and the maximum value for the end of the "data"
segment, will be forced to a different address. The new address will
be somewhere after the end of the data segment.

	The reason I got the crash, was that my main machine has a
maximum data segment of 512MB, with the end of the "data" segment at
#x28000000. The other machine, with MAXDSIZ=1024MB, had the end of the
data segment at #x48000000. SBCL's "dynamic space" was set up as
#x48000000 - #x88000000.

	On FreeBSD, the dynamic loader uses mmap() to place the
dynamic libraries required by the sbcl executable. These libraries
will be placed at the earliest free space after the end of the data
segment, which would be #x28000000 or #x48000000, in my case. Note
that #x48000000 is the same place where sbcl wanted to place the
dynamic space, and this caused the crash.

	Given all this, it seems sensible to place all 5 segments
(read-only-space, static-space, binding-stack, control-stack and
dynamic-space) at an address

	- above the data segment
	- above any dynamically loaded libraries (libc and libm, at
	  least, for FreeBSD)
	- below the 1GB or so reserved for the kernel virtual space

	I've just tried these values for FreeBSD:

(macrolet
    ((def-segment (segment-name segment-start segment-end)
	 `(progn
	   (defconstant
	       ,(intern (concatenate 'string segment-name "-START")) 
	     ,segment-start)
	   (defconstant
	       ,(intern (concatenate 'string segment-name "-END"))
	     ,segment-end))))
  #!+freebsd
  (progn
    (def-segment "READ-ONLY-SPACE" #x50000000 #x57fff000)
    (def-segment "STATIC-SPACE"    #x58000000 #x5ffff000)
    (def-segment "BINDING-STACK"   #x60000000 #x67fff000)
    (def-segment "CONTROL-STACK"   #x68000000 #x6ffff000)
    (def-segment "DYNAMIC-SPACE"   #x70000000 #xaffff000)))

	Note that this gives all segments except dynamic-space 128MB
each, and 1GB for dynamic-space. read-only-space and static-space are
only 15MB and 4 MB each in a fresh sbcl, which indicates that these
segments could be reduced in size; it is likely that the stacks, too,
could be reduced somewhat. Experimentation also indicates that I could
make the dynamic space extend all the way up to #xbf000000 or so
(under FreeBSD, that is).

	I'm not sure that this mapping should be made the default, but
I think that the read-only-space (at #x10000000-#x1ffff000) should be
moved up with the other segments. Further, given that it may be
desirable to move the segments around to accommodate different
configurations, I think that it should be easier to change the segment
values. One way would be to specify the memory area that should be
used for all segments, and the size of each segment. This could be
done in compiler/target/parms.lisp (as it is today), or it could be
done in a file generated by the build process.

	//Raymond.

-- 
Raymond Wiker
Ray...@fa...