Steel Bank Common Lisp

Dave Roberts wrote:
> Hi, Rob,
> 
> On Wed, 2005-08-03 at 07:53 -0400, Rob MacLachlan wrote:
> 
>>Maybe nobody is worried about this kind of efficency issue anymore, but 
>>I don't like the idea of relocating the heap on two grounds:
>>  1] the startup time required to relocate everything
> 
> 
> This is no more than the GC time of the memory occupied by the core
> file, right?

It's a lot worse, more like purify, since you're dirtying the entire 
heap.  In normal full GC (as opposed to incremental, which does even 
better) you only have to copy dynamic space and scan static space. 
read-only space (the largest due to code and constants) is entirely ignored.


>>To me, the idea of getting the unix loader to do the mapping sounds 
>>promising, though I have no idea if it will work.
> 
 >you need a 32-bit offset in the relocation
 > table for every pointer in the core file. And in a Lisp system, that
 > could be close to every other word of the core (okay, not that bad, >but
 > *lots* of pointers). Thus, your relocation table could be as large as
 > your core file is, effectively doubling the size of a core file

The pointer content is more like 20%, at least when I last checked.  But 
I thought someone was raising the possibility that we could still get 
fixed allocations if we did it via the unix loader.

I admit I don't understand the randomization approach, but this sounds 
insane to me.  Is lisp really the only application getting gored?  I 
would think that any application that used a heap-save approach would be 
in deep doo-doo.  What about emacs, for example?  What about any number 
of persistent object systems?

   Rob

On Wed, Aug 03, 2005 at 09:07:19PM -0700, Dave Roberts wrote:
> That might work. Basically just request an address through mmap, but
> don't use the MAP_FIXED flag. Although in a randomized memory system
> you'll almost always get something different than what you requested.

The mmap randomization doesn't actually work by assigning the address
at random, but by selecting the highest memory range below a certain
base value for the mmap. It's this base value which is randomized by a
modest amount. So if you're requesting memory that's not close to the
mmap base, you're likely to get it. It's unlikely that even the
lunatic fringe would actually do completely random mmaps. And this
isn't really a problem for SBCL; it runs just fine on the latest
vanilla kernels that do this randomization. 

(The Fedora kernel has patches that do additional randomization which
causes problems for SBCL, but they're similar in spirit. The initial
allocation of some spaces is offset a bit).

-- 
Juho Snellman

On Thu, Aug 04, 2005 at 08:55:39AM +0100, Christophe Rhodes wrote:
> From where I'm sitting, the inability to turn off randomization
> per-process, or to customize the randomness, is therefore a terrible
> misfeature: defeating perfectly reasonable programs before they've
> even started.

Turning the randomization off per-process seems to be possible (patch
attached, testing welcome).

-- 
Juho Snellman

Juho Snellman wrote:
> On Thu, Aug 04, 2005 at 08:55:39AM +0100, Christophe Rhodes wrote:
> > If the answer to this is having to play ELF games with our executable,
> > to reserve space before the kernel and linker decide where they're
> > going to live and to put shared libraries, then so be it; something
> > akin to Brian Mastenbrook's ELF-grovelling Darwin trick can probably
> > be adapted, no?
> 
> I had a look at this. Due to kernel bugs it's apparently impossible to
> reserve space without running into overcommit problems, no matter what
> elf tricks you play. For example the attached patch adds a @nobits
> section into the data segment and moves the spaces inside the section.
> Should work like a charm, but the kernel actually counts that space
> reservation against the system overcommit limit.

This is not a kernel bug but simply the definition of a LOADed NOBITS
section. It allocates the defined .space statically.


Thiemo

On Thu, Aug 04, 2005 at 06:48:01PM +0200, Thiemo Seufer wrote:
> > I had a look at this. Due to kernel bugs it's apparently impossible to
> > reserve space without running into overcommit problems, no matter what
> > elf tricks you play. For example the attached patch adds a @nobits
> > section into the data segment and moves the spaces inside the section.
> > Should work like a charm, but the kernel actually counts that space
> > reservation against the system overcommit limit.
> 
> This is not a kernel bug but simply the definition of a LOADed NOBITS
> section. It allocates the defined .space statically.

Is there a way to reserve VM address space without implying that it will
necessarily all be filled for purposes of kernel memory estimation?

-bcd

Brian Downing wrote:
> On Thu, Aug 04, 2005 at 06:48:01PM +0200, Thiemo Seufer wrote:
> > > I had a look at this. Due to kernel bugs it's apparently impossible to
> > > reserve space without running into overcommit problems, no matter what
> > > elf tricks you play. For example the attached patch adds a @nobits
> > > section into the data segment and moves the spaces inside the section.
> > > Should work like a charm, but the kernel actually counts that space
> > > reservation against the system overcommit limit.
> > 
> > This is not a kernel bug but simply the definition of a LOADed NOBITS
> > section. It allocates the defined .space statically.
> 
> Is there a way to reserve VM address space without implying that it will
> necessarily all be filled for purposes of kernel memory estimation?

Allow memory overcommit.


Thiemo

On Thu, Aug 04, 2005 at 08:43:36PM +0200, Thiemo Seufer wrote:
> > Is there a way to reserve VM address space without implying that it will
> > necessarily all be filled for purposes of kernel memory estimation?
> 
> Allow memory overcommit.

Right...

Since that requires root privileges and frobbing a system-wide knob, I
assume that means "no" from a standpoint useful to distributing SBCL
binaries that work everywhere.  :(

-bcd

Brian Downing wrote:
> On Thu, Aug 04, 2005 at 08:43:36PM +0200, Thiemo Seufer wrote:
> > > Is there a way to reserve VM address space without implying that it will
> > > necessarily all be filled for purposes of kernel memory estimation?
> > 
> > Allow memory overcommit.
> 
> Right...

Well, reserving some memory space _means_ to either have it backed by
memory/swap or to allow memory overcommit. If an admin insists in a
no-overcommit policy then his system has to provide the means for it.

> Since that requires root privileges and frobbing a system-wide knob, I
> assume that means "no" from a standpoint useful to distributing SBCL
> binaries that work everywhere.  :(

The alternative is to allow discontiguous dynamic memory and allocate
it on demand. This needs some serious work on the GC implementation
and nobody felt compelled to do it so far.


Thiemo

<ths@...> wrote:
> Brian Downing wrote:
>> On Thu, Aug 04, 2005 at 08:43:36PM +0200, Thiemo Seufer wrote:
>> > > Is there a way to reserve VM address space without implying that it will
>> > > necessarily all be filled for purposes of kernel memory estimation?
>> > 
>> > Allow memory overcommit.
>> 
>> Right...
> 
> Well, reserving some memory space _means_ to either have it backed by
> memory/swap or to allow memory overcommit. If an admin insists in a
> no-overcommit policy then his system has to provide the means for it.

Sure, but there are actually three overcommit settings:

  0: "The kernel attempts to estimate the amount of free memory left
     when userspace requests more memory." (default)
  1: Infinite overcommit
  2: No overcommit
  
AFAIK, reserving memory with a @nobits section in a PT_LOAD segment
counts as memory use for the heuristic that's used for case 0, while
reserving it with mmap doesn't. I feel this is a bug (from our point
of view there's no difference between the two cases), but I'm willing
to accept that it's really working as intended. After all:

 <jsnell> minion: advice about heuristic
 <minion> #11953: Of course, this is a heuristic, which is a fancy way
          of saying that it doesn't work.

-- 
Juho Snellman

On Thu, 2005-08-04 at 09:16 -0400, Rob MacLachlan wrote:
> Dave Roberts wrote:
> > Hi, Rob,
> > 
> > On Wed, 2005-08-03 at 07:53 -0400, Rob MacLachlan wrote:
> > 
> >>Maybe nobody is worried about this kind of efficency issue anymore, but 
> >>I don't like the idea of relocating the heap on two grounds:
> >>  1] the startup time required to relocate everything
> > 
> > 
> > This is no more than the GC time of the memory occupied by the core
> > file, right?
> 
> It's a lot worse, more like purify, since you're dirtying the entire 
> heap.  In normal full GC (as opposed to incremental, which does even 
> better) you only have to copy dynamic space and scan static space. 
> read-only space (the largest due to code and constants) is entirely ignored.

Gotcha. I'll have to go read that code. I'll admit to not really
understanding what happens during purify.

> >>To me, the idea of getting the unix loader to do the mapping sounds 
> >>promising, though I have no idea if it will work.
> > 
>  >you need a 32-bit offset in the relocation
>  > table for every pointer in the core file. And in a Lisp system, that
>  > could be close to every other word of the core (okay, not that bad, >but
>  > *lots* of pointers). Thus, your relocation table could be as large as
>  > your core file is, effectively doubling the size of a core file
> 
> The pointer content is more like 20%, at least when I last checked.  But 
> I thought someone was raising the possibility that we could still get 
> fixed allocations if we did it via the unix loader.

Okay, that's better. We still may be better off doing it ourselves.

That said, there are three techniques that I can think of. Let me try to
summarize what has been proposed:

1. We request a memory region via mmap and relocate the core file as we
load it. Dumping a core works roughly as today, modulo any small changes
needed to ease the later relocation.

2. We modify the SBCL ELF to reserve the memory spaces we need, but we
don't actually put anything into the ELF. The spaces have a @nobits
section. This keeps the memory randomizer in the kernel off our backs
for the space we want. We load the core file into the reserved memory as
we do today. Juho Snellman said he tried this and that it causes
problems in a system with smaller RAM than we want to reserve unless you
set /proc/sys/vm/overcommit_memory. IMO, this is just trading on problem
for another.

3. When we dump a core, we actually link the core file into an SBCL
executable with an ELF segment that specifies the properties we want.
The segment is non-relocatable. The issue I see here is that if we
reserve one or more large segments, large enough for the whole heap, not
just the actual core file, we suffer the same problem as Juho found with
#2.

4. As in #3, but the ELF segments for the SBCL memory regions are stored
in relocatable segments. We use the ELF loader to do our pointer
arithmetic. Again, this may suffer the same problem as #2.

5. As in #4, but we let the ELF loader relocate things where it wants,
then we go through and perform the pointer arithmetic ourselves. Again,
this may suffer the same problem as #2.

Are there any more that I have missed?

> I admit I don't understand the randomization approach, but this sounds 
> insane to me.  Is lisp really the only application getting gored?  I 
> would think that any application that used a heap-save approach would be 
> in deep doo-doo.  What about emacs, for example?  What about any number 
> of persistent object systems?

Emacs got screwed, too, but only when they performed the dump itself. I
guess they handle the reloading of their heap differently. I don't
understand why, exactly.

-- 
Dave Roberts <ldave@...>

On Thu, 2005-08-04 at 08:55 +0100, Christophe Rhodes wrote:

> Well, firstly, an allocation algorithm which relies on having a large
> amount of contiguous space (such as one supporting a simple semispace
> garbage collector) is doomed in this brave new world of memory
> randomization -- it is perfectly possible that, on a given run, there
> would not be room _at all_ in the address space to put a 512Mb heap,
> let alone the 1.5Gb heaps that I hear people want to use with SBCL.
> >From where I'm sitting, the inability to turn off randomization
> per-process, or to customize the randomness, is therefore a terrible
> misfeature: defeating perfectly reasonable programs before they've
> even started.

I agree. I told Arjan that not having a per-binary switch for this is a
problem. He said that he wanted one but the folks on the LKML thought
there wasn't a need and they shot it down (when randomization was part
of Red Hat's exec-shield, it was subject to turning on all of exec-
shield for the binary, which meant you had to explicitly ask for it by
compiling your executable as a PIE). I have a feeling that a Common Lisp
system isn't going to be weighty enough to reverse that decision, given
the way 95% of the programming world views Lisp.

Now, that said, Arjan also explained to me that the technique of using
MAP_FIXED to map large areas of memory at fixed locations is also
inherently brittle. He repeatedly stressed that any changed to just
about anything, from the kernel to a new version of libc, etc., could
break this. It wouldn't take memory randomization in the kernel. From
comments made on this list by others, it sounds like this has been a
common problem in the past.

Note that the other guys using GC, like Java, Perl, Python, etc., don't
seem to be suffering. So SBCL is obviously doing something different in
terms of the way it implements its GC. I'm not necessarily saying that
we're doing something "wrong" and they're right, only that it's hard to
argue your case when the majority of other folks seem to be succeeding.

> This doesn't mitigate the fact that I think that
> default address-space randomization is a terrible idea; 

I agree. They should have a switch. Persistent, per binary. Either
default to on or off, I don't care, but at least allow me to tell you
that I can't handle it and I want the old behavior. I made all those
arguments to Arjan.

-- 
Dave Roberts <ldave@...>

On Thu, 2005-08-04 at 16:41 +0300, Juho Snellman wrote:
> On Wed, Aug 03, 2005 at 09:07:19PM -0700, Dave Roberts wrote:
> > That might work. Basically just request an address through mmap, but
> > don't use the MAP_FIXED flag. Although in a randomized memory system
> > you'll almost always get something different than what you requested.
> 
> The mmap randomization doesn't actually work by assigning the address
> at random, but by selecting the highest memory range below a certain
> base value for the mmap. It's this base value which is randomized by a
> modest amount. So if you're requesting memory that's not close to the
> mmap base, you're likely to get it. It's unlikely that even the
> lunatic fringe would actually do completely random mmaps. And this
> isn't really a problem for SBCL; it runs just fine on the latest
> vanilla kernels that do this randomization. 
> 
> (The Fedora kernel has patches that do additional randomization which
> causes problems for SBCL, but they're similar in spirit. The initial
> allocation of some spaces is offset a bit).

Okay, gotcha. So it isn't as bad as completely randomizing every call to
mmap. That's better than I thought.

-- 
Dave Roberts <ldave@...>

On Thu, 2005-08-04 at 12:14 +0200, Thiemo Seufer wrote:
> Dave Roberts wrote:
> > Hi, Rob,
> > 
> > On Wed, 2005-08-03 at 07:53 -0400, Rob MacLachlan wrote:
> > > Maybe nobody is worried about this kind of efficency issue anymore, but 
> > > I don't like the idea of relocating the heap on two grounds:
> > >   1] the startup time required to relocate everything
> > 
> > This is no more than the GC time of the memory occupied by the core
> > file, right?
> 
> It adds more relocation types (and thus fixups) than are currently
> needed in the corefile loader. Worse, on each GC memory move those
> fixups need to be adjusted.

I'm not sure what algorithm you're talking about here? The one were the
loader does the fixups or we do? Either way, there seems to be a method
to simply tell the loader to add a fixed offset to every pointer in the
segment and be done with it. That doesn't require any other processing
after the core is in memory and relocated. You would do GC just like you
do now.

(Reading between the lines, I think you're thinking of the method that
ELF binaries use to typically relocate code in shared library segments.
In this case, there is a jump table that provides a level of
indirection. We would not use this, I think. Once we get our heap in
memory and all the pointers are consistent, I think we're done.)


> Or probably not, since it is mostly a workaround for security issues
> caused by the combination of incompetence and primitive implementation
> languages. It makes less sense to waste performance with memory
> randomization it the implementation language makes it hard to e.g.
> create buffer overflows.

Agreed completely. I said that to Arjan as well. Essentially this is
making up for sloppy C coding. But when you're a kernel programmer and
all you have is gcc, everything looks like a nail... ;-)

> [snip]
> > But as Brian pointed out, you need a 32-bit offset in the relocation
> > table for every pointer in the core file. And in a Lisp system, that
> > could be close to every other word of the core (okay, not that bad, but
> > *lots* of pointers). Thus, your relocation table could be as large as
> > your core file is, effectively doubling the size of a core file.
> 
> The usual way around this is to add one more level of indirection.
> Make pointers relative to the GC memory region, and add a constant
> (from a known memory location or a global pointer register) when
> dereferencing them.

Yea, but that's just painful for performance. Lisp already does a lot of
indirection when jumping to subroutines, etc., but this would force it
on every load or store to the heap. That's just excruciating.

-- 
Dave Roberts <ldave@...>

On Thu, 2005-08-04 at 14:51 -0500, Brian Downing wrote:
> On Thu, Aug 04, 2005 at 08:43:36PM +0200, Thiemo Seufer wrote:
> > Allow memory overcommit.
> 
> Right...
> 
> Since that requires root privileges and frobbing a system-wide knob, I
> assume that means "no" from a standpoint useful to distributing SBCL
> binaries that work everywhere.  :(

Yea, IMO, that's just as bad as forcing somebody to turn off memory
randomization.

-- 
Dave Roberts <ldave@...>

Dave Roberts wrote:
> On Thu, 2005-08-04 at 12:14 +0200, Thiemo Seufer wrote:
> > Dave Roberts wrote:
> > > Hi, Rob,
> > > 
> > > On Wed, 2005-08-03 at 07:53 -0400, Rob MacLachlan wrote:
> > > > Maybe nobody is worried about this kind of efficency issue anymore, but 
> > > > I don't like the idea of relocating the heap on two grounds:
> > > >   1] the startup time required to relocate everything
> > > 
> > > This is no more than the GC time of the memory occupied by the core
> > > file, right?
> > 
> > It adds more relocation types (and thus fixups) than are currently
> > needed in the corefile loader. Worse, on each GC memory move those
> > fixups need to be adjusted.
> 
> I'm not sure what algorithm you're talking about here? The one were the
> loader does the fixups or we do? Either way, there seems to be a method
> to simply tell the loader to add a fixed offset to every pointer in the
> segment and be done with it. That doesn't require any other processing
> after the core is in memory and relocated. You would do GC just like you
> do now.

E.g. the static symbols/funs don't live at a constant memory location
any more, which means either to add an indirection or to fix up the
offsets. Which means either to use relative pointer relocations
(I think R_386_RELATIVE in ELF parlance) and to adjust them on GC,
or to introduce an additional absolute relocation (R_386_32).

Other architectures load pointers via immediates in the code, this adds
some more relocation types. The linux 2.6 module loader does roughly
the same job, but without the GC complication, for well specified ELF
relocations, and often for C code which is supposed to be relocatable.
Just have a look to find out how much fun it is.

> (Reading between the lines, I think you're thinking of the method that
> ELF binaries use to typically relocate code in shared library segments.
> In this case, there is a jump table that provides a level of
> indirection. We would not use this, I think. Once we get our heap in
> memory and all the pointers are consistent, I think we're done.)
> 
> 
> > Or probably not, since it is mostly a workaround for security issues
> > caused by the combination of incompetence and primitive implementation
> > languages. It makes less sense to waste performance with memory
> > randomization it the implementation language makes it hard to e.g.
> > create buffer overflows.
> 
> Agreed completely. I said that to Arjan as well. Essentially this is
> making up for sloppy C coding. But when you're a kernel programmer and
> all you have is gcc, everything looks like a nail... ;-)

I wouldn't blame the kernel developers but rather RH customers who
demand a way to keep their crappy applications running without fixing
them.

> > [snip]
> > > But as Brian pointed out, you need a 32-bit offset in the relocation
> > > table for every pointer in the core file. And in a Lisp system, that
> > > could be close to every other word of the core (okay, not that bad, but
> > > *lots* of pointers). Thus, your relocation table could be as large as
> > > your core file is, effectively doubling the size of a core file.
> > 
> > The usual way around this is to add one more level of indirection.
> > Make pointers relative to the GC memory region, and add a constant
> > (from a known memory location or a global pointer register) when
> > dereferencing them.
> 
> Yea, but that's just painful for performance. Lisp already does a lot of
> indirection when jumping to subroutines, etc., but this would force it
> on every load or store to the heap. That's just excruciating.

I know only about the SBCL MIPS implementation in-depth, and there it
is roughly similiar as for C PIC code. I think same goes for all
architectures which can afford to reserve a register as global pointer.
That's everything except x86, x86-64 should be ok (modulo current
implementation insufficiencies).


Thiemo

Thiemo Seufer wrote:
>>Since that requires root privileges and frobbing a system-wide knob, I
>>assume that means "no" from a standpoint useful to distributing SBCL
>>binaries that work everywhere.  :(
> 
> The alternative is to allow discontiguous dynamic memory and allocate
> it on demand. This needs some serious work on the GC implementation
> and nobody felt compelled to do it so far.

The "allocate on demand" is more or less what the lazy patches do. The 
discontinuous dynamic memory is almost done on x86 as part of the 
threading implementation, AFAIU.

On the whole I think that for the moment we can get by with just moving 
the spaces round a bit, as shown by Juho. In the future we should maybe 
think of having a more general memory allocation for the dynamic space 
and making the static and read-only spaces also relocatable.

As I said in the whole lazy-patches story: looking like a C program is 
becoming a necessary evil :-S

Groetjes, Peter

-- 
signature -at- pvaneynd.mailworks.org
http://www.livejournal.com/users/pvaneynd/
"God, root, what is difference?" Pitr | "God is more forgiving." Dave 
Aronson|

Juho Snellman wrote:
> <ths@...> wrote:
> > Brian Downing wrote:
> >> On Thu, Aug 04, 2005 at 08:43:36PM +0200, Thiemo Seufer wrote:
> >> > > Is there a way to reserve VM address space without implying that it will
> >> > > necessarily all be filled for purposes of kernel memory estimation?
> >> > 
> >> > Allow memory overcommit.
> >> 
> >> Right...
> > 
> > Well, reserving some memory space _means_ to either have it backed by
> > memory/swap or to allow memory overcommit. If an admin insists in a
> > no-overcommit policy then his system has to provide the means for it.
> 
> Sure, but there are actually three overcommit settings:
> 
>   0: "The kernel attempts to estimate the amount of free memory left
>      when userspace requests more memory." (default)
>   1: Infinite overcommit
>   2: No overcommit

Infinite overcommit is only useful for very specialized applications.

> AFAIK, reserving memory with a @nobits section in a PT_LOAD segment
> counts as memory use for the heuristic that's used for case 0, while
> reserving it with mmap doesn't. I feel this is a bug (from our point
> of view there's no difference between the two cases), but I'm willing
> to accept that it's really working as intended.

The NOBITS section is essentially the same as

char current_dynamic_space[DYNAMIC_SPACE-END - DYNAMIC_SPACE_START];

only at a memory location outside the executable's normal .bss. Such
memory is normally not used (and freed) pagewise, and the kernel has
no way to know we want something different here.


Thiemo

On Aug 4, 2005, at 12:48 PM, Thiemo Seufer wrote:

> Juho Snellman wrote:
>
>> On Thu, Aug 04, 2005 at 08:55:39AM +0100, Christophe Rhodes wrote:
>>
>>> If the answer to this is having to play ELF games with our  
>>> executable,
>>> to reserve space before the kernel and linker decide where they're
>>> going to live and to put shared libraries, then so be it; something
>>> akin to Brian Mastenbrook's ELF-grovelling Darwin trick can probably
>>> be adapted, no?
>>>
>>
>> I had a look at this. Due to kernel bugs it's apparently  
>> impossible to
>> reserve space without running into overcommit problems, no matter  
>> what
>> elf tricks you play. For example the attached patch adds a @nobits
>> section into the data segment and moves the spaces inside the  
>> section.
>> Should work like a charm, but the kernel actually counts that space
>> reservation against the system overcommit limit.
>>
>
> This is not a kernel bug but simply the definition of a LOADed NOBITS
> section. It allocates the defined .space statically.

It looks like a bug to me. A read-only mapping shouldn't and  
*doesn't* count against the commit limit, and the section was marked  
readonly. However, the kernel ignores the request to make a nobits  
section readonly and always allocates it read/write. Actually it  
looks like it ignores everything about the section but its size, and  
just calls brk(). Thus it's impossible to make a non-file-backed  
readonly section. That sucks. Making the sbcl executable contain 8GB  
of zeros is probably not an acceptable solution. ;)

Here's a solution but it'd take some work:
a) store the core file in the executable (the elf loader will dtrt  
when there is actually data there)
b) make the dynamic heap loaded from the core file be able to be  
discontinuous from the main dynamic heap.
c) make the main dynamic heap relocatable/resizable at boot time.

After doing all that, the stuff that has to be at a fixed address  
will be guaranteed to get it no matter what else is on the system,  
and the dynamic heap can live wherever is free on the particular system.
James

James Y Knight wrote:
[snip]
> >This is not a kernel bug but simply the definition of a LOADed NOBITS
> >section. It allocates the defined .space statically.
> 
> It looks like a bug to me. A read-only mapping shouldn't and  
> *doesn't* count against the commit limit, and the section was marked  
> readonly. However, the kernel ignores the request to make a nobits  
> section readonly and always allocates it read/write.

This has nothing to do with the kernel. The linker collates the NOBITS
sections (together with others) into one ELF segment with the union of
attributes. You will need a custom linker script to change that.


Thiemo

On Aug 5, 2005, at 11:53 AM, Thiemo Seufer wrote:
> This has nothing to do with the kernel. The linker collates the NOBITS
> sections (together with others) into one ELF segment with the union of
> attributes. You will need a custom linker script to change that.
>
Yes, you're right the linker has already combined the sections before  
the kernel even sees it. I misread the output of readelf. However, I  
believe even a custom linker script would not help, as what I said  
above is still true (unless I'm wrong about that as well ;p).  
Whenever the kernel sees a section which has filesize < memsize, it  
appears to simply calls brk to expand the vm space, ignoring all  
flags on the section. binfmt_misc:769 in my kernel.

Anyhow, I believe the conclusions are still valid: it does appear  
that a readonly progbits section works and could be used instead of  
an external core file.
.section .foo,"xa",@progbits
.space 0x300000
.end

James

James Y Knight wrote:
> On Aug 5, 2005, at 11:53 AM, Thiemo Seufer wrote:
> >This has nothing to do with the kernel. The linker collates the NOBITS
> >sections (together with others) into one ELF segment with the union of
> >attributes. You will need a custom linker script to change that.
> >
> Yes, you're right the linker has already combined the sections before  
> the kernel even sees it. I misread the output of readelf. However, I  
> believe even a custom linker script would not help, as what I said  
> above is still true (unless I'm wrong about that as well ;p).  
> Whenever the kernel sees a section which has filesize < memsize, it  
> appears to simply calls brk to expand the vm space, ignoring all  
> flags on the section. binfmt_misc:769 in my kernel.

a) The relevant file is binformat_elf.c
b) do_brk is a simplified version of do_mmap.


Thiemo

On Aug 5, 2005, at 4:03 PM, Thiemo Seufer wrote:
> a) The relevant file is binformat_elf.c
> b) do_brk is a simplified version of do_mmap.

a) Indeed, I was reading, and meant to type "binfmt_elf.c". Dunno how  
that "misc" snuck in there.
b) do_brk does always map with read/write/execute permissions  
ignoring any flags specified in the binary. Doesn't matter if it's  
otherwise similar to mmap.

James

* James Y Knight (foom@...) [050805 10:49]:
> It looks like a bug to me. A read-only mapping shouldn't and  
> *doesn't* count against the commit limit, and the section was marked  
> readonly. However, the kernel ignores the request to make a nobits  
> section readonly and always allocates it read/write. Actually it  
> looks like it ignores everything about the section but its size, and  
> just calls brk(). Thus it's impossible to make a non-file-backed  
> readonly section. That sucks. Making the sbcl executable contain 8GB  
> of zeros is probably not an acceptable solution. ;)

Why not make it a sparse file?

    --larry

Larry D'Anna <smoof-ra@...> writes:

> * James Y Knight (foom@...) [050805 10:49]:
>> It looks like a bug to me. A read-only mapping shouldn't and  
>> *doesn't* count against the commit limit, and the section was marked  
>> readonly. However, the kernel ignores the request to make a nobits  
>> section readonly and always allocates it read/write. Actually it  
>> looks like it ignores everything about the section but its size, and  
>> just calls brk(). Thus it's impossible to make a non-file-backed  
>> readonly section. That sucks. Making the sbcl executable contain 8GB  
>> of zeros is probably not an acceptable solution. ;)
>
> Why not make it a sparse file?

I believe SBCL on Solaris already uses /dev/zero to obtain zeros :-) I
don't know if that helps.

Cheers,

Christophe