Steel Bank Common Lisp

* Bruno Daniel:

> I also tried to change some parameters in
> sbcl-1.0.5/src/compiler/x86/parms.lisp
> and recompiled everything, but to no avail. Is there any
> way to use the full 4 GB in my machine? Is there any standard
> parms.lisp for enabling, say, a 3 GB heap?

Use the amd64 architecture. 8-)

Florian Weimer wrote:
> * Bruno Daniel:
>
>   
>> I also tried to change some parameters in
>> sbcl-1.0.5/src/compiler/x86/parms.lisp
>> and recompiled everything, but to no avail. Is there any
>> way to use the full 4 GB in my machine? Is there any standard
>> parms.lisp for enabling, say, a 3 GB heap?
>>     
>
> Use the amd64 architecture. 8-)
>
>   
So the conclusion is that the problem is caused by the
type bits in pointer variables which reduce the address space?

Best regards,
   Bruno Daniel

* Bruno Daniel:

>> Use the amd64 architecture. 8-)

> So the conclusion is that the problem is caused by the
> type bits in pointer variables which reduce the address space?

No, the kernel needs some address space for itself, and the remaining
address space is fragmented by dynamic shared objects.

> No, the kernel needs some address space for itself, and the remaining
> address space is fragmented by dynamic shared objects.
>   
Is this the same in all languages/implementations or is it special
to sbcl?

Best regards
   Bruno Daniel

* Bruno Daniel:

>> No, the kernel needs some address space for itself, and the remaining
>> address space is fragmented by dynamic shared objects.
>>   
> Is this the same in all languages/implementations or is it special
> to sbcl?

The 3 GB limit affects everyone.  There are implementations which can
deal with the fragmentation issue, though.  Some garbage collectors do
not require a continuous heap, for instance.

Florian Weimer wrote:
> The 3 GB limit affects everyone. There are implementations which can
> deal with the fragmentation issue, though.  Some garbage collectors do
> not require a continuous heap, for instance.
>
Thank you very much for helping me to understand the problem.

Best regards,
   Bruno Daniel

Bruno Daniel <bruno.daniel@...> writes:
> Dear developers,
> 
> I'm using sbcl-1.0.5 on a linux machine which has 4 GB
> of RAM, but it doesn't seem to be possible to use more
> than about 1.3 GB from SBCL. Whenever I try to allocate
> more memory in some arrays, I get a heap exhaustion.
> 
> I tried to use the option --dynamic-space-size, but
> with more than 1600 as the parameter, I receive the
> following error message on the shell.
> 
> > sbcl --dynamic-space-size 1700
> mmap: wanted 268431360 bytes at 0x70000000, actually mapped at 0xe7cee000
> ensure_space: failed to validate 268431360 bytes at 0x70000000
> (hint: Try "ulimit -a"; maybe you should increase memory limits.)

You should be able to use a larger number than that (though not 4GB or
even 3GB) if you upgrade to 1.0.6 or newer, since the memory spaces
were rearranged recently.

-- 
Juho Snellman

Hi!

>=20
> You should be able to use a larger number than that (though not 4GB or
> even 3GB) if you upgrade to 1.0.6 or newer, since the memory spaces
> were rearranged recently.
??

Yes, the memory spaces were rearranged, but they were made SMALLER in
the actual version

Please correct me if I'm wrong

old parms.lisp (rev. 1.66)
#!+linux
(progn
  (def!constant read-only-space-start     #x01000000)
  (def!constant read-only-space-end       #x037ff000)

  (def!constant static-space-start        #x05000000)
  (def!constant static-space-end          #x07fff000)

  (def!constant dynamic-space-start       #x09000000)
  (def!constant dynamic-space-end         #x29000000)

  (def!constant linkage-table-space-start #x70000000)
  (def!constant linkage-table-space-end   #x7ffff000))

versus actual parms.lisp (rev. 1.69)
#!+linux
(progn
  (def!constant read-only-space-start     #x01000000)
  (def!constant read-only-space-end       #x010ff000)

  (def!constant static-space-start        #x01100000)
  (def!constant static-space-end          #x011ff000)

  (def!constant dynamic-space-start       #x09000000)
  (def!constant dynamic-space-end         #x29000000)

  (def!constant linkage-table-space-start #x01200000)
  (def!constant linkage-table-space-end   #x012ff000))


Nice to have would be a "BIG_SYSTEM" config option or similar for people
that need a lot of memory

Greetings
Marc

Marc Halbruegge <marc.halbruegge@...> writes:
> Hi!
> 
> > 
> > You should be able to use a larger number than that (though not 4GB or
> > even 3GB) if you upgrade to 1.0.6 or newer, since the memory spaces
> > were rearranged recently.
> ??
> 
> Yes, the memory spaces were rearranged, but they were made SMALLER in
> the actual version
> 
> Please correct me if I'm wrong

Not wrong per se, since the sizes of the affected spaces were indeed
reduced. It's just that this is a completely irrelevant fact :-) The
discussion is about the size of the dynamic heap. The sizes of the
other spaces don't really matter. The important part is that the
linkage table space was moved from above the dynamic space to below
it, and can thus no longer get in the way if the dynamic space is
grown.
  
> Nice to have would be a "BIG_SYSTEM" config option or similar for people
> that need a lot of memory

I don't think that makes a lot of sense, since the size can already be
selected at runtime.

-- 
Juho Snellman

Hi,
>=20
> Not wrong per se, since the sizes of the affected spaces were indeed
> reduced. It's just that this is a completely irrelevant fact :-) The
> discussion is about the size of the dynamic heap.=20
OK, sorry for hijacking the thread ;)

>> Nice to have would be a "BIG_SYSTEM" config option or similar for peop=
le
>> that need a lot of memory
>=20
> I don't think that makes a lot of sense, since the size can already be
> selected at runtime.
As I pointed out earlier, the size of the other memory spaces is
important as well:
http://sourceforge.net/mailarchive/forum.php?thread_name=3D465C5440.20808=
03%40unibw.de&forum_name=3Dsbcl-help

Greetings
Marc

Dear Mr. Snellman,

Juho Snellman wrote:
> I don't think that makes a lot of sense, since the size can already be
> selected at runtime.
>
>   
How do you do this in SBCL? Is it controlled by the C variable
dynamic_space_size, i.e. in the following way (using CFFI)?

(defcvar ("dynamic_space_size" dynamic-space-size) :int)
(incf dynamic-space-size 50000000)

Best regards
   Bruno Daniel

Bruno Daniel wrote:
> Dear Mr. Snellman,
> 
> Juho Snellman wrote:
>> I don't think that makes a lot of sense, since the size can already be
>> selected at runtime.
>>
>>   
> How do you do this in SBCL? Is it controlled by the C variable
> dynamic_space_size, i.e. in the following way (using CFFI)?

Well, not at _runtime_ currency, but at startup, using --dynamic-space-size.

Cheers,

  -- Nikodemus