Steel Bank Common Lisp

2009/11/3 Giovanni Gigante <giov@...>:
>
> Hello,
> I tried starting sbcl with --dynamic-space-size 4000 (that is, 4GB;
> unfortunately I really need that much memory).
> It refused to, saying "specified --dynamic-space-size too large". So is
> there an upper limit? (SBCL manual does not say it).
>
> Also, trying with a lower value, 3000, sbcl does not complain in the
> same way, but still fails to allocate it ("mmap: Cannot allocate
> memory"), despite the fact that this machine has 6GB RAM of which 5.9GB
> are free, according to "free" command.
>
> What's going on? Is there no way to use SBCL with a lot of memory?
>
> I am on debian linux 5.0.3, kernel 2.6.26-2-686-bigmem, SBCL 1.0.29

That's 32 bit, right?

It won't work.  If you want a single process to use more than about
3GB of virtual memory it will need to be a 64 bit process (on a 64 bit
machine).

-- 
Michael Wood <esiotrot@...>

Quoth Giovanni Gigante <giov@...>:
> What's going on? Is there no way to use SBCL with a lot of memory?

SBCL is compiled with a upper limit.  To increase that limit you have to roll
your own.

See this article:

 http://article.gmane.org/gmane.lisp.steel-bank.general/2536/

If you need 4GiB (4096 MiB) you should set dynamic-space-end to #x109000000.

Hope this helps.

Seb
-- 
Emacs' AlsaPlayer - Music Without Jolts
Lightweight, full-featured and mindful of your idyllic happiness.
http://home.gna.org/eap

Sebastian Tennant <sebyte@...> writes:

> Quoth Giovanni Gigante <giov@...>:
>> What's going on? Is there no way to use SBCL with a lot of memory?
>
> SBCL is compiled with a upper limit.  To increase that limit you have to roll
> your own.
>
> See this article:
>
>  http://article.gmane.org/gmane.lisp.steel-bank.general/2536/
>
> If you need 4GiB (4096 MiB) you should set dynamic-space-end to #x109000000.
>
Isn't that parameter changed by --dynamic-space-size? And the error is
not from sbcl, but from mmap failing to allocate memory.

-- 
With best regards, Stas.

Quoth Sebastian Tennant <sebyte@...>:
> Quoth Giovanni Gigante <giov@...>:
>> What's going on? Is there no way to use SBCL with a lot of memory?
>
> SBCL is compiled with a upper limit.  To increase that limit you have to roll
> your own.

Scratch that.  It's not an upper limit.  It's a default.  Looks like you're out
of luck unless you're on a 64 bit machine.

Seb
-- 
Emacs' AlsaPlayer - Music Without Jolts
Lightweight, full-featured and mindful of your idyllic happiness.
http://home.gna.org/eap

Giovanni Gigante wrote on Tue, Nov 03, 2009 at 01:36:16PM +0100: 
> 
> Hello,
> I tried starting sbcl with --dynamic-space-size 4000 (that is, 4GB; 
> unfortunately I really need that much memory).
> It refused to, saying "specified --dynamic-space-size too large". So is 
> there an upper limit? (SBCL manual does not say it).
> 
> Also, trying with a lower value, 3000, sbcl does not complain in the 
> same way, but still fails to allocate it ("mmap: Cannot allocate 
> memory"), despite the fact that this machine has 6GB RAM of which 5.9GB 
> are free, according to "free" command.
> 
> What's going on? Is there no way to use SBCL with a lot of memory?

There is no way to use all that memory in a single 32 bit process.

> I am on debian linux 5.0.3, kernel 2.6.26-2-686-bigmem, SBCL 1.0.29

There was the 4GB/4GB VM split patch the Redhat hat in Fedore Core 2
and parts of 3 IIRC for a while.  That would allow you to go close to
4 GB.

Is there a specific reason why you can't go 64 bits?

Martin
-- 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer@...>   http://www.cons.org/cracauer/
FreeBSD - where you want to go, today.      http://www.freebsd.org/

>>I am on debian linux 5.0.3, kernel 2.6.26-2-686-bigmem, SBCL 1.0.29
>>    
>>
>
>That's 32 bit, right?
>It won't work.  If you want a single process to use more than about
>3GB of virtual memory it will need to be a 64 bit process (on a 64 bit
>machine).
>
>  
>

My CPU is a Intel Q6600. I know that the right way would be a 64-bit 
kernel, but unfortunately at the time this machine was installed the 
wrong choice was made, and now it would be really painful to rebuild the 
whole system from scratch.
So, looking for a simpler solution, when I installed more RAM I switched 
to this 32-bit debian kernel "bigmem", which means it has PAE. I am no 
expert at these matters, but I had read that PAE allows for 64GB RAM 
maximum, with a limit of 4GB per process.
Which is why I tried --dynamic-space-size 4000. (a bit less than those 4GB).
Unfortunately, it doesn't seem to work.

So I still have two doubts:

1) what is the maximum theorical value for --dynamic-space-size in these 
conditions?

2) why mmap still fails under that value? (maybe it needs contiguous 
memory or something like that?) for example, on my system a value of 
2500 holds, while one of 3000 gives:

mmap: Cannot allocate memory
ensure_space: failed to validate -1149239296 bytes at 0x09000000
(hint: Try "ulimit -a"; maybe you should increase memory limits.)

and ulimit -a says:
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 55296
max locked memory       (kbytes, -l) 32
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 55296
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

which does not seems the source of problem.

giovanni

Giovanni Gigante wrote on Tue, Nov 03, 2009 at 03:41:15PM +0100: 
> 
> >>I am on debian linux 5.0.3, kernel 2.6.26-2-686-bigmem, SBCL 1.0.29
> >>    
> >>
> >
> >That's 32 bit, right?
> >It won't work.  If you want a single process to use more than about
> >3GB of virtual memory it will need to be a 64 bit process (on a 64 bit
> >machine).
> >
> >  
> >
> 
> My CPU is a Intel Q6600. I know that the right way would be a 64-bit 
> kernel, but unfortunately at the time this machine was installed the 
> wrong choice was made, and now it would be really painful to rebuild the 
> whole system from scratch.
> So, looking for a simpler solution, when I installed more RAM I switched 
> to this 32-bit debian kernel "bigmem", which means it has PAE. I am no 
> expert at these matters, but I had read that PAE allows for 64GB RAM 
> maximum, with a limit of 4GB per process.
> Which is why I tried --dynamic-space-size 4000. (a bit less than those 4GB).
> Unfortunately, it doesn't seem to work.

The PAE kernel does not extend the virtual memory available to a
single process in any way.

> So I still have two doubts:
> 
> 1) what is the maximum theorical value for --dynamic-space-size in these 
> conditions?

With a 32 bit kernel it is the same regardless of how much RAM you
have and regardless of whether you use PAE or not.

The only exception is the 4GB/4GB VM split patch which gives you a
gigabyte more.

> 2) why mmap still fails under that value? (maybe it needs contiguous 
> memory or something like that?) for example, on my system a value of 
> 2500 holds, while one of 3000 gives:
> 
> mmap: Cannot allocate memory
> ensure_space: failed to validate -1149239296 bytes at 0x09000000
> (hint: Try "ulimit -a"; maybe you should increase memory limits.)
> 
> and ulimit -a says:
> core file size          (blocks, -c) 0
> data seg size           (kbytes, -d) unlimited
> scheduling priority             (-e) 0
> file size               (blocks, -f) unlimited
> pending signals                 (-i) 55296
> max locked memory       (kbytes, -l) 32
> max memory size         (kbytes, -m) unlimited
> open files                      (-n) 1024
> pipe size            (512 bytes, -p) 8
> POSIX message queues     (bytes, -q) 819200
> real-time priority              (-r) 0
> stack size              (kbytes, -s) 8192
> cpu time               (seconds, -t) unlimited
> max user processes              (-u) 55296
> virtual memory          (kbytes, -v) unlimited
> file locks                      (-x) unlimited
> 
> which does not seems the source of problem.

You issue at that point is not only the available space, but
fragmentation.  

Although you have 3 GB total free, the OS mapped dynamic libraries and
assorted other junk inside it so you don't have a contiguous area of 3
GB.  The dynamic heap needs to be contiguous.

Martin
-- 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer@...>   http://www.cons.org/cracauer/
FreeBSD - where you want to go, today.      http://www.freebsd.org/

Martin Cracauer wrote:

>With a 32 bit kernel it is the same regardless of how much RAM you
>have and regardless of whether you use PAE or not.
>
>  
>
>You issue at that point is not only the available space, but
>fragmentation.  
>  
>

Thanks for your replies.

So basically if I want a sbcl able to handle about 4GB of (mostly 
double-float) data, there is no way other than using a 64bit kernel.

But since (correct me if I am wrong) in 64bit the size of those objects 
will double,  that will become 8GB.

And accounting for the fragmentation problem, probably even more than 
that. Maybe 10, 12 GB...
Right?

(btw, for the fragmentation issue, is there any way to mitigate it, 
rather than just throwing more chips at it? increasing swap perhaps? 
tweaking something else?)

giovanni

Giovanni Gigante <giov@...> writes:

> So basically if I want a sbcl able to handle about 4GB of (mostly 
> double-float) data, there is no way other than using a 64bit kernel.
>
> But since (correct me if I am wrong) in 64bit the size of those objects 
> will double,  that will become 8GB.

That is wrong.  The size of a double float is 64 bits, whether you use a
32-bit or 64-bit userland and kernel.

Best,

Christophe

Giovanni Gigante wrote on Tue, Nov 03, 2009 at 04:05:51PM +0100: 
> Martin Cracauer wrote:
> 
> >With a 32 bit kernel it is the same regardless of how much RAM you
> >have and regardless of whether you use PAE or not.
> >
> >  
> >
> >You issue at that point is not only the available space, but
> >fragmentation.  
> >  
> >
> 
> Thanks for your replies.
> 
> So basically if I want a sbcl able to handle about 4GB of (mostly 
> double-float) data, there is no way other than using a 64bit kernel.

And a 64 bit userland and a 64 bit SBCL.

> But since (correct me if I am wrong) in 64bit the size of those objects 
> will double,  that will become 8GB.

No, some objects double in size, other don't.  If you make heavy use
of lists (cons cells) you suffer, declared through array use doesn't.
Integers put into arrays or structs with their ranges declared to be
<= 32 bit will not grow and the like.

> And accounting for the fragmentation problem, probably even more than 
> that. Maybe 10, 12 GB...

No, there's no other overhead from fragmentation.

> (btw, for the fragmentation issue, is there any way to mitigate it, 
> rather than just throwing more chips at it? increasing swap perhaps? 
> tweaking something else?)

The fragmentation isn't in physical memory, it is in virtual memory.
The OS decided to map the dynamic loader and shared libs in what isn't
an edge of the virtual address space.  You could play with things like
the stack size ulimit which tends to shuffle things around.

But overall, is that kind of research worth it? If you need 4 GB
today, why do you think you won't need 6 GB next year?

Martin
-- 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer@...>   http://www.cons.org/cracauer/
FreeBSD - where you want to go, today.      http://www.freebsd.org/