From: Bruno D. <bru...@gm...> - 2007-07-05 09:46:33
|
Dear developers, I'm using sbcl-1.0.5 on a linux machine which has 4 GB of RAM, but it doesn't seem to be possible to use more than about 1.3 GB from SBCL. Whenever I try to allocate more memory in some arrays, I get a heap exhaustion. I tried to use the option --dynamic-space-size, but with more than 1600 as the parameter, I receive the following error message on the shell. > sbcl --dynamic-space-size 1700 mmap: wanted 268431360 bytes at 0x70000000, actually mapped at 0xe7cee000 ensure_space: failed to validate 268431360 bytes at 0x70000000 (hint: Try "ulimit -a"; maybe you should increase memory limits.) I also tried to change some parameters in sbcl-1.0.5/src/compiler/x86/parms.lisp and recompiled everything, but to no avail. Is there any way to use the full 4 GB in my machine? Is there any standard parms.lisp for enabling, say, a 3 GB heap? Best regards, Bruno Daniel |
From: Florian W. <fw...@de...> - 2007-07-05 09:55:25
|
* Bruno Daniel: > I also tried to change some parameters in > sbcl-1.0.5/src/compiler/x86/parms.lisp > and recompiled everything, but to no avail. Is there any > way to use the full 4 GB in my machine? Is there any standard > parms.lisp for enabling, say, a 3 GB heap? Use the amd64 architecture. 8-) |
From: Bruno D. <bru...@gm...> - 2007-07-05 10:24:27
|
Florian Weimer wrote: > * Bruno Daniel: > > >> I also tried to change some parameters in >> sbcl-1.0.5/src/compiler/x86/parms.lisp >> and recompiled everything, but to no avail. Is there any >> way to use the full 4 GB in my machine? Is there any standard >> parms.lisp for enabling, say, a 3 GB heap? >> > > Use the amd64 architecture. 8-) > > So the conclusion is that the problem is caused by the type bits in pointer variables which reduce the address space? Best regards, Bruno Daniel |
From: Bruno D. <bru...@gm...> - 2007-07-05 11:01:16
|
> No, the kernel needs some address space for itself, and the remaining > address space is fragmented by dynamic shared objects. > Is this the same in all languages/implementations or is it special to sbcl? Best regards Bruno Daniel |
From: Florian W. <fw...@de...> - 2007-07-05 10:35:38
|
* Bruno Daniel: >> Use the amd64 architecture. 8-) > So the conclusion is that the problem is caused by the > type bits in pointer variables which reduce the address space? No, the kernel needs some address space for itself, and the remaining address space is fragmented by dynamic shared objects. |
From: Florian W. <fw...@de...> - 2007-07-05 11:42:16
|
* Bruno Daniel: >> No, the kernel needs some address space for itself, and the remaining >> address space is fragmented by dynamic shared objects. >> > Is this the same in all languages/implementations or is it special > to sbcl? The 3 GB limit affects everyone. There are implementations which can deal with the fragmentation issue, though. Some garbage collectors do not require a continuous heap, for instance. |
From: Bruno D. <bru...@gm...> - 2007-07-05 11:58:39
|
Florian Weimer wrote: > The 3 GB limit affects everyone. There are implementations which can > deal with the fragmentation issue, though. Some garbage collectors do > not require a continuous heap, for instance. > Thank you very much for helping me to understand the problem. Best regards, Bruno Daniel |
From: Juho S. <js...@ik...> - 2007-07-05 15:07:53
|
Bruno Daniel <bru...@gm...> writes: > Dear developers, > > I'm using sbcl-1.0.5 on a linux machine which has 4 GB > of RAM, but it doesn't seem to be possible to use more > than about 1.3 GB from SBCL. Whenever I try to allocate > more memory in some arrays, I get a heap exhaustion. > > I tried to use the option --dynamic-space-size, but > with more than 1600 as the parameter, I receive the > following error message on the shell. > > > sbcl --dynamic-space-size 1700 > mmap: wanted 268431360 bytes at 0x70000000, actually mapped at 0xe7cee000 > ensure_space: failed to validate 268431360 bytes at 0x70000000 > (hint: Try "ulimit -a"; maybe you should increase memory limits.) You should be able to use a larger number than that (though not 4GB or even 3GB) if you upgrade to 1.0.6 or newer, since the memory spaces were rearranged recently. -- Juho Snellman |
From: Marc H. <mar...@un...> - 2007-07-06 07:58:51
Attachments:
signature.asc
|
Hi! >=20 > You should be able to use a larger number than that (though not 4GB or > even 3GB) if you upgrade to 1.0.6 or newer, since the memory spaces > were rearranged recently. ?? Yes, the memory spaces were rearranged, but they were made SMALLER in the actual version Please correct me if I'm wrong old parms.lisp (rev. 1.66) #!+linux (progn (def!constant read-only-space-start #x01000000) (def!constant read-only-space-end #x037ff000) (def!constant static-space-start #x05000000) (def!constant static-space-end #x07fff000) (def!constant dynamic-space-start #x09000000) (def!constant dynamic-space-end #x29000000) (def!constant linkage-table-space-start #x70000000) (def!constant linkage-table-space-end #x7ffff000)) versus actual parms.lisp (rev. 1.69) #!+linux (progn (def!constant read-only-space-start #x01000000) (def!constant read-only-space-end #x010ff000) (def!constant static-space-start #x01100000) (def!constant static-space-end #x011ff000) (def!constant dynamic-space-start #x09000000) (def!constant dynamic-space-end #x29000000) (def!constant linkage-table-space-start #x01200000) (def!constant linkage-table-space-end #x012ff000)) Nice to have would be a "BIG_SYSTEM" config option or similar for people that need a lot of memory Greetings Marc |
From: Juho S. <js...@ik...> - 2007-07-06 11:10:42
|
Marc Halbruegge <mar...@un...> writes: > Hi! > > > > > You should be able to use a larger number than that (though not 4GB or > > even 3GB) if you upgrade to 1.0.6 or newer, since the memory spaces > > were rearranged recently. > ?? > > Yes, the memory spaces were rearranged, but they were made SMALLER in > the actual version > > Please correct me if I'm wrong Not wrong per se, since the sizes of the affected spaces were indeed reduced. It's just that this is a completely irrelevant fact :-) The discussion is about the size of the dynamic heap. The sizes of the other spaces don't really matter. The important part is that the linkage table space was moved from above the dynamic space to below it, and can thus no longer get in the way if the dynamic space is grown. > Nice to have would be a "BIG_SYSTEM" config option or similar for people > that need a lot of memory I don't think that makes a lot of sense, since the size can already be selected at runtime. -- Juho Snellman |
From: Marc H. <mar...@un...> - 2007-07-06 12:23:43
Attachments:
signature.asc
|
Hi, >=20 > Not wrong per se, since the sizes of the affected spaces were indeed > reduced. It's just that this is a completely irrelevant fact :-) The > discussion is about the size of the dynamic heap.=20 OK, sorry for hijacking the thread ;) >> Nice to have would be a "BIG_SYSTEM" config option or similar for peop= le >> that need a lot of memory >=20 > I don't think that makes a lot of sense, since the size can already be > selected at runtime. As I pointed out earlier, the size of the other memory spaces is important as well: http://sourceforge.net/mailarchive/forum.php?thread_name=3D465C5440.20808= 03%40unibw.de&forum_name=3Dsbcl-help Greetings Marc |
From: Bruno D. <bru...@gm...> - 2007-07-06 14:16:46
|
Dear Mr. Snellman, Juho Snellman wrote: > I don't think that makes a lot of sense, since the size can already be > selected at runtime. > > How do you do this in SBCL? Is it controlled by the C variable dynamic_space_size, i.e. in the following way (using CFFI)? (defcvar ("dynamic_space_size" dynamic-space-size) :int) (incf dynamic-space-size 50000000) Best regards Bruno Daniel |
From: Nikodemus S. <nik...@ra...> - 2007-07-06 14:32:35
|
Bruno Daniel wrote: > Dear Mr. Snellman, > > Juho Snellman wrote: >> I don't think that makes a lot of sense, since the size can already be >> selected at runtime. >> >> > How do you do this in SBCL? Is it controlled by the C variable > dynamic_space_size, i.e. in the following way (using CFFI)? Well, not at _runtime_ currency, but at startup, using --dynamic-space-size. Cheers, -- Nikodemus |