[TCLCORE] Re: Hintermayer's string allocation patch

[Jim I. <ji...@ap...>]

On Wednesday, August 30, 2000, at 03:06 PM, Eric Melski wrote:

> John: with respect to memory running out, yes, all machines that Tcl runs 
> on will likely have virtual memory.  There is no argument that once you 
> start allocating large blocks of virtual memory, your application will 
> become basically useless.  This does not alter the fact that with the 
> original algorithm, you could not allocate, in a single Tcl obj, the 
> entirety of available memory, unless you were very very lucky and 
> happened to start with an object that was  
> (totalMemorySize/2)-(aFewBytes*2) and you appended aFewBytes.  Also, when 
> you had a very large data object, you also would likely have a very large 
> chunk of essentially wasted memory.  This patch enables you to more easily 
> use more available memory, and wastes far less memory. 
>  
>  

I guess I still don't understand the force of your argument.  

Most virtual memory systems give an address space to the application which is governed, not by the available memory on the machine, but by the size of the address space.  Actually, the available space is usually less than the absolute max, since the stack is up at the top, and most systems don't start it at the tip-top of the address space.  For instance MacOS X on a 32 bit processor loads the stack starting around 0xc000000 for some reason.  Still this is two Gig less the code space and stack space for the app for a 32 bit machine.  So the "totalMemorySize" in your above equation is at least 2Gig or so big on a 32 bit machine.  I don't think we need to worry too much about that; a Tcl programmer who reads more than a Gig into a single string is probably doing something wrong anyway, and if they really need to do that they should go out and buy themselves a 64 bit machine...  And we probably won't be running Tcl on any more 16 bit processors in the near future, most of the han!
d-held devices like Palm, etc use 32 bit processors.

Moreover, when you allocate a block of memory, all that happens is that that chunk of your address space is reserved, no  real memory or swap is set aside for it until you actually touch one of the pages.  So again, the act of doubling a 100 Meg string, for instance, doesn't require that you have 200 Meg of swap or memory.  Only the act of actually writing to all 200 Meg of it will do that.  And no algorithm will get around having to have swap space (at least) for the all the pages you actually touch.

This sort of fix would be great for classic MacOS, where you have an admittedly pretty lame memory allocation scheme.  But this doesn't strike me as a practical concern for Unix, and I doubt that it is for NT either.  I have no idea how good the Win9x Virtual memory scheme is - I can imagine there are all sorts of little DOS-bred goblins hiding in the corners, but one can always hope...

Jim
--
Jim Ingham                                   ji...@ap...
Developer Tools - gdb
Apple Computer

--
The TclCore mailing list is sponsored by Ajuba Solutions
To unsubscribe:  email tcl...@aj... with the 
                 word UNSUBSCRIBE as the subject.