Re: [MLton] max-heap setting for 64-bit applications

[Matthew F. <mat...@gm...>]

On Mon, Dec 14, 2009 at 12:45 PM, Wesley W. Terpstra <we...@te...> wrote:
> On Mon, Dec 14, 2009 at 5:50 PM, Matthew Fluet <mat...@gm...> wrote:
>> On Sat, Dec 12, 2009 at 8:24 AM, Wesley W. Terpstra <we...@te...> wrote:
>>> The mremap code does a memcpy, which is 100% sequential.
>> But, the Windows specific mremap could still fail --- note that the
>> growHeap function demands significant growth from mremap.  If that
>> fails, then it attempts the alloc/copy, but allowing for an alloc of a
>> heap down to the minimum size.
>
> This could only happen if you fragmentted the entire 16TB of VM.

I was thinking about the 32-bit case, in which case fragmenting the 4G
VM isn't terribly difficult.

> I bet it's probably always faster to grow in-place. As there have been
> no issues with the mremap code since I wrote it, I'm confident enough
> in it to switch it to the default, committed.
>
> It might also be worth writing an improved copy/move code-path that
> allocates as it goes, thus avoiding swapping. ie:
> (commit/copy/decommit)s in blocks of 4MB at a time, releasing the
> source reservation as the last step. Not as amazing as a real mremap
> which doesn't need to copy at all, but at least it would have the same
> 'trash or not thrash' behaviour.

On Windows, yes.  Reserve the whole target heap size, and during the
copy, commit/decommit as we go.

I don't think we can get the same kind of behavior on POSIX.