#68 On MacOS Golly allocates too much RAM
Either MacOS malloc() has changed behavior since the main parts of golly were written, or this is a long-standing bug. But if you configure a maximum memory of (say) 4GB, and run a hash algorithm, Golly can frequently appear to allocate way more memory than this (I typically see around 40% more memory than configured). The new MacOS allocator does two things that we don't expect. First, if you ask for 48,000 bytes, the allocator silently gives you 65,536 bytes (and you don't know, so you only use 48,000 bytes). Secondly, if you allocate a big region (say, 20M for a hashtable pointer array) and later free it, MacOS will never make that memory available smaller allocations, nor will it release it back to the system (unless the system undergoes specific memory stress). To reproduce this easily, just use something like
/usr/bin/time -l bgolly -v -h -M 4000 -m 32000000 unlimited-novelty.mc
and you'll see a peak memory footprint of 5.7GB instead of 4GB.
MacOS does have a system-dependent "malloc_size()" which can help the first, but similar calls on other systems have different names. And this won't help the fact that golly intentionally deallocates large hashtable arrays so they can be used for node memory.
I don't see this issue on Linux; I don't have access to Windows to test.
I have a patch in testing for this issue; I've increased the number of nodes in a node block so it's close to but not too close to a power of two, to fix the first issue, and I've modified freeing of the hashtable pointer arrays so we don't free but instead just make a big node block directly out of that memory, wresting memory control from the allocator. This appears to fix the issues.
I'm still testing, so I'm not releasing the patch yet, but I'm filing the bug so we can discuss this approach to start.
I don't think there's any great urgency on this; I haven't heard of this causing issues yet. But I figured better to fix it now while I've observed it and can fix it.
I've created a patch in the branch https://sourceforge.net/p/golly/code/ci/fixmemoryzoneissue/tree/
I've done some testing, but would appreciate more. This can likely wait for our next release.
I ran the same bgolly command before and after building with your patch (on M3 Macbook Air with 16GB RAM running macOS 15.6.1).
Before: memory peaked at 4.98GB
After: memory peaked at 3.92GB
I'm happy to include this patch in the next beta release (5.1b1) whenever that might be (I'm just waiting for new patterns that have been proposed on the forums).