maybe more, haven't checked entire codebase.
Avisynth uses GlobalMemoryStatus to determine the maximum amount it can allocate. This is incorrect - GlobalMemoryStatus returns the memory available to the *system*, not to applications. If you have 4GiB of RAM, it will return 4GiB, and a 32-bit process can only allocate 2GiB max.
I've got 8GiB of RAM on a 64-bit system - this causes avisynth to set the maximum memory too high and it eventually freezes up once it's done eating around 2GB of memory.
Avisynth should use GetSystemInfo, and use ((intptr_t)lpMaximumApplicationAddress - (intptr_t)lpMinimumApplicationAddress) to determine the maximum amount of memory the process can allocate. However note that this is still only maximum address space - fragmentation and any other use of memory will take from this.
Workaround for users:
Use SetMemoryMax(1024) or lower on all your scripts.