#4 Possible mistake when large pages are used, ullAvailPageFile?

[Alexander De Sousa]

I got the out-of-memory message yesterday, disp_err_mem(), and I dug a little bit to figure out what the problem was. Long story short, even though there was enough free memory, it was quite fragmented and the system couldn't allocate a 2MiB contiguous chunk.

However, I noticed the test that leads to the dialog may not be completely right if large pages are used; at RamDyn.c, line 183:

if (mem_stat.ullAvailPageFile >= MINIMAL_MEM && (NtAllocateVirtualMemory(NtCurrentProcess(), &ptr_table[index], 0, &alloc_size, alloc_flags, PAGE_READWRITE) == STATUS_SUCCESS)) {
   memcpy(ptr_table[index] + block_offset, buf, current_size);
   n_block++;
} else
   disp_err_mem();

I understand allocated memory is always resident in physical memory when using large pages, shouldn't the first condition check against ullAvailPhys instead of ullAvailPageFile like with regular pages?

It wouldn't have been meaningful in my case because it was the NtAllocateVirtualMemory call failing with 0xC000009A, indicating there were insufficient system resources, but available physical memory was reported to be ~6.5GiB at the time, while available pageable memory was ~3.87GiB (for some reason).

Also, when large pages are used, perhaps it would be good to fall back to regular pages when allocating a large one isn't possible, not to have data loss in that scenario.

[v77]

With what you had as free pageable memory, testing ullAvailPageFile should not have generated an error. So we can indeed assume that the system could not allocate a 2MB contiguous block.
This means that the system does not even try to unfragment the memory in order to get a 2MB contiguous block. And if the system does not try to move anything, this also means that I should test ullAvailPhys, as you suggested.

However, a lot of users use this feature because they want their data to be kept in physical memory, often for privacy reasons. If this guarantee does no longer exist, then there is no meaning to use this feature, as it's not even faster than the virtual allocation.
And falling back to AWE is excluded as it's a completely different management.

I agree that we should avoid data loss as much as possible.

So finally, what to do? Giving options to the user?
Anyway, it will require to think that thoroughly.

[Alexander De Sousa]

This means that the system does not even try to unfragment the memory in order to get a 2MB contiguous block.

Hmm.. I think the system was under high memory pressure at the time and perhaps it was unable to do so. There are 32 GB of RAM, but there was an opened browser window with a lot of tabs opened, a VM with 8 GiB allocated to it was running, and I was also decompressing an archive, which was using %Temp%; that's when the message was displayed.

If the system does not try to move anything, this also means that I should test ullAvailPhys, as you suggested.

My thoughts were that ullAvailPhys would always be the relevant metric when large pages are used because they are never paged out, I thought the available page file-backed memory only applied to memory pages that could end up there, i.e., when "Virtual" is chosen.

Microsoft's phrasing seems odd to me, their docs define ullAvailPageFile as "The maximum amount of memory the current process can commit, in bytes.", wouldn't it be any process, not just the current (a specific) one?

a lot of users use this feature because they want their data to be kept in physical memory

I am one of them as well haha, I usually have an AWE-based disk created on startup to hold the temp directory, I don't mind AWE's performance penalty at all, it is more than enough for my needs, but I was experimenting with large pages since I saw they were an option ^^,

Nothing meaningful was lost, it was just temporary files from the archive extraction, but I took the opportunity to look into it and that's when I realized it was due to memory fragmentation.

I used the VM that was running to build RamDyn with a debug message to get some visibility when disp_err_mem() is called, and created another RAM disk with the same settings to test and assess whether the problem was memory fragmentation or something else:

So finally, what to do? Giving options to the user?
Anyway, it will require to think that thoroughly.

I am unsure as well, I don't think the system being unable to allocate 2MiB contiguously would happen regularly for most users; especially if they use the RAM disk sparingly. The longer the system has been on, the higher memory usage is, or the more allocation / deallocation events happen, the more fragmented RAM becomes, but still.

I haven't changed that drive back to using AWE yet, and I haven't encountered the problem again, but I haven't tested it thoroughly either (uptime is but a couple of hours and it hasn't been too stressed). In any case, I saw there was a note about fragmentation in the tooltip that appears when selecting the memory type, I wonder if Windows "defragments" RAM at any point, so to speak.