Performance impact of dynamic RAM disk size

[MichaIng]

Hello,

I am using the AIM Toolkit for TEMP folders, browser caches (Opera,
Chromium-based), and as default download folder, on Windows 11.

I did use it with NTFS, dynamic size, virtual memory type, and it did
generally worked well. But what I recognised with downloads, is that
they sometimes did no appear as really finished for a while in Opera,
despite that they have been fully downloaded. I could manually
access/open downloads in the Windows Explorer, but in the Opera download
manager/list, they were not accessible, as if the download was still
running. That solved itself after some seconds up to some minutes.

With one of the recent Opera/Chromium updates (it could have been a
Windows update as well), things became a lot worse: websites started to
load very slowly, UI elements were unusable for several seconds on every
navigation step. In browser console I recognised that assets, which were
downloaded instantly, kept showing up as "pending" for a longer while,
which matched the delays/hangs of the website elements. This made me
draw the connection between the hanging downloads and hanging assets, as
if the actual write to the RAM disk was delayed for some reason.

The first attempt to narrow it down, by disabling the dynamic size,
allocating the full size statically instead, was a bull's eye: The
website hangs were gone, and also downloads are now instantly shown as
finished and accessible in the browser's download manager/list.

What I assume is that the dynamic allocation can be very slow, even for
tiny files. The data maybe resides is in some sort of buffer (funny in
case of a RAM disk), and can be accessed in Windows Explorer, but it has
not really been written to the filesystem yet. And it can take a while
until it is, until which at least some other programs, like Chromium,
cannot access it yet.

This happens with 8 GB RAM disk with 32 GB overall RAM, and >50% free
(actual usage + RAM disk < 50%). So there is always more than enough
free RAM.

Pre-allocating everything does work well, but maybe there is a way to
avoid these delays in case of dynamic allocation?

Best regards,
Micha

[v77]

Very interesting but I am afraid that it will be difficult to reproduce.

There is still something I would like to know. When you see these delays, what is the CPU usage of ramdyn.exe?

[MichaIng]

RamDyn.exe was the right hint: It remains at 0.0% - 0.1% CPU usage, but I can literally watch it filling the RAM very slowly, after the download finished:

• E.g. fresh reboot, its RAM usage in task manager shows ~50 MiB with Windows + Thunderbird + Opera cache initialized.
• Then I start a ~200 MiB download, which finishes in ~20 seconds, and I can see RamDyn.exe RAM usage climbing, but slower than the download.
• After the download finished, RamDyn.exe RAM usage keeps climbing until ~950 MiB, but at a quite slow rate, about0 - 30 MiB/s, sometimes stopping for 1-2 seconds in between.
• Only after the end ~950 MiB RAM usage have been reached, the download is shown as really finished in Opera.
• After some more seconds, RAM usage drops to the actually expected ~250 MiB, with the ~50 MiB initial cache + 200 MiB download. I wonder why it does consume ~4x more in between.

At the moment, I cannot replicate the slow browsing issue anymore. Maybe the download tests caused some condition, like freed continuous free memory space or so. I just changed back from fixed size to dynamic size, and will keep an eye on the RamDyn.exe process when experiencing any performance issues.

Btw, if I may ask another question: Since creating the RAM disk with 8 GiB fixed size takes a few seconds, which caused some content in the mount folder (I use a mount point instead of a drive letter) to be created earlier by something else, probably system temp folder, or Thunderbird (autostart) cache or so. And then, the RAM disk mount fails. So I switched on fast format. What I immediately recognized is that the previously (with static size) shown "System" process with 8 GiB RAM usage was not present anymore, and it never actually appeared. Hence I wonder which process owns the RAM disk usage with fast format enabled. Or whether there is another way to observe at which point the full size has actually been allocated.

[v77]

Yes, in your test, the RAM usage should be ~250 MB, not 950, no matter the settings (at least if your download is only one file). This point is highly suspect, but not necessarily linked to the delays.

For the rest, I am perplexed. I hope the author of the driver will have a better idea.

About the RAM usage with static ramdisks, with fast format, the RAM is still reserved, but the "System" process does not show it as long as the memory pages are not accessed. And that is why the creation will fail if you attempt to create a 100 TB ramdisk.

[MichaIng]

I tested downloads with Firefox, and there I cannot replicate the observation above: Memory usage raises by the expected ~200 MiB only, and shows up as finished immediately, after the actual download did.

To rule out that it is an Opera-specific issue, or related to the fact that its caches (including GPU/shader/code/system caches) are linked to the RAM disk as well, I tested downloads with a fresh Brave browser instance, with caches left on the default disk dirs: It behaves exactly like Opera above. Hence it is a Chromium-related issue.

A visible difference in how Chromium-based browsers and Firefox do downloads is this:

• While downloading with Chromium, the file appears with a .zydownload suffix, while "zy" is a vendors-specific, "op" in case of Opera, "br" in case of Brave. After the download (actual file transfer) finished, it is renamed to the actual target filename. If this is not an actual rename, but a copy at some lower level, that could explain the additional temporary memory usage, though not that it is 4x higher. Of course that would be a very dump/inefficient behavior that would need to be fixed in Chromium. However, in Windows Explorer, the renaming appears immediate, and the file can be immediately accessed e.g. with 7-Zip, while it is not shown as finished/accessible in Chromium yet, and the RamDyn.exe memory usage still raises.
• Firefox pre-creates an empty file with the target filename, and downloads to a file with .part suffix. That part file shows as (exactly) 4 KiB size in Windows Explorer throughout the whole download, and the target file remains empty (0 bytes). After the download, the part file vanishes, and the target file gets its final size immediately. The download is shown as finished in Firefox, and no post-download RamDyn.exe memory usage increase.

Maybe, an intermediate Chromium version aligned some behavior in how downloads and website asset downloads are done, and reverted in again, since browsing websites is still performing fine again. I'll have a look into Chromium commits and reverted commits between 149 => 150 => 151, maybe I can find something related to asset downloads.