#2039 Virtual memory is mistakenly reported full

[mirh]

Procedure to reproduce this is here.
Just try to extract the file with 7-zip with those pagefile conditions.

It should return "Not enough storage is available to complete this operation" error.
I almost suspect it's a Wndows bug, but since I know next to nothing about memory allocation I might be wrong.

[Igor Pavlov]

Please write details of problem here without external links:
What conditions?
What exact file?
Do you extract it with 7-Zip or run self-extracting exe?
If error with 7-Zip, what version of 7-Zip?
What exact error message?
What OS?

[Igor Pavlov]

If it's sfx-exe error, maybe it's virtual memory fragmentation problem.
They use 512 MB dictinary for LZMA and 7-Zip-sfx tries allocate 512 MB block for decompression.
There is only 2 GB space in 32-bit mode.
But virtual memory can be fragmented sometimes.
For example, some DLLs can split 2 GB space to small blocks.
So 7-Zip can't allocate continuous 512 MB block.

Maybe ASLR also hurts it?

You can try "Process Explorer" from sysinternals.
Look base address for exe file and all DLLs when sfx is running.
You can sort "DLL" by "Base" address. Maybe it will show some dlls with "bad" Base adresses.

[mirh]

File is this.

Conditions are commit limit - commit charge < 560MB.
Where limit (at least in microsoft terminology I'm aware of) hasn't to be intended as "absolute maximum" but just the sum of "physical ram + actual pagefile allocation".
Normally, when under "heavy load" you'd just expect the OS (W7 64 bit btw) to just increase the latter, profit.

Instead it seems like 7-zip doesn't even attempt to allocate the thing. Or worse, that Windows for some reason is barring it in the first place.

[Igor Pavlov]

I don't know about "commit limit - commit charge" problem now.
So can you prove that it's "commit limit" problem?

Probably you must check possible "virtual memory fragmentation" at first.
Maybe you can see "virtual memory fragmentation", if you look
DLL's in "Process Explorer".
Try to find some unusual "bad" DLLs with bad Base addresses.

[mirh]

So can you prove that it's "commit limit" problem?

Yes, I can prove that. I put pagefile on manual for testing (with a very low minimum), then tried to slowly increase it. Maximum was several GBs more so it wasn't plainly about lack of memory period.
Once there was enough pagefile to have enough free commit.. Puf, 7-zip worked and didn't complain anymore.
If I -say- open firefox (which "steals" around 150MB or something) it doesn't work again until I someway "make room".

I don't know how much looking for framentation may help then.
Anyway.. This should be it.

[Igor Pavlov]

That sfx probably needs about 550 MB.
So does it mean that ANY program (32-bit and 64-bit) can't allocate 550 MB in your case?
If so, then it's not "strange" problem. It's some usual problem.
In that case you have problems with almost any program that uses big data.

But "virtual memory fragmentation" problem is unusual problem.
"virtual memory fragmentation" means that some 32-bit programs can't allocate big continuous blocks.
For example,
1) you can allocate 30 GB in 64-bit program
2) you can allocate 1.5 GB in 32-bit program if all allocated blocks are small
3) but you can't allocate some continuous block (512 MB-1GB) in 32-bit program.

Note that unusual thing with that 7-Zip sfx is that it needs BIG CONTINUOUS block. Most other programs don't need big continues blocks.

I suppose most systems can allocate CONTINUOUS 512 MB block.
But maybe there are some rare cases (maybe with some strange DLLs in system) that doesn't allow it.

[mirh]

So does it mean that ANY program (32-bit and 64-bit) can't allocate 550 MB in your case?

I'm not sure how I could test for this.
Especially because I mean.. usual programs only "gradually" allocates memory. And Windows then increase pagefile up to its actual maximum when short of.
Which then it doesn't touch/decrease anymore for the remainder of the session.

So it could even be that after I open and quit a videogame, decompression now finally work because pagefile passed from a bare 400MB to 800. Which seems quite counterintuitive if it was just for your continus block hypothesis.

Also, I'm not sure what 32 bit (or sfx) has to do with this. I'm directly using x64 7zG.

[Igor Pavlov]

So you say that your windows doesn't like big steps in allocation?

If so, you must have same situation when you extract or test that archive
with 7-Zip (32-bit or 64-bit) probably. 7-Zip in that case also allocates same 512 MB block as 7z-sfx.

So try to test that archive by 7-Zip (32-bit and 64-bit).
Do you have error messages?

[mirh]

So you say that your windows doesn't like big steps in allocation?

I.. never thought to that in this quite simple way, but yes I'd say this.
I just was unsure on whether I was doing myself something wrong or what.

If so, you must have same situation when you extract or test that archive

This is exactly what I was always doing!