Although size_t is a 64 bit value in 64 bits, malloc appears to fail when trying to allocate more than the 32-bit 2 GB. Is this true, and if so, is there an alternative to malloc in MinGW64 which would go beyond the 2GB limitation?
Have you tried HeapAlloc or other *alloc API?
As far as I know is max heap-size for 64 bit Windows msvcrt 0xFFFFFFFFFFFFFFE0ULL size. Nevertheless you need to take care that you have enough virtual-memory for this. Please check your maximum for workspace you have specified.
Hi Jon and Kai,
Thanks for your replies. I will try HeapAlloc, but would rather avoid it to keep the code as portable as possible.
The heap size is not the problem: the application happily allocates 24 Gigs… but malloc appears to fail when I try to make it allocate more than 2G in one SINGLE CHUNK - is there a way of checking this with the MinGW64 developers (or else I'm doing something wrong)?
I've stumbled on this post:
that suggests it may not be malloc the culprit, but filelength, as I'm trying to allocate space returned by filelength on a file larger than 2GB.
But the post is more than a year old, so maybe filelength was fixed since then… so the question remains open.
Thanks & cheers,
Solved (sorts of):
Looking into MinGW includes, I found that filelength indeed seems to return a long (32 bits), but there is _filelengthi64 declared - and that correctly reports the 64 bit length of the file. The subsequent malloc indeed works correctly, allocating 3GB in that single chunk.
It would be nice to have the standard filelength return size_t, so that the code would be the same for all compilers… but at least there is a workaround.
Can you explain exactly (for a newbie) how you overcame this limit. I'm compiling with Ming64 on a windows enterprise 64 bit OS and can't get malloc to use more than 2 GB of memory…
Your help is apprecated
I found that MinGW64's malloc actually does NOT have the 2GB limit, I'm using it to allocate more than 4 Gigs without any problem.
My problem was caused by the fact that I was calling malloc with a long parameter, and longs are 32 bits even in MinGW64. Maybe you have a similar problem - to find out, try to call malloc(0xEE6B2800), i.e. allocate about 4 Gigs: I'm confident you'll get all those 4 Gigs, and will then just need to find out where exactly the amount you want to allocate goes through a long to end up smaller than 2 Gigs.