Re: [Valgrind-users] 2gb limit on linux

[John R. <jr...@bi...>]

> I see , so what is maximum ram per application and where i can change it ?

If the process is a 32-bit process on 32-bit hardware, then the maximum user
virtual address space is 3GB, and you cannot change it.  The maximum physical
RAM for one process is also 3GB, regardless of whether the CPU supports PAE
or not.  PAE (Physical Address Extension) is a feature of 32-bit i686 CPUs
which allows them to use more than 4GB of physical RAM; the limit is a 36-bit
physical address, which is 64GB.  But any user-mode process can use only
3GB of that.

Also, for 32-bit hardware, there is a kernel compile-time parameter
which may limit the per-process user address space to less than 3GB;
common values are 2.5GB or 2GB.  Such a decision is common for "server"
configurations because keeping track of thousands of simultaneous
processes, or large amounts of physical RAM (more than 5GB or so)
requires more kernel address space, and 1GB (4GB - 3GB) often is not enough.
So the good news about an i686 with PAE and 64GB of RAM is that
the system as a whole can use all 64GB.  But the bad news is that
any one process might be limited to only 2.5GB or 2GB at a time
just because it is harder (requires more kernel address space)
to keep track of it all.   You'd be better off with only 4GB of RAM,
3GB of user virtual space, and fewer other processes to contend with.
[Get your own machine, don't share with anybody else.]

If the process is a 32-bit process on 64-bit hardware, then the maximum
virtual address space is 4GB, and you cannot change it.  The maximum
physical RAM per 32-bit process also is 4GB.  PAE is irrelevant because
it applies only to 32-bit CPUs.

If the process is a 64-bit process on 64-bit hardware, then current
hardware CPU implementations limit the maximum virtual address space to
40, 44, or 48 bits (1TB, 16TB, 256TB; the chip ignores the top 24, 20,
or 16 bits of an address except to verify that they are the sign
extension of a 39, 43, or 47-bit address) and current Linux kernels
limit the user-mode address space to one bit less (one half the maximum.)

In all cases [all three above] the Linux kernel allows the system admin-
istrator to set a policy which further limits the total virtual space used
by all simultaneous process to be the maximum of the amount of physical
RAM and the size of swap space.  This is the "no overcommit" policy: assume
that whenever a process asks for pages of address space, then eventually
the process will use it all, and the kernel must be prepared to honor
100% of the commitments that it has made.  However, for many workloads
the "no overcommit" policy is far too pessimistic; RAM will be under-
utilized because processes tend to use less than they ask for, especially
when averaged over all processes at any instant in time.  Often it is
reasonable to overcommit (let the sum of the allocations that have been
granted exceed the maximum of RAM and swap space) by 50%, 100%, or more.

> Does using pae(supports 64GB ram) is increasing this limit(per
> application not whole ram usage)?

No, PAE has essentially nothing to do with just one process.  PAE is for
allowing a 32-bit CPU to use more than 4GB of RAM to support many processes
whose individual address spaces are each at most 3GB (and often 2.5GB or 2GB)
but whose total instantaneous RAM utilization, summed over all processes,
may be larger than 4GB.

--