Re: [Lse-tech] Re: Hugetlbpages in very large memory machines.......

[Andy W. <ap...@sh...>]

Here is the next installment of HUGETLB memory accounting.  With the stack
applied (to 2.6.4) HUGETLB allocations are be handled separately to those
for normal pages.  The set has been tested lightly on i386.  Other
architectures have not yet been compiled (testers please).  Currently there
are no tunables for overcommit.  Again patches attached, ask if you need
them inline.

This patch has an interesting and I believe correct side effect.  Memory is
now committed when a hugetlb segment is initially requested, even before it
is attached.  Thus it is no longer possible to shmget many large segments
and have them fail to attach.

The patch list below ...  Comments??

-apw

010-overcommit_docs:		documentation changes
015-do_mremap_warning:		cleanup exit handling to prevent warning
050-mem_acctdom_core:		core changes to create two accounting domains
055-mem_acctdom_arch:		architecture specific changes for above
060-mem_acctdom_commitments:	splits vm_committed into a per domain count
070-mem_acctdom_hugetlb: 	use vm_committed to track HUGETLB usage
075-em_acctdom_hugetlb_arch:	architecture specific changes for above

The first two patches are cosmetic fixes, either in documentation or to
remove a warning later in the game.

The third and fourth patches patches set the scene.  These are the most
tested and it is these that I hope Anton can test for us with his "real
world" failure mode. These two patches introduce the concept of a split
between the default and hugetlb memory pools and stop the hugtlb pool being
accounted at all.  This is not as clean as I would like, particularly the
need to check against VM_AD_DEFAULT in a few places.

The fifth patch splits the vm_committed count into a per domain count and
exposes the domain in the interface.

The sixth and seventh patch converts hugetlb to use the vm_commitment
interfaces exposed above.