Re: [uml-user] memory use

[Paolo G. <bla...@ya...>]

On Saturday 18 December 2004 16:55, Rog...@gd... wrote:
> > -----Original Message-----
> > From: Paolo Giarrusso [mailto:bla...@ya...]
> > Sent: Thursday, December 16, 2004 2:05 PM
> > To: use...@li...
> > Cc: Sala, Roger
> > Subject: Re: [uml-user] RE: User-mode-linux-user digest, Vol 1 #1829 -
> > 11 msgs
> >
> >
> > [...]
> >
> > > In an attempt to test this, I rebooted and repeated all the
> >
> > above steps,
> >
> > > except I lied to the uml and told it that it had three
> >
> > times more memory
> >
> > > than the $TMPDIR partition.
> > >
> > >  `linux ubd0=/dev/hda6 ubd1=/dev/hda5 mem=180000K
> > > eth0=tuntap,,,192.168.0.254`
> > >
> > > `top` on the guest now happily reported 153240K available
> >
> > at the start.
> >
> > > Everything went down similarly when I ran the `find`; the
> >
> > host sucked down
> >
> > > its available memory very quickly -- although this time it
> >
> > did eventually
> >
> > > swap.  The guest gobbled up its available memory also, but
> >
> > at a slower
> >
> > > rate. I expected it to crash after it used 60000K, and then
> >
> > 120000K, but it
> >
> > > used up just about all its reported "available" memory and
> >
> > still kept
> >
> > > running for a while.  It never used any of its own swap.
> > >
> > > Continuously running `df /mnt/ram6` showed the free memory
> >
> > eventually go
> >
> > > right to zero, shortly before the guest segfaulted.
> >
> > Checked the value without the cache size?
>
> Thank you for pointing out the buffers/cache component of free/used memory,
> but after reading up on it a bit, its my impression that, although this
> memory is readily usable if needed, it is still memory that has been
> written to -- and thus should show as used when running `df` on the host's
> ram disk/$TMPDIR.
>
> > > My impression is that the guest is not using its
> >
> > "dedicated" ram or swap
> >
> > > very efficiently.
> >
> > 2.6.9 kernels, on any arch, often go OOM instead of swapping
> > (discussed
> > somewhere on http://lwn.net/Kernel).
>
> Thanks again.  I turned up the swappiness on the guest kernel to 100 and it
> indeed swapped.
>
> > > It also seems to be able to get into the host's ram
> > > somehow.
> >
> > Hmm, quite unlikely...
>
> I'm inclined to agree with you.  I think a more likely explanation for what
> I'm seeing is that `top` is reporting all the memory that has been
> malloc'd, whereas `df` on the ram disk is reflecting only the pages that
> have actually generated a page fault and been allocated.
You're right for df; the total usage from top follows the same rule, while for 
each process you can know the total malloced and so on memory in the "VIRT" 
column. It's the virtual address space size, which also includes all the 
mmap()ed memory, all the stack and so on.... while "RES" only counts the 
pages which are actually brought in RAM.

> I'm also suprised how much host memory usage the guests generate.  When I
> start a uml, the host usage jumps by about 40M, which is just about all
> given back as soon as the uml is halted.  If I start a uml and run the
> `find`, the host's free memory is quiclky depleted -- in this case about
> 130M.  Again when the uml is halted, just about all the 130M is freed. 
> This is above and beyond the 65M ramdisk that is carved out before the
> guest is started, and which is not given back when the guest is halted.
>
> This, along with the "Host Memory Usage" uml page leads me to believe that
> creating a ramdisk (for the sake of partitioning resources) isn't buying me
> that much since the host is (copying?) all the guest's pages anyway.
Try with a tmpfs, which is the commonly used and recommended road... I still 
think that a ramdisk should be the same thing, but however...

I also remember reading that the RAM is doubly counted... search on 
kerneltrap.org for "Using RAM as swap", and you'll find in the article also 
something about this... (even if, in that case, the problem was that the 
buffers used RAM, i.e. the RAM used as ramdisk, is also the SWAP used 
memory).
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729
http://www.user-mode-linux.org/~blaisorblade