Re: [Lush-users] Problem with garbage collection

[Ralf J. <jue...@cs...>]

Hi Fabian,

I looked into this a little bit and don't think that anything
is broken. I see that the list data is released when it
becomes unreachable (you can repeat the sequence of creating
and releasing the list without seeing the processes' memory
usage going up).

I guess what's happening is that some code is requesting
heap memory after the memory-heavy list has been created and
this allocation is made at some unusually high address. As
long as this piece of memory is in use the OS cannot shrink
the data segment of the process even though the heap is almost
empty. If that's the case then the high memory reading from
ps is widely off the mark. It's an annoyance for sure, but no
no physical memory is lost thanks to virtual memory management.

It would be interesting to see the origin of this hypothesized
memory request. If I find time I will dig a little more.

Cheers,
Ralf



On Thu, 14 Oct 2010, Fabian wrote:

>
> Hi all,
>
> there's a problem with the garbage collector.
> Here is an example:
> You create a list of idx2 (approximately 1.2GB)
> and then try to erase it again:
>
> ;;execute the following:
> (meminfo) ^P(sh (concat "ps u " (str (getpid))) )
> ;;create the list
> (discard (setq l-test (mapfor (i 0 150) (double-array 100 100 10 10)))) ;;1.2GB
> ;;erase it. The first time everything works
> (setq l-test ()) (gc)
> (meminfo) ^P(sh (concat "ps u " (str (getpid))))
> ;;-----------------------
> ;;but now create the list a second time:
> (discard (setq l-test (mapfor (i 0 150) (double-array 100 100 10 10))))
> ;;erase it
> (setq l-test ()) (gc)
> ;;now you are left over with the garbage
> (meminfo) ^P(sh (concat "ps u " (str (getpid))))
>
>
> The first time you create the list and erase it again everything works
> fine, but when you do it a second time the last line gives you:
>
> ;;output from: (meminfo)
> Small object heap: 256.00 MB in 65536 blocks (830 used)
> Managed memory   : 3.18 MB in 178539 + 291 objects
> Memory used by MM: 6.52 MB total
> = ()
> ;;output from: ^P(sh (concat "ps u " (str (getpid))))
> ("USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME
> COMMAND"
>  "fabian    3700 43.8 58.3 1525364 1205040 pts/14 Ss+ 10:11   0:04
>  /usr/local/bin/lush2" )
>
> so the output of meminfo says everything is OK, but unfortunately it is
> not. "ps u" says, that there is still 58.3% of memory allocated,
> and it is impossible to get rid of it.
> On my system with 2GB ram, this is 100% reproducible. When you want
> to check this on your system you should take into account how much
> memory you have. For example if you have 4GB, then choose "(mapfor (i 0 300) .."
> Further observations:
> * When I choose a smaller list "(mapfor (i 0 130) .." then everything works fine !!
> * you have to mix idx with normal lists. When you create only one idx,
>  or only one big list everything works.
>
>
> With best regards
> Fabian
>
>
> ------------------------------------------------------------------------------
> Beautiful is writing same markup. Internet Explorer 9 supports
> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
> Spend less time writing and  rewriting code and more time creating great
> experiences on the web. Be a part of the beta today.
> http://p.sf.net/sfu/beautyoftheweb
> _______________________________________________
> Lush-users mailing list
> Lus...@li...
> https://lists.sourceforge.net/lists/listinfo/lush-users
>