2009/10/21 Guillaume Cottenceau <gcottenc@gmail.com>
On Tue, Oct 20, 2009 at 4:46 PM, Simon Arnaud <mazwak@gmail.com> wrote:
> 2009/10/20 Guillaume Cottenceau <gcottenc@gmail.com>
>>
>> Hello,
>>
>> Using my "booh-classifier" program, a rather severe memory leak is
>> exhibited on 64-bit architecture (both debian and mandriva, with
>> versions for example ruby-1.8.7-9p174 and ruby-gtk2-0.19.2) but not on
>> 32-bit architecture.
>
> Do you have any pointer where booh-classifier might be leaking ?

I'm pretty much convinced it's rg2, so it should be in rg2's general
object binding system, but then..

I meant where in booh-classifier you would think it was happening.
 
> And how to check if it leaks memory ?

Start it once first, by invoking "booh-classifier". In
Edit/Preferences, set the cache memory use to 100 MB.

Stop it, start it again by invoking "booh-classifier -v3
/path/to/photos" where the path is where you store tens to hundreds of
photos (booh-classifier will recurse, so the path may be the top
directory of some classified photos already in subdirectories).

Wait for the loading to finish, to stop the console flooding. Then
when navigating in photos (left/right keyword arrows), more photos
will be loaded and occasionally, the GC will be triggered to help
ruby's GC to notice that some rg2 objects are not references anymore,
before the size of the process grows too much (why the GC needs to be
fired explicitely is a separate, old issue, which remained
unanswered).

On an x86 architecture (for example, i686 Intel(R) Pentium(R) 4 CPU
3.00GHz GNU/Linux), you'll regularly notice in the console something
looking like:

VmSize: 216148
too much RSS, stopping preloading, triggering GC
GC in 0.184628 s
VmSize: 67052

E.g. triggering the GC allows to reclaim memory and the process not to
hammer the system too much (second VmSize figure goes below 100M)

On an x86_64 architecture (for example, x86_64 Genuine Intel(R) CPU
3.20GHz GNU/Linux):

VmSize: 335124
too much RSS, stopping preloading, triggering GC
GC in 0.083613 s
VmSize: 210428

then a while later (not necessarily immediately, so make sure to
navigate more than a couple of photos, if the problem doesn't exhibit
quickly):

VmSize: 254844
too much RSS, stopping preloading, triggering GC
GC in 0.068304 s
VmSize: 254844

then:

VmSize: 257780
too much RSS, stopping preloading, triggering GC
GC in 0.065692 s
VmSize: 257780

etc

Of course, you can also witness in "top" that the process will grow
again and again and again.


I tried on debian Sid 64, ruby 1.9 (I had to make a few changes).

vmsize at startup is around 350 MB, so, way after the 100 MB limit.

I see what you describe.

However, I replaced line 525

+ free_cache([])
- gc
- get_mem

And I was not able to reproduce the problem.

See if it works for you too.

Why it would do that on 64 bits, and not 32 is a mistery.


Simon Arnaud