Re: [Squeak-VMdev] Re: GC improvements

[Andreas R. <and...@gm...>]

John,

> You miss the point that after N MB of memory growth, we do a full GC
> event.
>
> The logic to do the full GC is either in the smalltalk code running as
> active memory monitoring, or can be moved into the image.

My criticism here is that the *default* without running that extra code is 
not sensible. In other words, without active monitoring and memory tuning 
you will indeed run out of memory. And that's just not acceptable - there 
are plenty of people who will want to run a VM with as little overhead as 
possible.

> I'm not growing the image endlessly. I've attached two jpegs of
> before/after memory end boundary charts when a person was working
> in a croquet world, not a borderline case.

You're making my point ;-) Without the extra GC monitoring code (which 
people may not have, may not be aware about, may not want to run) the system 
would indeed rapidly grow beyound reasonable limits.

> Also two jpegs from a seaside application (again not a borderline case)
> which were generated by doing:

Yup. Again making my point - there need to be sensible defaults in the VM 
before this strategy can make sense. You may want to be able to "tweak it 
away", e.g., set it to "unreasonable limits" to be able to have manual 
control from inside the image, but the VM needs to react sensibly even 
without that extra tuning code.

Bottom line: If we want these changes, we need a sensible mechanism in the 
VM to avoid unbounded memory growth.

> Jerry has to confirm what he did and if it was repeated mostly the
> same, but it did do 65,000 to 70,000 young space GC and it appears
> we reduced the young space GC time by 40 seconds.  This does result in
> more full GC work (5) since I tenure about 16MB before doing a Full GC,
> but that accounts only for an extra second of real time...

Agressive tenuring would probably work around some of the problems we've 
seen in the past (see 
http://croqueteer.blogspot.com/2005/01/need-for-speed.html) though in 
general, it seems as if Croquet performs significantly better if you allow 
for more allocationsBetweenGCs - it seems as if the working set of Croquet 
is larger than your average Squeak working set. This also explains the stats 
you are getting - you're tenuring like mad because youngSpace is too small 
and then at some point where you hit fullGC you are "back to normal". This 
probably shouldn't be "fixed" by tenuring but rather by tweaking the 
allocationsBetweenGCs.

Also notice that the stats look like with the agressive growths logic we are 
doing more fullGCs than without them (this isn't totally clear from looking 
at the pictures but I would expect more spikes in the before part if it had 
more fullGCs). This is not necessarily a good thing - I would rather spend a 
few percent more on IGC than having to run fullGCs in Croquet.

Cheers,
  - Andreas