Menu
▾
▴
#271 Object creation/deletion speed and memory usage
Object creation, deletion, and memory usage is ~10x worse in Tcl8.6.4/Itcl4.0.3 than in Tcl8.5.18/Itcl3.4.2.
Demo script:
package require Itcl namespace import itcl::* class A {} set i 0 puts [time {A $i; incr i} 10000] set i 0 puts [time {delete object $i; incr i} 10000]
Results:
+-------+------------+------------+-----------+ | Ver | Create | Delete | Memory | +-------+------------+------------+-----------+ | 3.4.2 | 6.0635 µs | 3.4106 µs | 6.724 MB | | 4.0.3 | 50.0183 µs | 40.6675 µs | 86.556 MB | +-------+------------+------------+-----------+ | ratio | 8.2490 | 11.9238 | 12.872 | +-------+------------+------------+-----------+
Ticket http://core.tcl.tk/itcl/tktview?name=04470f9067 improved the speed immensely, but the current inefficiency may still hinder any future projects using Itcl.
Hi is there any news on this one? We used [incr Tcl] 3.3 and Tcl 8.4 we upgraded to Tcl 8.6 and [incr Tcl] 4.0.3 and now we're really suffering from this one. Memory leaking and really slow. Any hints as to reason or workaround? Thanks!
If that's not a typo, then updating to Itcl 4.0.4 will help you
a lot. Maybe not enough, but much better than 4.0.3.
If Itcl 4.0.4 still can't meet your needs, then I'd at least
recommend that you use the Tcl 8.6.5 / Itcl 3.4.3
combination so at least some of your foundation is
up to date.
If you can separate your two problems -- 1) performance,
and 2) memory leaks -- and report demo scripts for the
leaks, that would be of very great assistance in finding
and pluggin them. Only against Itcl 4.0.4 please! :)
Yep it's 4.0.3, thanks for the advice we'll try 4.0.4. Regarding demo scripts, ours is practically identical to the one above! Thanks again!
Using [incr Tcl 4.0.4], packaged with Tcl 8.6.5, there's a very big improvement in performance, but memory is leaking, my process grows by about 6MB each time I run the above example and memory use doesn't drop afterwards.
An end-user reports that using oo directly instead of [incr Tcl] greatly reduces the leak. Maybe that helps locate the problem.
I'm getting hassle building itcl 3.4.3 on Tcl 8.6.5. If it's known that this is possible, I'll persist.
ok, I did a simple check, and there does appear to be
a possible leak to track down and plug. Will look into it.
For build problems, we can only help with some
indication what the problem is. Itcl 3.4.3 builds
against Tcl 8.6.5 headers for me on a Centos Linux
system. Open a new ticket when you have info
for us.
Thanks for the help testing Itcl!
Leak: many thanks.
Building 3.4.3 with 8.5.6 issues: assume this is my mistake/problem. I just wanted to check, before going deeper in, that it was worth continuing, i.e. this is not something that fundamentally can't work.
You typo'd that, right? 8.6.5 and not 8.5.6 ?
Sorry. 8.6.5 is what I'm building against, and 3.4.3 on top of it
~~~~
Reading between the lines above, if there's a demo of
a leak in Tcl's own oo::* system the Tcl devs would
like to be made aware of it.
Certainly ... our measurements with 8.6 show that many or even most of the plugins we use have leaks to some extent. We don't suffer much from them because they don't get triggered by loading or are not used at very high rates. Incr Tcl seems to suffer more when small-scale objects are frequently created or deleted in large numbers, just like in the example, which unfortunately our end user wants to do. I advised him slightly against this but he is a very keen OO programmer and I believe he sees [incr Tcl] as the best way to get around the (real or perceived) faults of Tcl. In practice this is currently the main issue we are having with Tcl 8.6. He will in any case be happy with the performance improvement because he is finding that just creating a combobox takes ages. If I'm honest I slightly regret migrating to 8.6 now, but the same end user wanted it and we didn't know in advance we would hit this kind of trouble.
In any case many thanks for your help!
Timeline for Itcl development is here:
http://core.tcl.tk/itcl/timeline
Over the last few days I plugged several memory
leaks observed in cycles of simple object creation
and deletion. You should be able to see improvement.
If you have trouble working from the dev systems
and need something more like a source code release
for testing please make that known here.
Thanks, since it's not me so much as my end user who is impacted, then for me it would be better wait for next minor release, so I can quickly and safely pass it on.
Cycles of creation/deletion of instances of an empty class
now appear leak free. No claims beyond that.
Excellent; will you try with a trivial example non-empty class?, e.g.. with one member variable and one class method; I would expect if that is OK then we can be very confident of a big improvement. Thanks again very much for the effort.
Good advice. I just ran this test script:
package require Itcl
itcl::class A {
public variable v
constructor {} {set v 1}
method get {} {return $v}
}
while 1 {
A foo
foo get
itcl::delete object foo
}
...and I detect no memory usage growth.
Wonderful, really thanks. So now I need to work out the best way to integrate it; do you happen to know when is the next minor Tcl/Tk release that would include this?, or should I update [incr Tcl] only and leave our underlying Tcl/Tk 8.6.4 as-is?
Don, there's a subtle difference between the original test and the one you ran, isn't there?
The one you ran doesn't terminate.
We have noticed that memory usage stays stable if you continuously run with creation and deletion or repeatedly source the same script in the same interpreter.
The original example leaks about 600K each iteration. That is better than the 6MB per iteration seen with [incr Tcl 4.0.4], packaged with Tcl 8.6.5 (comment 2-June) but still seems to be a leak when interpreter exits.
Putting the example in a namespace eliminates the leak:
package require Itcl
namespace import itcl::*
namespace eval X {
class A {}
set i 0
puts [time {A $i; incr i} 10000]
set i 0
puts [time {delete object $i; incr i} 10000]
}
No... previous comment is red herring, repetition shows leak.
Could you confirm how Ictl_FinishCmd gets called, leading to Itcl_DeleteStack? What might fit the pattern we measure is that when the number of objects fits within a certain number there is no leak; when there is a large number, it appears to leak. This comment may be a hint:
OK ... I rebuilt with a modified IctlFinishCmd repeating this pattern for all the hash tables mentioned in infoPtr (except infoPtr->objects). Then the per-iteration leak goes away.
while (1) { hPtr = Tcl_FirstHashEntry(&infoPtr->instances, &place); if (hPtr == NULL) { break; } Tcl_DeleteHashEntry(hPtr); } Tcl_DeleteHashTable(&infoPtr->instances);The attachment (no claims on quality) does a couple of other things:
I should point out there are two calls to Tcl_DeleteHashTable on the same table (procMethods) in the attached "fix", which when removed is quite stable. Note: if one believes the Tcl_DeleteHashTable documentation it should not be necessary to delete all the entries first. I didn't feel confident to continue with more changes.
Thank you for these submissions. I'd like to adopt them, but for my own process, I'd really like to be able to replicate the leak for myself so I can see the problem they solve. I'm not having any luck with that. Can you spell out step by step how I can see the leak so I can discover what difference we have in our testing? Thanks.
If you run the original test case in a single process e.g. tclsh, and then repeat in the same interpreter, no leak. If you exit that interpreter, and the process it is running in, no leak is visible, but of course the process has exited. If you e.g. run in a multithreaded environment, the interpreter exits, but the process continues. If you run in another interpreter in another thread, then the heap allocations done by the first interpreter have not been correspondingly deleted. You won't want to build our executable, but it might be possible to reproduce this with thread package, e.g. thread::create -script DemoScript.tcl ? where demoscript is as above.
I believe the reason repeating the demo script in the same interpreter doesn't leak is that it happens to create variables of the same name, so the hash tables that haven't been deleted still contain the same keys. You (we) might equally be able to reproduce the problem in a single interpreter by finding a way to make successive iterations in the same interpreter create new object names.
I am sorry I can't be more helpful. Actually for me the immediate leaking problem is solved with the provided file (after removing the silly mistake). I suspect the solution isn't the most minimal possible solution. e.g. I think it should be be possible to just delete the hash tables, without running through the contents.