#4863 possible memory leak?

obsolete: 8.6b1.1
open
5
2011-07-28
2011-07-28
Anonymous
No

Using the latest ActiveState 8.6 and the Thread package on Ubuntu based distro, 32 bit, kernel 2.6.35-30
I ran the following script 4 times:

time {set ::t [thread::create]; thread::release $::t} 10000

Here are the results from htop, one line for each invocation, first line is a state before the first run.
VIRT RES SHR
15268 3296 1824
32884 4632 1964
33104 4936 1972
33328 5160 1972
33592 5384 1972

There seems to be 100-300 byte leak for each invocation of the script.

Discussion

  • Alexandre Ferrieux

    Could this just be history building up, in an interactive shell ?
    If not, try increasing the [time] itercount and checking the effect on the slope.

     
  • miguel sofer

    miguel sofer - 2011-07-28

    To remove history problems do one of:
    - run the thing in an infinite loop (with an [after 10] in there), watch what top has to say about mem
    - do [proc history args {}] before the test

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2011-07-29

    Ran a new test, non-interactive, wrapped in a proc, 100000 iterations. The results are:

    VIRT RES SHR
    START 32476 4192 1960
    END 36284 7088 1960

     
  • Alexandre Ferrieux

    OK thanks. Now what about 1*100000 vs. 10*10000 ? This to decide whether it leaks rather "per [times]" or "per [thread::*]" invocation.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2011-07-29

    Here are the 10*10000 results:

    32872 4652 1960
    34120 4964 1960
    34384 5244 1960
    34664 5504 1960
    34888 5732 1960
    35116 5956 1960
    35380 6204 1960
    35584 6428 1960
    35868 6672 1960
    36092 6916 1960

     
  • Zoran Vasiljevic

    Try this:

    time {set ::t [thread::create -joinable]; thread::release $t; thread::join $t} 10000

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2011-07-30

    Results for:
    time {set ::t [thread::create -joinable]; thread::release $t; thread::join $t} 10000

    ran ten times ---

    24392 4284 1968
    24616 4528 1968
    24844 4752 1968
    25068 4996 1968
    25292 5220 1968
    25516 5464 1968
    25744 5692 1968
    25968 5936 1968
    26192 6160 1968
    26416 6404 1968

     
  • Don Porter

    Don Porter - 2011-08-01

    any progress?

     
  • Zoran Vasiljevic

    Not yet. I do not think we can do anything here fast, as most probably
    this is some deep finalization issue in Tcl library. The extension itself
    does not leak at this point but the core could. I do not have resources at
    the moment for deeper analysis but will throw a valgrind againt it just to
    be 100% sure. If you aim at speedy releasing 2.6.7 I think you should not
    wait.

     
  • Zoran Vasiljevic

    This is not the issue of the threading extension. I have tried the Tcl 8.4 core
    with 2.6.6 thr-extension on Linux and Mac and double-checked with valgrind.
    There are no leaks there at all. The Linux top shows pretty much same virtual
    process size no matter now often I repeat the following command;

    time {set ::t [thread::create -joinable]; thread::release $t; thread::join $t}

    I run if 1000000 times and then check vith valgrind and top and there is no
    leak. I guess later (8.5/8.6) cores might not be leak-free; i haven't tested those releases as we are still using 8.4 core.

    Unfortunately I missed the new move-fro-sf-cvs so I do not yet) khow where
    are the new sources and I need to catch-up with that.

     
  • Twylite

    Twylite - 2011-08-01

    Using 8.6b2rc1 I can reproduce a leak on WIN32 using the time{} loop recommended by Zoran. Only the Windows build leaks about 135,000 bytes per iteration. Task Manager's thread count confirms that all additional threads have stopped.

    For reference, a simple interp create/delete loop doesn't seem to leak memory, implying that this isn't an interp leak.

     
  • Zoran Vasiljevic

    I think the problem is in the Tcl finalization area.
    The interp itself is clean. The finalization isn't :-(
    Some years ago I contemplated to rewrite the
    finalization code using refcounts but this was a
    major undertaking and I never found time for it.
    The 8.4 seems to be more stable in this respect
    which is why we still stick to it.

    In the meantime, creating a set of worker threads
    upfront (or using tpool with fixed number of workers)
    should keep the memory in the "reasonable" area.
    Also, turning off the greedy threading allocator that
    never releases memory to the system and using the
    regular system malloc instead could also be a good
    move. I recall we turned that off some years ago and
    I cannot recall our app becomming slower (yet it
    became a fifth of its original memory footprint!).

    I think this could be a hard nut to crack. In some days
    I will get more spare time to checkout the latest head
    and see what all happened there since the last time
    I looked at it.

     
  • Don Porter

    Don Porter - 2011-08-02

    Using the usual tcltest harness for memory leak testing,
    I tried this:

    testConstraint memory [llength [info commands memory]]
    test thread-8.1 {Bug 3381371} -constraints memory -setup {
    proc getbytes {} {
    set lines [split [memory info] "\n"]
    lindex $lines 3 3
    }
    package require Thread
    } -body {
    set end [getbytes]
    for {set i 0} {$i < 5} {incr i} {
    set t [thread::create -joinable]
    thread::release $t
    thread::join $t
    set start $end
    set end [getbytes]
    }
    expr {$end - $start}
    } -cleanup {
    rename getbytes {}
    unset -nocomplain i start end t
    } -result 0

    It shows no leak, using the trunk of Tcl & Thread.

    What is it that's claimed to be leaking? Apparently
    it's not Tcl_Alloc'ed memory? Or is this a leak I
    should not be able to see testing on Linux?

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks