Menu
▾
▴
[Tcl-bugs] [ tcl-Bugs-3382401 ] Thread pool leaks memory
|
From: SourceForge.net <no...@so...> - 2011-07-29 22:18:30
|
Bugs item #3382401, was opened at 2011-07-29 12:34 Message generated for change (Comment added) made by hobbs You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=110894&aid=3382401&group_id=10894 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: 80. Thread Package Group: None Status: Open Resolution: Accepted Priority: 7 Private: No Submitted By: Christian Gollwitzer (auriocus) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Thread pool leaks memory Initial Comment: Using thread pool to do a computation in parallel with many thousand tasks, I noticed, that tpool leaks memory. The script attached demonstrates the loss both in ActiveTcl 8.5.10.1 on Windows/64 and in tcl8.6 HEAD on Mac OSX. My real computation ran for several hours on 6 cores and the leaked memory summed up to several hundred MB. I could not locate the leakage any further, first I thought the culprit was the tpool::wait command with the list result, but replacing it with a simple "after" in the test case has no influence. Also, tpool::release can not free the leaked memory. ---------------------------------------------------------------------- >Comment By: Jeffrey Hobbs (hobbs) Date: 2011-07-29 15:18 Message: I am able to repro the leak on Windows with ActiveTcl 8.5.9.2 and 8.4.19.5 using Thread 2.6.7 (not final). ---------------------------------------------------------------------- Comment By: Andreas Kupries (andreas_kupries) Date: 2011-07-29 14:59 Message: Oh, the question about '... did you try 2.6.6 release' was for Christian ... ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2011-07-29 14:58 Message: yes. there are no significant core-code related changes there. if 2.6.6 does not leak, the 2.6.7 will not either. ---------------------------------------------------------------------- Comment By: Andreas Kupries (andreas_kupries) Date: 2011-07-29 14:54 Message: You have seen that Don Porter has RC for 2.6.7 up ? See tcl-core mailinglist. ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2011-07-29 14:51 Message: Hey, thanks Andreas! I think I will need to do some reading and learning new stuff... Apropos reported leak... did you try 2.6.6 release? I saw I have already corrected some things related to tpool already... ---------------------------------------------------------------------- Comment By: Andreas Kupries (andreas_kupries) Date: 2011-07-29 14:48 Message: Thread sources. See http://core.tcl.tk/ in general, and http://core.tcl.tk/thread/timeline?y=ci specifically. Further http://wiki.tcl.tk/28127 and http://wiki.tcl.tk/28126 ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2011-07-29 14:45 Message: I think I would need to release a new version. My own private sandbox seems to work fine. The 2.6.5 that gets distributed with Mac OSX seems to leak. But I need to figure out where are the sources! I heard SF is not hosting the sources any more... ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2011-07-29 14:27 Message: yes. this eats up memory somewhere. will double-check and fix. thanks for reporting this! ---------------------------------------------------------------------- Comment By: Christian Gollwitzer (auriocus) Date: 2011-07-29 13:55 Message: OK, so try the third version. I shortened the inner loop and made more iterations. Now you'll see, that also virtual memory winds up. ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2011-07-29 13:46 Message: The virtual memory is what counts. The real memory is not important here. If virtual memory does not increase you do not have leak. The OS versioning is weird by Mac, yes. I also have SnowLeopard but 10.6.7. This is no problem. ---------------------------------------------------------------------- Comment By: Christian Gollwitzer (auriocus) Date: 2011-07-29 13:17 Message: My platform: 91-65-210-59-dynip:Programmieren chris$ tclsh8.4 % parray tcl_platform tcl_platform(byteOrder) = littleEndian tcl_platform(machine) = i386 tcl_platform(os) = Darwin tcl_platform(osVersion) = 10.8.0 tcl_platform(platform) = unix tcl_platform(threaded) = 1 tcl_platform(tip,268) = 1 tcl_platform(tip,280) = 1 tcl_platform(user) = chris tcl_platform(wordSize) = 4 % PS: Funny version. My OS is Snow Leopard 10.6.8, why does it say 10.8.0? ---------------------------------------------------------------------- Comment By: Christian Gollwitzer (auriocus) Date: 2011-07-29 13:14 Message: I've updated the script for 8.4 without max. If I run it with tclsh8.4 threadpool_leakdemo84.tcl then the value in activity monitor "physical memory" winds up, every ten seconds +0.1MB. It starts from 3.0 and goes up to 6MB. You are right that "virtual memory" does not change; I wonder, why. ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2011-07-29 12:47 Message: I cannot confirm this on Max OSX using Tcl8.4 % array get tcl_platform osVersion 10.7.0 byteOrder littleEndian tip,268 1 threaded 1 machine i386 platform unix os Darwin tip,280 1 user zoran wordSize 4 % set tcl_version 8.4 When I start your script it ocmplains about math max function. I remove that out of the code (it is not really related to anything thread-wise) and run the script. It allocates some 30MB virtual memory and stays there no matter howmany times i run your leak proc. I gues this has something to do with new(er) Tcl cores. I have not tested anything later then 8.4. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=110894&aid=3382401&group_id=10894 |