From: Chuck E. <ChuckEsterbrook@StockAlerts.com> - 2002-02-23 22:33:24
|
On Friday 22 February 2002 08:59 pm, Jay Love wrote: > Chuck- > > Am I seeing things, or are the sessions leaking memory again? > > Lower the session expiration time. Hit the CountVisits servlet > several thousand times and watch the memory creep up. Then, as the > sessions expire, I don't see the memory coming down like it should. > > Do you see this? > > Jay Basically, yes, but read on: I'm using SessionMemoryStore with a time out of 1 on Python 2.1.1 on Linux with Webware cvs. I cleared out my Sessions/*.ses before launching the app server. The memory grew and after the sessions had expired (according the Admin servlet), the memory was the same, which smells like a leak, BUT... What's also interesting is that when I reran the test program, after expirations, the memory didn't grow anymore except for a couple blips (that came back down). It was like Python would not let go of the memory it didn't need, but it was certainly willing to reuse it. On a third run, I got the same thing: just a little more creep. On the 4th run, no creep at all. On the 5th run, I lose 8K right away and then another 8. On the 6th run, no creep at all. I'm letting the sessions expire between runs. So is this retain-but-reuse-memory a Python thing? A Linux thing? WebKit? My program is in CVS at: Webware/WebKit/Research/TestForLeaks.py Give it a spin. See what you think. -Chuck |
From: Baruch E. <ba...@ev...> - 2002-02-24 06:52:32
|
On Sun, 2002-02-24 at 00:33, Chuck Esterbrook wrote: > > I'm using SessionMemoryStore with a time out of 1 on Python 2.1.1 on > Linux with Webware cvs. I cleared out my Sessions/*.ses before > launching the app server. > > The memory grew and after the sessions had expired (according the Admin > servlet), the memory was the same, which smells like a leak, BUT... > > What's also interesting is that when I reran the test program, after > expirations, the memory didn't grow anymore except for a couple blips > (that came back down). > > It was like Python would not let go of the memory it didn't need, but > it was certainly willing to reuse it. Python is written in C and the usual way the C library (libc6 aka glibc2) is that only large memory blocks are allocated by mmap and are released to the OS upon free. For smaller memory blocks the C library will request more memory for the process and will not return it until the end of the program. The memory that is freed is not released to the OS but is reused by the C library. Ofcourse fragmentation of memory allocation can cause the need of the C library to request more memory. I believe that what you see can be attributed to the actions of the C library and is less likely to be a memory leak in Webware, though that needs a more explicit checking, which I have no idea how to do. Maybe Python exports some information on its memory use? Baruch p.s. In the last few days I've restarted working with WebKit/MiddleKit/Cheetah/FFK and it's just so nice, at least once I figure out the easy way for each component. |
From: Ian B. <ia...@co...> - 2002-02-23 23:09:07
|
On Sat, 2002-02-23 at 16:33, Chuck Esterbrook wrote: > I'm using SessionMemoryStore with a time out of 1 on Python 2.1.1 on > Linux with Webware cvs. I cleared out my Sessions/*.ses before > launching the app server. > > The memory grew and after the sessions had expired (according the Admin > servlet), the memory was the same, which smells like a leak, BUT... > > What's also interesting is that when I reran the test program, after > expirations, the memory didn't grow anymore except for a couple blips > (that came back down). Well, as I remember the allocated memory doesn't cause any load on the machine until it is actually used -- once it's actually uesd, I don't know if it can be put back without freeing it. Maybe Python's being smart by not freeing memory that it will probably need later (since an application will usually not shrink in memory except temporarily). Maybe it actually is using resources, but again Python doesn't free memory quickly because there's a good chance it will be needed again -- if that was the case, I'd imagine there'd be some command-line or compile-time option to turn that off. Maybe Python would actually free that memory after some time...? Anyway, the behavior is probably deliberate, and probably a good idea. It may or may not be the same on Windows. I think top should show some distinctions between these different types of memory use. Ian |