|
From: Julian S. <js...@ac...> - 2006-08-25 21:11:07
|
> I've also noticed that bitmaps are never free'd, apparently because even > client free's never cause the associated client bits to be thrown away. For > the puprose of drd, I guess we have to assume that the application does not > have memory management errors, and therefore free should be able to discard > shadow allocations. > > overall this leads to every allocation causing two entire pages to be > allocated and never be free'd, which leads to quickly eating up all > available system ressources. Oh, that's really bad :-( Thanks for digging it up. So, indeed the problem is made a lot worse by the bitmaps and client data being interleaved in the address space. > I've tried the aspacemngr handling, but it only made it worse. Hmm, you mean using VG_(am_shadow_alloc) to grab the bitmaps? That's strange. I've just been playing around with this, but am getting confused. I can't get a konq startup below 2GB. > On the > bright side, I didn't have to implement freelist handling since free was > never called :) I think that's a clue :-) Ok. Question for Bart. Running with --trace-segment=yes gives a clue. I _think_ the problem is that drd is generating thousands of segments. konq is single threaded, but is doing pthread_mutex_lock/unlock and I believe this is creating segments, which AIUI is correct. Unfortunately it never throws any segments away (hence Dirk's observation that no bitmaps are ever freed). Instead I get this kind of thing --29272-- new bitmap at 0x71EBBC10 (4096 4096 1024) --29272-- New segment for thread 1 with vc [ 1: 3215 ] --29272-- new bitmap at 0x71EC1CE8 (4096 4096 1024) --29272-- New segment for thread 1 with vc [ 1: 3216 ] --29272-- new bitmap at 0x71EC7DC0 (4096 4096 1024) --29272-- New segment for thread 1 with vc [ 1: 3217 ] --29272-- new bitmap at 0x71ECE2A8 (4096 4096 1024) --29272-- New segment for thread 1 with vc [ 1: 3218 ] --29272-- new bitmap at 0x71ED90B0 (4096 4096 1024) --29272-- New segment for thread 1 with vc [ 1: 3219 ] --29272-- new bitmap at 0x71EDF598 (4096 4096 1024) --29272-- New segment for thread 1 with vc [ 1: 3220 ] --29272-- new bitmap at 0x71EE4250 (4096 4096 1024) --29272-- New segment for thread 1 with vc [ 1: 3221 ] --29272-- new bitmap at 0x71F0A278 (4096 4096 1024) --29272-- New segment for thread 1 with vc [ 1: 3222 ] --29272-- new bitmap at 0x71F0EF30 (4096 4096 1024) --29272-- New segment for thread 1 with vc [ 1: 3223 ] --29272-- new bitmap at 0x71F15008 (4096 4096 1024) and every 16 new segments, thread_discard_ordered_segments tries to dump old segments, but I guess it fails: --29272-- Discarding ordered segments -- min vc is [ 1: 3214 ], max vc is [ 1: 3214 ] Not sure what's going on (I don't understand this vector clock stuff hardly at all). Is it right that thread_discard_ordered_segments always computes min and max vc's to be the same? J |