From: Alex P. <pe...@in...> - 2007-01-15 21:54:15
|
Dmitry Yemanov: > That said, personally I think that we could live without pools, although it would require a few workarounds to be implemented. > > However, Alex has tested various memory handling models and AFAIK he discovered that in the multi-threaded environment pools allow to noticeably decrease lock contention in the memory manager. As you know, most of the allocations are done from the request and transaction pools which are bound to particular threads. So, when I was talking about the pools infrastructure in my proposal, I followed Alex's conclusions. I suppose he should explain his findings here. The problem off pools is really one of the most important when merging fb3. On the one hand, they increase complexity of programming. On the other - it will be serious change in our internal structure (remember - vulcan still works with pools and uses delete by pool). You see, all our branches still do work with pools. Even this single argument, taken together with the fact that fb3 is proposed to be _merge_ version with minimum new features added, may be enough to keep pools in it - I can imagine number of memory leaks we will have to fix in other case! But additionally to this fact we must take into account possible conflicts in memory manager. To estimate possibility of such conflicts I've measured time, which multi-threaded (but always with single runnable thread) fbserver spends inside memory manager (between locking of it's mutex and unlocking it). Certainly, this differed much depending upon tasks, and varied between 5 (long select with GROUP BY from very big table) and 15 (restore using GBAK of database with big metadata but not too much of user data in it) percents. As a good approximation we may use 10% for firebird 1.5/2 memory manager. Vulcan memory manager is twice faster, therefore for it this will be about 5%. Assuming random distribution of memory manager invocations (for async MT system this seems to be true), we get following possibilities of conflict when asking for memory: N Cpu %% 1 0 2 5 4 14 8 30 16 54 Certainly, starting with approximately 4 processors number of conflicts will not grow so fast - each conflict leads to additional system call, which leads to incompatibly big delay and greatly decreases overall performance. You see - with existing memory manager it's hard to talk about more than 4CPU systems. Do we need such problems in fb3? There are ways to solve a problem. They are: - decreasing number of calls to memory manager (presence of internal compiled requests cache, for example, should help - but I'm afraid we will not have such a cache in fb3.0) and - building conflictless memory manager. And here take into account, that use of many pools is de facto memory manager with low conflict possibility. As long as single thread works with dedicated pool conflicts are impossible. Therefore - why drop multiple pools from fb3? The main reason is that they "make coding much more complex". I may agree with this argument. But please, show me - what means "much" in this case? How many hard-to-find bugs happened due to multiple pools? What else serious problems did you meet? Need to type a bit more words in constructors (like "someObject(getPool(), par, ...)" instead of "someObject(par, ...)" or even "someObject(getPool())" instead of use of default constructor) does not seem to be enough serious reason to kill multiple pools. And use of "FB_NEW(pool)" instead of "new" too. |