From: Devon H. O'D. <dev...@gm...> - 2015-02-03 21:30:24
|
2014-10-30 18:05 GMT-07:00 J.T. Conklin <jt...@ac...>: > Daniel Pocock <da...@po...> writes: > > > - 3.7 adds a new dependency, Concurrency Kit > > Given the problems you describe with Concurrency Kit, is it simply > premature for ganglia to depend on it? From what I can tell, its > use is isolated to lib/hash.[ch], and at first glance looks like it > an alternate reader/writer lock implementation could be selected at > runtime. Are the performance benefits of CK rwlocks really worth the > added complexity, especially as ganglia continues to depend on APR, > which also provides a rwlock implementation? > Originally, I added CK in for the spinlocks because it seemed they would be held for very little time. However, it seems like a number of people are running into scenarios where gmetad usage spikes. I don't know what APR's rwlock has over pthreads, but it might make sense to switch back to a lock type that goes to sleep, given that there's significant contention over these locks when iterating the hash table. (Actually, I just looked at APR's rwlock implementation and it seems to be just a thin wrapper around pthreads on any OS I care about. So I'd be fine with that replacement.) After giving it a fair amount of thought over the last few months, I think our use case would not suffer greatly. Most of our performance improvements came with the ability to multiplex requests to rrdcached over TCP instead over a unix domain socket. That all said, there was a reason that I introduced a dependency to CK. The current hash table implementation avoids callers needing to think about thread safety by doing a huge amount of allocations and copies, which really hurts performance (especially on iterations, which have a huge impact with our number of metrics). gmetad is a great use case for Concurrency Kit in general, and I had hoped that putting it in would make it easier to do more performance work on it in the future. (Sadly, I just haven't had the time.) I'll catch up with Vlad about testing our metrics with APR rwlocks and see if that's going to work for us. (I think it will.) However, I would also like to solve whatever CK packaging problems are outstanding, as I would like to use it in future improvements. --dho --jtc > > -- > J.T. Conklin > > > ------------------------------------------------------------------------------ > _______________________________________________ > Ganglia-developers mailing list > Gan...@li... > https://lists.sourceforge.net/lists/listinfo/ganglia-developers > |