Re: [Ganglia-developers] gmond python module interface

[Brad N. <BNI...@no...>]

>>> On 1/28/2009 at 7:59 AM, in message <200...@im...>,
Kostas Georgiou <k.g...@im...> wrote:
> On Wed, Jan 28, 2009 at 06:09:48AM -0800, Gilad Raphaelli wrote:
> 
>> 
>> I think this is a well thought out email and I'm a little surprised at
>> the lack of response to it.  Is it because no one is actually using
>> the gmond python module interface and hasn't had to make these types
>> of decisions?  I don't have a single gmetric script/module that only
>> collects one metric and have used both the mutli-threaded collector
>> approach and the single-thread hopeful/naive approach (having written
>> the mysql metric module you mention).  I agree that the multi-threaded
>> design seems less prone to problems but in practice I haven't had any
>> problems either way.  That being said, I will be trying some of your
>> suggestions or moving to threads for that mysql module when time
>> permits.
> 
> You could also modify gmond to run each plugin in a new thread, no sure
> if it is easy/possible though since I haven't looked at that part of the
> code yet.
> 
>> My frustration with the gmond python module interface is that it's not
>> actually a complete replacement for gmetric scripts as I use them.
>> Needing to know all of the metrics that a module will report before
>> runtime makes for a lot of upfront work creating .pyconf files and
>> doesn't allow for adding new metrics without restarting gmond.  Being
>> able to deploy one gmetric script that conditionally reports gmetrics
>> based on what's running/hardware installed/etc on a box is a big
>> advantage for a gmetric script over conditionally generating pyconf
>> files and then still having to conditionally collect metrics in the
>> actual gmond module.  I expect most users just stick with the gmetric
>> script in this case and handle scheduling themselves?
> 
> I agree here, my suggestion in the "Wildcard Configuration" was to add a
> metric_autoconf function in the plugin and get gmond to use that to get
> the collection_group from there. In any case for the modules that I
> develop I call this function from main so I can do python module.py -t >
> module.pyconf with a similar effect.
> 
> This doesn't solve all problems though, I would like to be able to just
> start mysql in a host or add a new disk for example and have metrics for
> them without touching the configuration at all. Maybe extending the
> configuration so in a .pyconf you can write something like
> collection_group {
>   autoconf = 300
> }
> to have gmond interogate the plugin every 5 mins to get a new collection
> group will be enough, I need to think about it a bit more...
> 

The issue is more about the internal hash tables and arrays that are initialized at startup directly from the information that is found in the .conf files.  In order to make gmond recognize metrics without them being explicitly configured in the .conf file, gmond needs to be changed so that the internal tables can be allocated on the fly.  It can certainly be done, it is just a matter of taking the time to figure it all out and make the changes compatible with the current functionality.


Brad