[Shinken-devel] my thought about the graphite broker module

[David G. <dav...@gm...>]

Graphite is a great piece of software and resolve at least performance and
distributed metrics needs. The graphite broker module is a nice starting
point but i think we could go a little bit further.

First the way we send data to carbon is based on time (hook_tick). This way
it is not easy to evaluate the amount of data that will trigger flushing to
carbon (if we use pickle). So if the time is not correctly set (say every 5
minutes), we could reach a memory full when dealing with really high amount
of checks. I think it should be better to define the trigger on an amount
of data.

If we are not using carbon relay for high availability the queue is going
to grow and grow and grow again. There is no security to prevent a memory
full. see below :

            try:
                self.con.sendall(packet)
            except IOError, err:
                logger.error ("Failed sending to the Graphite Carbon
instance network socket! IOError :%s" % str(err))
                return

            # Flush the buffer after a successful send to Graphite
            self.buffer = []

may be flushing the queue to disk (avoid loosing metrics) when the queue is
more than a specific length. and process it again when carbon is available
again.

Last (but not least) the way we define path is not really good as we cannot
use the schema based storage retention. The way it is defined we cannot
define different retention for different metrics. So it is really annoying
to use only one schemas for every metrics. What about defining a macro
based discriminant so we can use it to build path and define specific
schemas based on this. For exemple, i want to have different retentions for
aix and linux load average (yeah i know bad example). Adding a macro like
GRAPHITE_PREFIX (aix and linux) and using it to build path would result in
something  like aix.load.1min and linux.load.1min so i can define two
different storage schemas (and so two differents retentions).

may be i'm wrong since i started to learn graphite yeasterday ;-)

cheers

david guenault