On Thu, Jan 3, 2013 at 3:12 PM, Francois Mikus <fmikus@acktomic.com> wrote:
Happy new year from this side of the ocean!
Hi :)
 

[...]

Graphite has built-in holt-winters support, trendlines, functions for comparing a time period versus a previous time period, etc.
Yes, the main code is at graphite/webapp/graphite/render/functions.py:def holtWintersAnalysis(series, bootstrap=None). But like I said to Hermann, HW is a nightmare to manage, even if we add to Graphite parameters to it, instead of the hard coded parameters, you will have to hack several times the parameters to get something near mine. The key is the dimension where you smoothing (in fact it's not a smoothing, it's an update of an average based on history). When you got "quick" changes during the weeks (or 2 months are Graphite is using it this cycle), you will have an O(N) error accumulation (each average compute will add it's error from the previous value). For values like disks it's not a problem, the variation is slow, so HW will work without too much problems. But for things as network consumption (and I think you know what I means here) what is the rule for 8h55 and 9h05? None. All users are connection (in fact nearer to 9h20 because of coffee pause...).

That's why I used my algorithm, to bypass this O(N) errors problem. One other thing, is the computation time. In Graphite, you can have HW, but it will look at 2 months value for just one value GET. Here we are doing something nearer to reporting than to classic transactional. Aggregate/pre-compute (tell the name you want here) is the key. O(1) versus O(N) computation -> I choose the first one :)

 

We should concentrate on integrating better calling of the graphing functions of graphite and a (new) storage mechanism that is only used for Shinken.
I don't understand this part. Do you means integrate this in carbon-agregator? (to bypass the 0(n) thing). I personally don't have time for this, but I can explain the algorithm to Graphite folks if they want it ;)
 

What you are doing is great, but in the longer term it may be better to focus on a backend that has more support and ecosystem and that can be used natively within Shinken. I think the ideas you have can be applied against Graphite to provide really really good stuff!!
Yes, should be better of course. But I don't have time to learn another tool code for hacking such thing. I need it now for a project, and here it work well. That's ok for me :D


Jean
 

Hermann, there is already an existing Shinken module to send data to Graphite (carbon), have you looked at it?

Cheers,

xkilian