Re: [mh] generating rrd graps for things other than weather

[Lieven H. <li...@li...>]

Hey Dan,

Op 27-dec.-2013, om 18:23 heeft Dan Bemowski <dbe...@ph...> het volgende geschreven:

> So, just to I have this straight I am going to use an abbreviated example taken from the weather_rrd_update.pl code that comes with MH.  In this, each DS defines a sensor value to show on the graph (correct?)

Each DS does define a sensor value.

> , then each RRA defines the graph resolutions that are possible for each MIN, AVERAGE and MAX aggregate storing data for a defined period (1 day, 2 days) at each resolution (2 min, 5 min)?  The primary resolution for the first RRA is at 6 hours using an AVERAGE aggregate of values every minute, correct?  I hope I said that right?

The primary resolution depends on $RRD_STEP in the code. For the MisterHouse weather code this value is set to 60 seconds if I’m correct. Basically if you say 

RRA:AVERAGE:0.5:1:801 

then you tell the tool to store a value in the database every $RRD_STEP seconds, and do this for the last 801 $RRD_STEP seconds.

Why do you need ‘AVERAGE’ then? Well, you can push values to the RRD at a higher speed than defined by $RRD_STEP and then RRD will store only the average of those values every $RRD_STEP seconds.

Now, back to the code. I think the comment that states ‘for 6 hours’ is not correct. Since the $RRD_STEP is 60 seconds in weather_rrd_update, 801 times step_size is 13 hours instead of 6 hours. This might be the source of the confusion you are experiencing. It is probably something nobody has noticed up to now because 13 hours > 6 hours and the graphs show to be correct. The only disadvantage is that the database is bigger than it is absolutely required.

> 
> One last question I had in my post on this.  I am trying to understand the RRA's 0.5 value which is said to be "xff" by mr Oetiker in the beginners guide page that I listed.  What does the 0.5 represent or mean?  When I create my RRDs for graphing I can certainly use the 0.5 value that I see everyone else primarily using, but I like to know what it means.

See http://oss.oetiker.ch/rrdtool/doc/rrdcreate.en.html

xff The xfiles factor defines what part of a consolidation interval may be made up from*UNKNOWN* data while the consolidated value is still regarded as known. It is given as the ratio of allowed *UNKNOWN* PDPs to the number of PDPs in the interval. Thus, it ranges from 0 to 1 (exclusive).

If you fail to feed data into the database for a certain amount of time, these points in time will be marked ‘unknown’. If you start to consolidate the rate (e.g. average data points) then the off factor determines from what point on the result of the consolidation function will also be unknown. If you set it to .5 then half of the values are allowed to be unknown before the consolidated results is also unknown.

Kind regards, 
 Lieven.

> 
> On Fri, 2013-12-27 at 14:48 +0100, Lieven Hollevoet wrote:
>> 
>> Hey Dan,
>> 
>> Op 25-dec.-2013, om 08:40 heeft dbemowsk <dbe...@ph...> het volgende geschreven:
>> 
>> > My next question in all of this has to do with data history.  I understand
>> > that the concept of RRD revolves around the whole round robin concept where
>> > the data works on a fixed length timeline where data overwrites data from
>> > the beginning once the saved points reach the end of the data set.  My
>> > question is, is there a way to archive data?  What I mean is, if you have an
>> > RRD that collects data for a 24 hour period, and when the next day starts
>> > effectively overwriting the first days data, is there a way to look back at
>> > a graph of data from a few days ago?  I see that the weather_rrd_update.pl
>> > file creates a CSV file.  Is this what the CSV is used for?  If so, I am
>> > guessing that it just keeps appending data to the end of the CSV, correct?
>> 
>> Nope, the idea of RRD is that you can use it stand alone. You define data sources (DS) that keep track of values. Then for every data source you create RRAs (round robin archives) with variable ‘timespan' per datapoint. E.g. suppose that you store a new temperature every 5 minutes, then your primary ‘resolution’ for the first RRA van be 5 minutes.
>> Then, for longer archiving you can define RRAs that keep track of your data over longer timespans. You then need to define the consolidation function for that datasource. Suppose you define secondary RRAs with a resolution of 1 hour per datapoint. The RRD framework can then take the values from the first high-resolution RRA, apply a consolidation function on those datapoints, and then calculate the value for the lower resolution dataset. The consolidation function can be e.g. average, minimum or maximum. This way you can keep track of values over longer time (e.g. a year) in 1 hour intervals. If you want to go even further you can then store values on a daily basis for 10’s of years. 
>> 
>> > 
>> > My last question has to do with Data Sources (DS).  In the example, they
>> > define one DS.  In looking at the weather_rrd_update.pl file, they define a
>> > number of them.  Does each DS represent a different line (color) on the
>> > graph?
>> 
>> Nope, a DS is a dataset. You can see it as a representation of a sensor that you read out and that you store the values in the database. To create the graphs you use the values from the RRAs or you calculate values based on the data in those RRAs.
>> 
>> I hope this clarifies it a bit further.
>> 
>> Kind regards,
>>  Lieven.
>> 
>> 
>