[Perfparse-devel] RE: Use of RRD

[Cook, G. <GW...@ma...>]

Yves Mettier wrote:
>> Garry, Yves,
>>=20
>> I want to talk about RRD, however I have lost the original thread.
>>=20
>> Let me explain my worries about using RRD.  Please correct me where
>> I am going wrong.
>=20
> My comments will be my explanations.
> Garry has his own opinion (I agree with him for most of what he said).
>=20
>>=20
>> Round Robin Database gives a method of storing a fixed period of data
>> which is stored locally and may be graphed by many available open
>> source programs.  This is a fast self-purging method liked by many
>> people.=20
>>=20
>> But what are we proposing?
>>=20
>> i) Using just RRD.
>>=20
>> ii) Using RRD for storage of Binary data, and a DBMS for everything
>> else.=20
>>=20
>> iii) Using PP as is, but also store data in RRD.
>=20
> iv) Using RRD for storage of Binary data, and a DBMS for *all*
>=20
> Well, propose everything !
> Some may only use the perfparse binary (not the CGI...) to
> create RRD files and use them
> with other tools. Some may need database storage. Some may
> need both...
>=20
> I my mind, there is only a --rrd-dump=3Drrdfile option to add
> in perfparse.c and to support.
> Those who want to graph the data from the rrdfile will use
> their own tools, and maybe
> some may send us a contribution ?

At this point I'm not sure what to propose. I was originally proposing
that we give users the ability to use ONLY RRD, if they wish. But I was
afraid that this might no longer be possible, due to some of the
features that have already been implemented.=20
Is this definitely the case? What is needed that cannot be stored in
RRD? The groups I guess, and raw data, although I don't understand what
part of that can't be stored, the strings I suppose. Yeah, I'm starting
to see why we cannot use only RRD. Perhaps you guys can just throw in
the option that Yves mentioned above and we can move on...


>> I have seen some comments about using (i), just RRD.
>>=20
>> If we do this, the product we will have effectively is not PerfParse.
>> - There will be no storage of raw data.
>=20
> For now, this is not a problem for me: I use only binary data.
>=20
>> - The extra information we plan, like a range of WARN and CRITICAL
>> values will not be supported.
>=20
> This is a choice I can do for here. My need is that the extra
> informations are parsed. I don't mind they are not stored.
> And there is a hack we can do: store them as another value.
> The rrdfile will grow
> faster, with 5 values instead of 1 (val, warnmin, warnmax,
> critmin, critmax)

I'm not sure that you guys are correct here. The range values can all be
stored, and it is not really a hack. I think you can store many values
in an RRD, if that is your choice.

=20
>> - The product will not be client/server, and therefore the whole of
>> PP must be using on a single machine.
>=20
> I need this. So I need the database (except if we make
> another server/client system, but
> this is not planned, and we have enough of them for now)

I use all of my tools on a single machine anyway. In fact I prefer it
that way. Should I ever go into business building network monitoring
appliance servers, I wouldn't want to have to tell people that they must
buy at least two servers. I understand that this just gives me more
flexibility, and it is optional to have a second server, but if we were
to pull a lot of the data out of MySQL and put it into RRD, I don't
really think there would be a need for a second server, as I think it
would perform much better.


>> Effectivelly, the entire PP CGI is redendent and we should probably
>> advise users to use one of the existing mature RRD viewers.
>  This will
>> mean that PP is reduced to just the 'perfparse' program, which it's
>> self is reduced to just storing binary values in an RRD database.
>>=20
>> I don't think we want to do this. :)
>=20
> I think that some want it. You would not have so many mails
> about it otherwise.

Yes. If you want users to use PP, RRD support is necessary. Those who
want/need it are going to go over to Nagiosgraph otherwise.


>> With option (ii), using RRD for the binary data.  This also presents
>> some problems:=20
>>=20
>> - Extra data, like range of WARN and CRIT cannot be supported in
>> full. Although two lines for each can be stored to indicate range.=20
>> Messy!=20

You may be right, I'm not sure. It's been a long time since I looked at
RRDTool in detail, although I use it every day for my MRTG graphs.
However, I think you'd be surprised at what you can do with RRDTool.

=20
> The users make the choice.
>=20
>> - Half of PP is client/server, half is local.
>=20
> This is good for data storage. Few data central, a lot of
> data on each monitored machine.
> If the CGI could ask each monitored machine for the data, it
> would be better for data
> storage : save the data with your usual backup procedure. It
> would also be better for
> network traffic: only the needed values go on the network.
> Sadly, it is more complex to implement this because of the
> client-server issue.=20
>=20
>> - Referential Integrity is hard to maintain using two different
>> database types.=20
>> - Lots of re-coding to get the chart to read RRD data.
>=20
> About this: probably like you, I fully disagree about the CGI
> reading RRD data.
> I agree with feeding RRD files, but exploitation of RRD files
> has to be done with other
> tools (that can be part of the perfparse project, one day, why not ?)
>=20
>> Again, I wonder whether this is worth it, although there might be an
>> argument.=20
>>=20
>>=20
>> My feeling is that we should provide an option the user can select
>> when compiling:=20
>>=20
>> 1. Store data as RRD for their own use, or not.
>> 2. Store data as PP format, or not.

3. Store data as PP format, and be able to dump to RRD at regular
intervals, perhaps as data is purged from MySQL.

> Yes :)
> My own reflexion results in the same conclusion :)
>=20
> Yves
>>=20
>> In the second case, we are effectively providing a method to import
>> data into RRD, and the users chooses not to use any of the other
>> features we provide.=20
>>=20
>> As always, I look forward to your comments.
>>=20
>> Ben



Garry W. Cook, CCNA
Network Infrastructure Manager
MACTEC, Inc. - http://www.mactec.com/
303.308.6228 (Office) - 720.220.1862 (Mobile)