Menu ▾ ▴

Support for rrd files created on mips

Help
Alexander Meyer
2012-08-24
2015-01-04

  • Alexander Meyer

    Alexander Meyer - 2012-08-24

    Hi,

    is there any way support for this will be added anytime soon?

    Regards,

    Alexander

     

  • Alexander Meyer

    Alexander Meyer - 2012-08-24

    I dug up a bit more info:

    The validate_rrd function finds 8.642135e+130 at offset 16, which makes it think the rrd is 64bit, which it isn't.
    According to http://oss.oetiker.ch/rrdtool-trac/wiki/PortableRrdFormat it should be simple to convert a mips rrd file to an x86 one using these two macros:

    #define endianflip(A) ((((uint64_t)(A) & 0xff00000000000000LL) >> 56) | \
                           (((uint64_t)(A) & 0x00ff000000000000LL) >> 40) | \
                           (((uint64_t)(A) & 0x0000ff0000000000LL) >> 24) | \
                           (((uint64_t)(A) & 0x000000ff00000000LL) >> 8)  | \
                           (((uint64_t)(A) & 0x00000000ff000000LL) << 8)  | \
                           (((uint64_t)(A) & 0x0000000000ff0000LL) << 24) | \
                           (((uint64_t)(A) & 0x000000000000ff00LL) << 40) | \
                           (((uint64_t)(A) & 0x00000000000000ffLL) << 56))

    #define mips2x86(A)   ((uint64_t)(A) == 0x7fffffffffffffffLL \
                                           ? 0x000000000000f87fLL \
                                           : endianflip(A))

    Unfortunately my coding skills are insufficient to implement it myself.

     

  • Igor Sfiligoi

    Igor Sfiligoi - 2012-08-24

    Yes, should not be too difficult.

    But I will need a RRD to test it against.
    Can you please provide one?

    Thanks,
      Igor

     

  • Alexander Meyer

    Alexander Meyer - 2012-08-26
     

  • Igor Sfiligoi

    Igor Sfiligoi - 2012-09-07

    This one does not seem to be created on MIPS.

    The magic double number in the header uses the same encoding  as AMD64: 

    $ hexdump mips.rrd |head
0000000 5252 0044 3030 3330 0000 0000 0000 0000
0000010 [b]252f c7c0 2b43 5b1f[/b] 0001 0000 000c 0000
$ hexdump example_amd64.rrd |head
0000000 5252 0044 3030 3330 0000 0000 0000 0000
0000010 [b]252f c7c0 2b43 5b1f[/b] 0008 0000 0000 0000

    Can you print out what 

    rrdtool info mips.rrd

    shows on the machine that created the file?

    That would help decoding the file.

    Thanks,
      Igor

     

  • Alexander Meyer

    Alexander Meyer - 2012-09-07

    sure: 

    filename = "mips.rrd"
rrd_version = "0003"
step = 60
last_update = 1344890986
ds[total].type = "GAUGE"
ds[total].minimal_heartbeat = 120
ds[total].min = 0.0000000000e+00
ds[total].max = 6.4000000000e+04
ds[total].last_ds = "30"
ds[total].value = NaN
ds[total].unknown_sec = 46
rra[0].cf = "AVERAGE"
rra[0].rows = 576
rra[0].cur_row = 328
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[1].cf = "MIN"
rra[1].rows = 576
rra[1].cur_row = 106
rra[1].pdp_per_row = 1
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = NaN
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[2].cf = "MAX"
rra[2].rows = 576
rra[2].cur_row = 70
rra[2].pdp_per_row = 1
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = NaN
rra[2].cdp_prep[0].unknown_datapoints = 0
rra[3].cf = "AVERAGE"
rra[3].rows = 432
rra[3].cur_row = 303
rra[3].pdp_per_row = 6
rra[3].xff = 5.0000000000e-01
rra[3].cdp_prep[0].value = 0.0000000000e+00
rra[3].cdp_prep[0].unknown_datapoints = 1
rra[4].cf = "MIN"
rra[4].rows = 432
rra[4].cur_row = 53
rra[4].pdp_per_row = 6
rra[4].xff = 5.0000000000e-01
rra[4].cdp_prep[0].value = inf
rra[4].cdp_prep[0].unknown_datapoints = 1
rra[5].cf = "MAX"
rra[5].rows = 432
rra[5].cur_row = 238
rra[5].pdp_per_row = 6
rra[5].xff = 5.0000000000e-01
rra[5].cdp_prep[0].value = -inf
rra[5].cdp_prep[0].unknown_datapoints = 1
rra[6].cf = "AVERAGE"
rra[6].rows = 540
rra[6].cur_row = 212
rra[6].pdp_per_row = 24
rra[6].xff = 5.0000000000e-01
rra[6].cdp_prep[0].value = 0.0000000000e+00
rra[6].cdp_prep[0].unknown_datapoints = 1
rra[7].cf = "MIN"
rra[7].rows = 540
rra[7].cur_row = 293
rra[7].pdp_per_row = 24
rra[7].xff = 5.0000000000e-01
rra[7].cdp_prep[0].value = inf
rra[7].cdp_prep[0].unknown_datapoints = 1
rra[8].cf = "MAX"
rra[8].rows = 540
rra[8].cur_row = 476
rra[8].pdp_per_row = 24
rra[8].xff = 5.0000000000e-01
rra[8].cdp_prep[0].value = -inf
rra[8].cdp_prep[0].unknown_datapoints = 1
rra[9].cf = "AVERAGE"
rra[9].rows = 450
rra[9].cur_row = 375
rra[9].pdp_per_row = 288
rra[9].xff = 5.0000000000e-01
rra[9].cdp_prep[0].value = 3.4820000000e+03
rra[9].cdp_prep[0].unknown_datapoints = 90
rra[10].cf = "MIN"
rra[10].rows = 450
rra[10].cur_row = 100
rra[10].pdp_per_row = 288
rra[10].xff = 5.0000000000e-01
rra[10].cdp_prep[0].value = 1.5000000000e+02
rra[10].cdp_prep[0].unknown_datapoints = 90
rra[11].cf = "MAX"
rra[11].rows = 450
rra[11].cur_row = 224
rra[11].pdp_per_row = 288
rra[11].xff = 5.0000000000e-01
rra[11].cdp_prep[0].value = 1.7400000000e+03
rra[11].cdp_prep[0].unknown_datapoints = 90
     

  • Igor Sfiligoi

    Igor Sfiligoi - 2012-09-08

    One more thing.

    Would you mind creating another RRD on the same machine, but one that has at least 3 DS's?
    I am trying to figure out the structure, and the DB name block does not make sense… seeing a few may help.

    Thanks,
      Igor

    PS: You are not using a custom rrdtool, build, right?
    (i.e., modified source code)

     

  • Igor Sfiligoi

    Igor Sfiligoi - 2012-09-08

    Cannot be "vanilla 1.2.30".

    The rrd version should be "0003", not "0001"! 

    rrd_version = "0001"

    I will still try to figure out what's going inside this file, but you should sort out things on your side as well!

    Igor

     

  • Alexander Meyer

    Alexander Meyer - 2012-09-08

    Whoops, my bad! I created this with an old version of the lib, which must have been 1.0, but also vanilla. I updated the files on my webserver, the links are the same as in the last post.

    Regards,

    Alexander

     

  • Igor Sfiligoi

    Igor Sfiligoi - 2012-09-08

    OK, I think I have nailed it.

    The code have been committed to CVS.

    The only essential change is in
    javascriptrrd/v0/src/lib/rrdFile.js

    It seems to work, but I would appreciate you trying it out, too and confirming it indeed works for you.

    Cheers,
      Igor

     

  • Alexander Meyer

    Alexander Meyer - 2012-09-08

    I only ran a few tests with the example scripts but so far it looks as if everything is working.

    Thanks a lot for your help, now I don't have to generate csv files with a shell script every minute any more ;)

    Cheers,

    Alexander

     

  • Alexander Meyer

    Alexander Meyer - 2012-11-30

    Hi again,

    I don't know if this is already known or even relevant, but I just wanted to let you kno the library also seems to work with RRDs generated on the raspberry pi.

    Cheers,

    Alexander

     

  • Rami

    Rami - 2013-09-19

    I am trying to use this library with RRD files created on the FritzBox 7390, which uses an Ikanos Fusiv Core (MIPS 24Kc V4.12) processor, and all I am getting are messages about the Magic number at 16. The device is running RRDtool v1.2.30.

    Attached is an example of an RRD file generated on that device, which is giving me errors with the JavascriptRRD libraries, can anything be done about it?

     
    cpu.rrd

  • Igor Sfiligoi

    Igor Sfiligoi - 2013-11-01

    Hi Rami.
    Sorry for the long absence.

    Looks like you re using a different median system here.

    I think I know how to fix it though...
    will let you know when it is done.

    Igor

     

  • Igor Sfiligoi

    Igor Sfiligoi - 2013-11-01

    OK, the head of CVS now has a fix that works with your example rrd file.

    Please test it, if you can.

    Will wait a few more days before cutting a new release.

     

  • Rami

    Rami - 2013-11-23

    Hello Igor,

    The new version is working perfectly with the RRD files I have.... THANKS A LOT....

     

  • AndyAndy

    AndyAndy - 2015-01-04

    Aloha, an interesting factoid on a related matter is below. Igor, I'm guessing you don't want to monkey with javascriptrrd for rrd_version 1 files for MIPS. I'll gladly take a patch if you wanted to chew on this (I can get you test files), but I've worked around.

    I am using openwrt Attitude Adjustment (waiting for Chaos Calmer to come out), and was having a ton of problems with rrd files produced with the collectd-mod-rrdtool and librrd1 packages (rrdtool 1.0.50). I run a D-link DIR-825 B1 (Atheros AR7161 rev 2, MIPS 24Kc V7.4).

    The observation was that my rrd plots with javascriptrrd had ping values like 1.234e+123, i.e. wickedly out of range. I confirmed it was just an endian/alignment/... issue by rrdtool dump/restoring the file. Then I found this thread :) And spent one heck of a long time digging into javascriptrrd's source code twiddling with switch_endian in various places. I couldn't get it to work at all.

    Long story short, I found that Rami's file was rrd_version 3 whereas mine were rrd_version 1. Being naughty, I gave my openwrt a good kick to move to a modern librrd:
    opkg remove --force-depends librrd1 rrdtool1
    opkg install librrd rrdtool
    cd /usr/lib
    ln -s librrd.so.2.0.15 librrd.so
    ln -s librrd.so.2.0.15 librrd.so.0
    ln -s librrd_th.so.2.0.15 librrd_th.so
    ln -s librrd_th.so.2.0.15 librrd_th.so.0
    And this fixed everything. Openwrt prefers the ancient librrd apparently because 1.2.x is "much larger", I presume in disk which is hard-to-find on most routers (I have extra storage).

     

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.