Hello! How about using MySQL features?
Recently i've placed my perl-based netflow collector here
This program collects and stores data in MySQL database. Now it can work with 5 and 9 versions of NetFlow protocol. Of course this is quite raw and new project, but i can continue development in your direction.
To view some data from collector, you can use following example of SQL script.
SELECT
FROM_UNIXTIME(`datetime`-CEIL((`sysuptime`-`first`)/1000)) AS first,
1000 - MOD(`sysuptime`-`first`, 1000) AS firstms,
FROM_UNIXTIME(`datetime`-CEIL((`sysuptime`-`last`)/1000)) AS last,
1000 - MOD(`sysuptime`-`last`, 1000) AS lastms,
INET_NTOA(`srcaddr`) as `srcaddr`,
INET_NTOA(`dstaddr`) as `dstaddr`,
INET_NTOA(`nexthop`) as `nexthop`,
`input`,
`output`,
`dpkts`,
`doctets`,
`srcport`,
`dstport`,
`tcp_flags`,
`prot`,
`tos`,
`src_as`,
`dst_as`,
`src_mask`,
`dst_mask`
FROM `v5` WHERE `device_id`=1
LIMIT 0, 30
This script shows data v5 version. If you have a good skills in SQL, you can any data you need.
If you interested, i can tell you, how to get another data, include v9 protocol.
Last edit: dream-hunter 2013-05-11
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I love working with SQL and this looks valuable to those who might want to query the data via SQL. I chose to use RRDtool 'databases' to store the data because their method of consecutive averaging (e.g., Last 24 Hours, Last 7 Days, etc.) is familiar to network admins (i.e., MRTG, RRDtool) and it dramatically reduces the amount of stored data required to track flows over the long term.
Thanks for your contribution!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
...i've talked about storing the raw data. by my opinion it is much useful to use SQL database instead of files. Simplier access, sortings etc.
Of course RRDTools are better for storing processed data.
Last edit: dream-hunter 2013-05-15
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello! How about using MySQL features?
Recently i've placed my perl-based netflow collector here
This program collects and stores data in MySQL database. Now it can work with 5 and 9 versions of NetFlow protocol. Of course this is quite raw and new project, but i can continue development in your direction.
To view some data from collector, you can use following example of SQL script.
This script shows data v5 version. If you have a good skills in SQL, you can any data you need.
If you interested, i can tell you, how to get another data, include v9 protocol.
Last edit: dream-hunter 2013-05-11
I love working with SQL and this looks valuable to those who might want to query the data via SQL. I chose to use RRDtool 'databases' to store the data because their method of consecutive averaging (e.g., Last 24 Hours, Last 7 Days, etc.) is familiar to network admins (i.e., MRTG, RRDtool) and it dramatically reduces the amount of stored data required to track flows over the long term.
Thanks for your contribution!
...i've talked about storing the raw data. by my opinion it is much useful to use SQL database instead of files. Simplier access, sortings etc.
Of course RRDTools are better for storing processed data.
Last edit: dream-hunter 2013-05-15