Re: [Nfdump-discuss] Adding a new filter primitive, org_id
netflow collecting and processing tools
Brought to you by:
phaag
From: Vegard V. <veg...@un...> - 2008-03-28 15:27:13
|
On Wed, 05 Mar 2008 08:46:25 +0100 Vegard Vesterheim <veg...@un...> wrote: > On Tue, 04 Mar 2008 13:08:52 +0100 Peter Haag <pet...@sw...> wrote: > >> If aggregation is required, a new field in the master record would >> be required holding the org label. An appropriate function mapping >> the IP address to an org label, while filling the master record ( >> ExpandRecord function ) could be a Berkely db or similar b-tree >> lookups with custom lookup function allowing, not only to lookup a >> single IP address but a IP range to org mapping. As input for this >> b-tree you may use the file as described above. The filter then can >> be extended to this new org label in the master record. > > Aggregation is important, yes. > >> Hope this helps > > Your input was very helpful, thanks. > > I will study the nfdump source to see where we need to create hooks > for our new filter primitive. I have now implemented a new primitive named 'ipinfo'. I am now able to invoke nfdump as: nfdump -P ip2org.cdb -R .. 'src ipinfo uio.no and dst ipinfo uninett.no' The file ip2org.cdb is a CDB file containing mappings ip-prefix -> 'some ip-prefix-related info' Implementing aggregation (and statistics) is somewhat trickier however, because the existing code is mostly based on info found in the netflow header, and therefore aggregation can be performed by doing bit-manipulations. I see a few possible solutions: 1) Simply use the existing algorithm for index_cache generation and use the existing infrastructure for index collisions to do aggregation. This is ugly, but easy. 2) Somehow mix the string value from my ipinfo-directive into the calculation of the index_cache. 3) Do a reverse lookup from my CDB-file giving me a list of IP-prefixes to do aggregation on, so instead of having only one IndexMask, I need to loop over several. Any further hints on how to proceed from here? - Vegard V - |