SNMP interface index conversion functionality patch.
NetFlow iptables module for Linux kernel
Brought to you by:
aabc
Menu
▾
▴
#58 SNMP interface index conversion functionality patch.
I've made a patch that implements interface name to SNMP-index conversion functionality ( like in fprobe-ulog ).
Each rule consists of interface base name and SNMP-index base separated by colon ( e.g. eth:100,ppp:200 ).
Final SNMP-index is sum of corresponding SNMP-index base and interface number.
For example, SNMP-index of interface eth1 will be converted to 101, index of ppp15 will be converted to 215.
If interface name did not fit to any of conversion rules then SNMP-index will be taken from kernel.
Variable, which contains conversion rules, is called 'snmp_r'.
Values can be written both via sysctl entry net.netflow.snmp_r
( for example `echo eth:100,ppp:200,vlan:300 > /proc/sys/net/netflow/snmp_r` )
or as an modprobe commandline argument
( for example `modprobe ipt_NEFLOW snmp_r=eth:100,ppp:200,vlan:300,br:400` ).
By default, this variable is empty.
If variable is empty, conversion functionality is disabled.
To compile module with snmp conversion functionality,
before building module you must run configure script with option --extra=SNMP_A
( for example ./configure --extra=SNMP_A ).
This functionality was very useful for me.
I hope, it will be useful for other people too.
Oh, sorry didn't noticed this mail (got into spam box because of last spike of spam from SF). Thanks!
Can you explain more what is general purpose of this feature? Why indexes need to be converted? And how it's related to SNMP?
1. Index conversion is useful for futher processing of netflow statistics - router can have, for example, ppp or sit interfaces. Every time when one and the same interface of such type become up, it has the same name, but different index. And if such interface has dynamic ip-address, it is useful twice, i think.
2. Interface index, generated when dev_new_index() is called, is placed into net_device.ifindex field. Net-snmp takes interface index from this field.
While converting ifindex my patch will not change it - so, after conversion in netfow data it will be not 'pure' ifindex - it will be a some kind of inetrface number. If conversion fails, ifindex in netflow data is 'pure' again.
I hope, i answered your questions.
Unfortunately you didn't answer my third question. But thanks anyway.
If you sill want to clarify - why we need to convert index "for SNMP" (or "by SNMP") if SNMP take index from ifindiex which is not converted. How all this converting is related to SNMP at all?
Maybe it's my formulation mistake.
Of course, ifindex after conversion is a some kindof interface number, but not a SNMP index, and it can be 'visible' only in netflow data.
Previous comment is posted by me.
Forgot to log in (
So, basically it is 1) not "SNMP index conversion" but "ifindex conversion", and 2) not for some SNMP (standard, etc) reasons but just for fun of it, right?
Right.
Previous comment is posted by me to.
Forgot to log in again (
Out of curiosity, did this patch ever get put in? This functionality is very handy on hosts with multiple interfaces to identify which interface a flow used.
Kevin, no, this patch did not get in, and it also violates RFC.
http://tools.ietf.org/html/rfc5102
5.1.3. ingressInterface
Description:
The index of the IP interface where packets of this Flow are being
received. The value matches the value of managed object 'ifIndex'
as defined in RFC 2863. Note that ifIndex values are not assigned
statically to an interface and that the interfaces may be
renumbered every time the device's management system is
re-initialized, as specified in RFC 2863.
I believe that it does not really makes sense to be so strict on this, as in case of Linux, where interface index may change even while processing single flow (ppp+ interface), there is no way for netflow collector to determine which physical interface was used when recording specific flow (I personally really loathe this behavior).
Even worse, due to parallelism while initializing network interfaces, the order may different after each reboot, which may (and usually will) confuse collectors/analyzers even more, especially when they are not "augmented" by SNMP (which is seldom the case).
Cisco has ifIndex persistence and it is proven to be quite useful: http://www.cisco.com/c/en/us/support/docs/ip/simple-network-management-protocol-snmp/28420-ifIndex-Persistence.html
Please also note that RFC 2836 does not prohibit persistence and static assignment (https://tools.ietf.org/html/rfc2863#section-3.1.5) - it only mandates that different interface will never get previously allocated number, but allows same interface to get once allocated number.
Since Linux does not support "static" assignment "out of the box", it makes sense to allow this on different level.
And finally, since this feature (if ever implemented) is only a feature, those who want to stick to unpredictable interface numbering are free not to use it :)
Thanks for your comment, Alexander. But. Netflow just reports state of the device. If device is smart, like Cisco with ifIndex persistence, then interface number is more meaningful than in case of dumb device, like Linux. But that is responsibility of device to be dumb or smart in assigning interface numbers, not of logging subsystem. Plus. From my glance of IPFIX it seems to be possible to report additional information about interfaces, like interface name or description. Check http://www.iana.org/assignments/ipfix/ipfix.xhtml look at interfaceName, interfaceDescription. Of course collectors should support these fields too.
Reporting interface name or description is quite expensive operation - both bandwidth-wise and CPU-wise, especially if you have thousands of flows reported every second.
It looks that making Linux kernel "smart" in this regard is not an easy task, but implementing some "workaround" in logging module is relatively easy (which is mostly done anyway).
On the other hand, what do you lose if this mapping feature will be implemented? After all, that is why the software was invented - to make "dumb" devices "smart" :)
Some other projects offer mappings of this kind (softflowd is one of them), why not ipt_netflow? I am quite sure that many people will really appreciate this.
It is not expensive, since these reports supposed to be occasional. It also could be even separate type of packets (options templates, probably, I'm not sure w/o examples and rfc is unclear in this regard). To report such statistics/info it doesn't even need to be in the same kernel module, it could be coded in separate perl/etc script, who report stat/info with different Observation Domain Id. NetFlow/IPFIX is designed with possibility of many exporters at once.
I think if there is more standardized way we should push it instead of hack.
For example. How would you identify interfaces uniquely? eth0 eth1 could be created in different order (not sure 100%, but I think I saw this), tunnel interfaces could have sequentially change id (like tun1 tun2) on every reload. How to number bridges? What should be the numbers if I have interfaces like this: eth6 eth7 eth0 eth1 eth2 eth3 bond0 bond1 bond0.500 bond1.803 bond1.804 bond1.805 ? If I add bridges ifindex to 400 (as in patch example) these numbers are still non-persistent in the range.
Many people will not appreciate if 'solution' does not really solve the problem, unreliable, prone to mistakes, or does not work fast.
I think, to switch to NFv9/IPFIX with info packets is best and standardized way. But it maybe be needed to push the idea to coders of Netflow collectors.
Check http://www.plixer.com/blog/netflow/interface-names-via-netflow/
Note also, problem is when SNMP and Netflow ifindex are different (like in Huawei or like in proposed patch). But by default they should match.
Last edit: ABC 2014-04-16
Interface names in Linux may be persistent (and usually are, if people care about which interface is which), most modern distributions supports this - more on this here http://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
Bridges are numbered as system administrator assigns them, actually, almost any interface is named explicitly and (mostly) persists. The only exceptions are ppp+ interfaces (there is no reliable and standard method to (re)name them). Tunnels could be named explicitly too, of course.
I am not saying that solution offered in the patch is the best, but the idea is right - there should be a way to map interface names to numbers - same name to same number, which algorithm should be used is another question.
Periodical reporting of changing numbers adds unnecessary complexity to processing - as we have to match which interface had specific number at specific point in time, and while this is doable - this is more complex (and requires more processing) comparing to simple 1:1 name:number mapping, especially when interface names are stable enough and have a system, and it doesn't fit in existing accounting/processing frameworks, as many of them are not expecting such a mess in numbering.
Example from my case - there are couple systems, in HA-mode, each has few physical interfaces and many "virtual" (VLANs, ppp, tun etc). Sometimes, when one or another system takes over, interface is created (and deleted on secondary), thus VLANs like eth0.100, eth1.200 (which have persistent numbers) are assigned different ifIndex every time. There are times when interfaces needs to be shutdown and recreated, thus renumbering takes place.
A bunch of ppp interfaces (VPNs & co.) adds to this mess - you never know what specific ifIndex means, so now I've eth3.1 with ifIndex 778 (it was below 20 initially), additional problem is that those indexes are different on each system (for same physical links).
Since it is important (from accounting point of view) to know on which interface specific flow was collected, the only reliable way to do this is to make ifIndex persistent (for example eth1.200 is always 1200, eth0.100 is always 100 etc).
Of course, "in a perfect world", ifIndex in SNMP and in Netflow should match, but in practice (called "Linux based routers") this is not the case, unfortunately, and proposed feature is the easy (and quite solid, I'd say) fix for this.
Sure, I can make an application which will store somewhere in database (or even in file) timestamp of host:ifName:ifIndex change, then modify accounting software to take this information into account - but this makes things more complicated and not as universal as simple persistence - yes, it will not fit 100% of all cases, but 90% - most likely will.
Your link mentions: "Just about all NetFlow Analyzers get the interfaces names of the routers and switches using SNMP." - yeah, of course... Especially when analyzer is run once per day and numbers have changed many times... people who wrote analyzers didn't collect information from Linux routers, probably.
Using "option template" could (theoretically) help - however, it will not cover all collectors and analyzers, and cases when ifIndex changes (but interface is not - like reconnected ppp client or reconfigured VLAN) while collecting single flow is not covered at all.
You say:
and in same time:
Which is contradictory in my opinion.
Idea is good but we can't implement it in a good way? Propose algorithm too.
Additional problem to avoid:
What if, for example, ppp interface 200+ifindex after some time collide with eth1.200 ifindex 1200?
Unnecessary? If this is 'unnecessary', maybe this whole idea is unnecessary too. Many people now live with ifindex like this, and in the world of linux+snmp ifIndex is like this too. Probably for them such mapping was not necessary.
Necessary or not, this reporting in option template is more standard. And equality of SNMP ifIndex to Netflow interface is RFC standard (they are even called INPUT_SNMP/OUTPUT_SNMP in rfc3954).
And all other solutions I have seen is mess and hack.
Your argumentation currently is to show good goal which is not achievable properly, and on account of achieving it (fantasy) implement hackish mess which does not achieve it (reality). Many people will not appreciate solution which doesn't actually and reliably reach the good goal and only add confusion and mistakes. So, propose also algorithm that everybody (who need this feature) will agree to be good, understandable, and reliable.
No contradiction here - "not the best" was related to the proposed implementation - it is not really effective, though it is OK for not very loaded routers.
ppp numbering is not continuous (numbers are reused in names, unlike kernel-assigned numbers) - so you have to have at least 1000 simultaneously active ppp interfaces to collide in this case, but even then, if you expect so many, what prevents you from defining ppp (and other dynamic) base above all "static" interfaces?
To be honest, I don't see any confusion here. Most systems have limited (or even very limited) set of "static" interfaces (ethernet & co), so they easily map to small set of numbers. It may be dynamic (for numbered interfaces, like eth0, eth1 or eth2.100, eth2.110) or static (fixed exact-match mapping like br0:100).
Interfaces may be mapped not only based on names but on (say) MAC address, type or physical location (PCI etc), so we have 1-64 for physical ethernet (or like) ports, then base index for VLANs on every physical interface, and finally, base for each dynamically allocated interface like ppp, something like this:
lo:0
eth+:1
eth0.vlan:1000
eth1.vlan:5000
eth3.vlan:9000
tun:20000
ipsec:30000
ppp:50000
V9 allows more than 16 bits for interface indexes so we have more than enough intervals for everything, but even 16 bits (V5) is sufficient for most (if not all) applications.
Really? Match my argumentation above to your statement - is it not "properly enough"? Is it really "hackish mess"? Or may be you could elaborate - why it is "hackish mess" and where did I made mistake so it does not "reliably reach the good goal and only add confusion and mistakes"?
Well, many people live without ipt_netflow - but you wrote it, nevertheless? :) Many people live with "dumb" hardware and inflexible software - but it doesn't mean that they like it, sometimes they simply have no choice and no experience or skills to improve or change this.
Of course, many people live with those unpredictable and confusing ifIndex numbers in net-snmp (the only, and unfortunately, not bug-free implementation of SNMP), but it doesn't mean that this is OK for them - google for "linux snmp persistent ifIndex" and you will see that many are looking for solution - for instance: http://sourceforge.net/p/net-snmp/mailman/net-snmp-coders/thread/20051123222939.GA12173@rigacci.org/
Even here, in this thread, at least three persons (including me) are interested in persistence.
Sorry, but kernel interface numbering has nothing to do with SNMP - this is Linux-internal and (to keep things easy) net-snmp exposes kernel provided indexes - but this is not set in stone, different SNMP implementation may use name-based mapping (this was also discussed many times).
It could even happen that someone will use different SNMP daemon for Linux (not net-snmp) which doesn't use ifindex from kernel - thus ipt_netflow will report totally incorrect indexes - what would you do then?
Not your argumentation is 'hackish mess' but proposed solution. I didn't say your argumentation is 'hackish mess' but 'implementation' is mess. It is mess and hack, obviously, because it is non-standard, break standard, error prone, hard to understand and thus not solves problem. Example I given: ifIndex+offset where offsets are many could collide. Second example I given: in formula ifIndex+offset, offset is user defined, ok, but ifindex increment is still essentially 'random', thus it don't make things really ordered, it orders something and leave rest unordered.
Last edit: ABC 2014-04-17
It seems that you have completely mistunderstood something here.
ifIndex increments (that I am talking about) are not random at all, they are constant based on interface name. For instance, if our rules are:
eth:1
br0:100
ppp:10000
all interface names starting with "eth" and suffixed with numbers will be always reported with ifIndex 1 + (number parsed from name), and for ppp it will be 10000 + (ppp number parsed from name), and finally, interface br0 will always have ifIndex 100.
The idea is to disregard whatever kernel report as ifIndex and provide our own based on our rules (computed or mapped from names), and we can make sure that same name will get same index - this is exactly what I am talking of. How this could be "hack" and "error prone" if we clearly define the rules? Is this really "hard to understand", that defining rule like eth:1 you will have always ifIndex 1-100 for eth0 to eth99?
On the other hand, how it is implemented now, is error prone - say, eth1.100, initially having index 11, may be deleted and created again, and it will get completely random ifIndex - depending on how many interfaces were created since it was deleted.
As to "standards"... the ifIndex value provided by ipt_netflow has nothing to do with SNMP indexes (I've mentioned this already), this is Linux internal ifIndex. Whatever SNMP agent is installed on Linux, it may provide it's own ifIndex which may be different from whatever kernel thinks (example - mini_snmpd, used in some embedded systems).