#58 SNMP interface index conversion functionality patch.

[Valentin V. Yankin]

I've made a patch that implements interface name to SNMP-index conversion functionality ( like in fprobe-ulog ).

Each rule consists of interface base name and SNMP-index base separated by colon ( e.g. eth:100,ppp:200 ).
Final SNMP-index is sum of corresponding SNMP-index base and interface number.
For example, SNMP-index of interface eth1 will be converted to 101, index of ppp15 will be converted to 215.
If interface name did not fit to any of conversion rules then SNMP-index will be taken from kernel.

Variable, which contains conversion rules, is called 'snmp_r'.
Values can be written both via sysctl entry net.netflow.snmp_r
( for example `echo eth:100,ppp:200,vlan:300 > /proc/sys/net/netflow/snmp_r` )
or as an modprobe commandline argument
( for example `modprobe ipt_NEFLOW snmp_r=eth:100,ppp:200,vlan:300,br:400` ).

By default, this variable is empty.
If variable is empty, conversion functionality is disabled.

To compile module with snmp conversion functionality,
before building module you must run configure script with option --extra=SNMP_A
( for example ./configure --extra=SNMP_A ).

This functionality was very useful for me.
I hope, it will be useful for other people too.

[ABC]

It seems that you have completely mistunderstood something here.

I reread patch proposal and see that I was really misunderstood the idea. Thanks for bringing my attention to it. To discard ifindex gives more meaning to the idea than I thought before.

As to "standards"... the ifIndex value provided by ipt_netflow has nothing to do with SNMP indexes (I've mentioned this already), this is Linux internal ifIndex. Whatever SNMP agent is installed on Linux, it may provide it's own ifIndex which may be different from whatever kernel thinks (example - mini_snmpd, used in some embedded systems).

This is basically not true.

[Alexander Demenshin]

Finally... Obviously, adding "base" to almost random ifIndex is pointless, I couldn't even imagine that it could be interpreted this way.

This is basically not true.

It is, like it or not - no RFCs or STDs mandate that SNMP agent must use ifIndex reported by Linux kernel (or any other kernel), and previously mentioned mini_snmpd does exactly that - assigns interface numbers in order provided by the user.

It does not matter what was the intention of kernel developers, what really matters is the value reported by SNMP agent.

[ABC]

No RFC say to believe Linux? Lol. RFC is by nature of law is general and not executive.

It does not matter what was the intention of kernel developers, what really matters is the value reported by SNMP agent.

It doesn't really matter what was the intention of mini_snmpd developers, what really matters is the value reported by kernel. Sorry, just wanted to reply to this fun argument. If some lame SNMP agent want to disobey convention that's problem of that agent solely.

But, anyway, I will think about that idea more now. Thanks!

[ABC]

Periodical reporting of changing numbers adds unnecessary complexity to processing

If this is 'unnecessary',..

Well, many people live without ipt_netflow - but you wrote it, nevertheless? :)

'Unnecessary' was your argument, not mine. And I just prove/show/answer to you that appellation to necessity is wrong argumentation. So you don't need to tell me that I did something unnecessary for existence of Universe.

Sorry, but kernel interface numbering has nothing to do with SNMP - this is Linux-internal and (to keep things easy) net-snmp exposes kernel provided indexes - but this is not set in stone, different SNMP implementation may use name-based mapping (this was also discussed many times).

Whoah! Sophistry detected.

Kernel interface numbers have a lot to do with SNMP. They even called ifindex to reflect their MIB nature, they are 'not renumebered' while uptime as SNMP requires, and basically they are source and designed to be source of SNMP ifIndex value. There is nothing non-SNMP in them.

It could even happen that someone will use different SNMP daemon for Linux (not net-snmp) which doesn't use ifindex from kernel - thus ipt_netflow will report totally incorrect indexes - what would you do then?

I will LOL.

Explain this:

root@el6:~# cat /proc/net/dev_snmp6/eth0
ifIndex                                 2

[ABC]

Proposed patch only account first number after matched interface name. I.e. with mask eth:100 interface eth1.200 will be 100+1, and not 100+1200.

+__u16 conv_snmp_index ( __u16 ifindex, const char *name )
+{
+    if ( !list_empty ( &snmp_a_list ) ) {
+        const struct snmp_a *psnmp_a;
+        list_for_each_entry ( psnmp_a, &snmp_a_list, list )
+            if ( 0 == strncmp ( psnmp_a->name, name, psnmp_a->len ) )
+                return ( simple_strtoul( &name[ psnmp_a->len ], NULL, 0 ) + psnmp_a->base );
+    }
+    return ifindex;
+}

[Alexander Demenshin]

I believe than VLANs deserve special treatment - it is unlikely that all 4094 will be used, but entire range will be consumed (base..base+4094) if it will be done "brutally", not mentioning that interfaces are not always have numbers by itself (I've interface "switch" and all VLANs are named like "switch.100"), and even VLAN interfaces don't have to have VLAN tag embedded into the name.

More useful approach will will something like I've proposed before, i.e. "exact match wins" and "map entire VLAN range":

eth1.100:1100 # exact match
eth1.vlan:2000 # 2000 + VLAN id (for cases when there are really many VLANs)

but of course, there is a bit more to do to make it proper...

[ABC]

eth1.vlan:2000 # 2000 + VLAN id (for cases when there are really many VLANs)

This proposition is bit out of simple logic of the patch, nor fprobe-ulog. Is module supposed to patch keyword 'vlan' and then extract vlan from somewhere? Unnecessary complicated!

You can check what info device have: https://github.com/torvalds/linux/blob/master/include/linux/netdevice.h I don't see vlan id clearly.

but of course, there is a bit more to do to make it proper...

If you don't suggest what you need I will do it wrongly, since I don't need this feature at all.

[ABC]

Btw, do you plan to actually use or already using ipfix/netflowv9? Almost nobody report me using it, everybody seems to be using v5 which is quite limited.

[Alexander Demenshin]

I do plan to use v9 soon enough, primarily due to IPv6 support.

There is vlan_dev_priv() function which returns private device data, which, in turn, contains vlan_id.

Minimum useful functionality, from my point of view, will be static exact-match mapping, like this:

eth0:1
eth1:2
eth1.100:100
*:1000 base for all interfaces not explicitly covered by mapping rules (kernel ifIndex + base will be send)

It is simple enough, doesn't need expensive strtoul() conversions on every lookup (this will hurt badly on loaded system), and will cover most common cases.

And please, don't get me wrong - I am not forcing you to do all the dirty work, I'll take a deeper look into this and probably could help (though last time I touched kernel code about 15 years ago, not counting my patch for procfs here, so I need time to get familiar with recent kernels).

[ABC]

Simple strtoul could be much less expensive than strcmp search in the linked list. vlan_dev_priv is struct, btw, not function, and I dont see link to it from struct net_device.

[Alexander Demenshin]

Take a look: http://lxr.free-electrons.com/ident?i=vlan_dev_priv

There is a struct with same name, sure - which is returned by this function (it takes net_device pointer as argument), and, by the way, there is a member called vlan_info (in net_device), but it is a bit too complicated to use (and surely very expensive).

strtoul() is more expensive as it needs more ops for every character processed, and we still need strncmp() for prefix match.

[ABC]

strtoul or better atoi could be implemented easily and be very fast. I think about taking what is done in the proposed patch (w/o accessing any additional data from the dev struct), except move call to interface resolver (conv_snmp_index) from packet processor to exporter (which is called much less frequent). So in your case instead of eth0:1,eth1:2,eth1.100:100 it will be eth1.:100,eth:1 (note order).

To log vlan id from interface structs for ipfix/v9 could be additional feature. They have some vlanId, dot1qVlanId elements.

[ABC]

Hello, I implemented SNMP-index conversion rules based on the patch from this ticket and discussion with aldem, see

https://sourceforge.net/p/ipt-netflow/code/ci/271d15912ddd37dd1f319b7007b2089bb929dbff/

./configure --enable-snmp-rules to enable functionality.

[ABC]

FYI. I also made support in netflow v9/ipfix mode to periodically export list of interface names with descriptions. Similar to what cisco do. With this list you don't even need to translate ifIndex, because you can have full device names. (Additional field 'description' will have driver name, speed, port OR ifAlias (ip link alias) setting. See README in git head.

[ABC]

Added you into CREDITS file as prototype author. You may wish to send update or additional info.

To view latest version of CREDITS file: https://sourceforge.net/p/ipt-netflow/code/ci/master/tree/CREDITS