#1352 Add option to bypass some interfaces

[Collet Thibaut]

Hi,

Monitoring interfaces is costly in term of cpu as interfaces are regurlaly dumped by the run_alarms function through ioctl calls.
With a high number of interfaces snmp becomes no more responding as it needs more than 100% of cpu.

In some circumstances there are no interest to monitor specific interfaces. Avoiding to monitor these interfaces can reduce snmp cpu need.

For that i have added an option (called ignore_if) in configuration file to ignore any interfaces with a name starting with a specific pattern.

Test environement
Software version: commit 2f7dd0e2aebd495d0c528 of the Tue Sep 19 05:38:13 2017 +0200 from Niels Baggesen
Operating system: Linux (Ubuntu16.04 kernel 4.4 but distribution and kernel are not important)

Test plan to check functionality and measure performance:
1. Measure cpu load of snmp on a reference platform with 8 interfaces

1. Create 1000 dummy interface called dummyX (X goes from 1 to 1000).
   Each dummy interface has an IP address. See script below:
   for i in seq 0 3; do
   for j in seq 1 250; do
   let "k=i*250+j"
   ip link add dummy$k type dummy
   ip addr add 192.168.$i.$j/32 dev dummy$k
   ip link set up dev dummy$k
   done
   done

2. Measure cpu load of snmp when:

   • dummy interfaces are monitored
   • dummy interfaces are ignored (thanks the line "ignore_if dummy" in
     the snmpd.conf file

3. Check that ignored interfaces (if any) are not reported by a snmpwalk

Performance result (get with htop for cpu load and perf record for cost of run_alarms function)
- system with 8 interfaces all monitored:
* cpu load less than 1%
* run_alarms function takes 50% of this load:
- 26 % for mteTrigger_run
- 16 % for ifTable_container_load
- 5 % for ipAddressTable_container_load
- system with 1008 interfaces all monitored
* cpu load between 20 and 60%
* run_alarms function takes 86% of this load:
- 3 % for mteTrigger_run
- 81 % for ifTable_container_load
- 2 % for ipAddressTable_container_load
- system with 1008 interfaces 8 monitored, 1000 ignored
* cpu load less than 2%
* run_alarms function takes 64% of this load:
- 4 % for mteTrigger_run
- 57 % for ifTable_container_load
- 2 % for ipAddressTable_container_load

Adding ignored interfaces in the system has a very low impact on snmp cpu load and snmp is still usable to monitor the rest of the system.

[Vincent Jardin]

This patch is OK for me.

[dave finley]

Hello, this would be a very useful patch for snmpd for us. We have servers with upwards of 15K ipsec tunnels concurrently, each with their own unique SVTI interface. These servers are polled using snmpget/walk and the logic in current logic in snmpd that processes all Linux interfaces has caused problems for us due to all of the overhead associated with processsing upwards of 15K SVTI interfaces, when we dont need those statistics anyway. We would like to be able to able to "exclude" these SVTI interfaces somehow from normal SNMPD interface processing during a polling operation. Thx in advance.

[Sam Tannous]

I submitted a patch to ignore interface caching (with -y commandline option). My patch was a bit heavy-handed since users lose all updates for certain if-mib counters. I like this better.

patch was 1325: Add option to disable caching for large number of interfaces

Thanks,
Sam

[Cruz]

Hello, there is an error while applying the patch, could it be?

root@debian:/usr/local/src/net-snmp-code# patch -p1 < IgnoreInterface.patch
patching file agent/mibgroup/etherlike-mib/data_access/dot3stats_linux.c
patching file agent/mibgroup/if-mib/data_access/interface.c
patching file agent/mibgroup/if-mib/data_access/interface_linux.c
Hunk #1 FAILED at 701.
1 out of 1 hunk FAILED -- saving rejects to file agent/mibgroup/if-mib/data_access/interface_linux.c.rej
patching file agent/mibgroup/ip-mib/data_access/ipaddress_ioctl.c
patching file agent/mibgroup/ip-mib/data_access/ipaddress_linux.c
patching file include/net-snmp/data_access/interface.h
root@debian:/usr/local/src/net-snmp-code#

[Bart Van Assche]

Closing this because a modified version of this patch has been checked in. See also [b0f60ff42e01] ("Reduce CPU load for IF-MIB on Linux servers with large number of interfaces").