#1354 Linux Kernel Virtual Routing and Forwarding (VRF) patch

[Sam Tannous]

Since the Linux kernel added support for Virtual Routing and Forwarding (VRF) in version 4.3

https://www.kernel.org/doc/Documentation/networking/vrf.txt

Linux users could not use snmpd in its current form.

For UDP sockets, snmpd is passing a cmsg with IP_PKTINFO set and setting the interface index to 0, in Linux (with VRF), this will override the bind to VRF which is set by 'vrf exec' command.

So to avoid breaking VRF, this patch will use sendto() instead of sendmsg() and let the kernel handle the routing decision and keep VRF working.

Thanks,
Sam Tannous
Cumulus Networks
(tested patch against V5-7-patches)

[Bill Fenner]

Hi Sam,

Falling back to sendto() means that the source address will not be used, a regression on multi-homed hosts like most routers.

I looked through the documentation you pointed to and did not see a specification of why passing an ifindex value of 0 overrides the VRF. Do you have a pointer to why this happens? Maybe we can find a way to not override the VRF but still specify the source address.

Thanks,
Bill

[Sam Tannous]

Hi Bill,

The linux kernel is calling udp_sendmsg() and this sets ipc.oif =
sk->sk_bound_dev_if;
where sk_bound_dev_if is the SO_BINDTODEVICE settings right after that
code it checks for msg_controllen.

If IP_PKTINFO exists in the msg, then ipc.oif is reset to the passed
in ifindex
which for snmp is 0 that breaks the VRF binding by undoing the sk_bound_dev_if

I think the way to not unbind the VRF is to not pass in an ifindex of 0.
It's not clear to me why snmpd adds a cmsg with the ifindex of 0.

Thanks,
Sam

On Fri, Feb 16, 2018 at 12:41 PM, Bill Fenner
fenner@users.sourceforge.net wrote:

Hi Sam,

Falling back to sendto() means that the source address will not be used, a
regression on multi-homed hosts like most routers.

I looked through the documentation you pointed to and did not see a
specification of why passing an ifindex value of 0 overrides the VRF. Do you
have a pointer to why this happens? Maybe we can find a way to not override
the VRF but still specify the source address.

Thanks,
Bill

---

[patches:#1354] Linux Kernel Virtual Routing and Forwarding (VRF) patch

Status: open
Group: backport-needed
Labels: linux
Created: Thu Feb 15, 2018 07:28 PM UTC by Sam Tannous
Last Updated: Thu Feb 15, 2018 07:28 PM UTC
Owner: nobody
Attachments:

linux-vrf-patch.diffs (979 Bytes; application/octet-stream)

Since the Linux kernel added support for Virtual Routing and Forwarding
(VRF) in version 4.3

https://www.kernel.org/doc/Documentation/networking/vrf.txt

Linux users could not use snmpd in its current form.

For UDP sockets, snmpd is passing a cmsg with IP_PKTINFO set and setting the
interface index to 0, in Linux (with VRF), this will override the bind to
VRF which is set by 'vrf exec' command.

So to avoid breaking VRF, this patch will use sendto() instead of sendmsg()
and let the kernel handle the routing decision and keep VRF working.

Thanks,
Sam Tannous
Cumulus Networks
(tested patch against V5-7-patches)

---

Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/net-snmp/patches/1354/

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/

[Bill Fenner]

When you set ipi.ipi_ifindex to a non-zero value, the ipi.ipi_spec_dst.s_addr is ignored. Zero means "Use whatever interface you think is best"; nonzero means "use this interface and also use the source address of this interface". Consider the case of:

#                     10.1.1.0/24
#                ---------------------
#                 |                 | eth0
# manager ---- router1           router2 --- loopback 10.3.3.3
#                 |                 | eth1
#                ---------------------
#                     10.2.2.0/24

If manager is talking to 10.3.3.3, and router1's route to 10.3.3.3 is via 10.1.1.x, then the ifindex points to router2's eth0 (it's the received ifindex) - and if router2 sends using ipi_spec_dst 10.3.3.3 (to try to make sure the manager sees the packet from the same address that it queried) and ipi_ifindex to eth0, then the packet will come from 10.1.1.x.

[David Ahern]

Bill: Regarding your comment "When you set ipi.ipi_ifindex to a non-zero value, the ipi.ipi_spec_dst.s_addr is ignored.", that is not true for Linux. If in_pktinfo contains an address it is used for the source address.

[Bill Fenner]

Hi David,

I was referring back to the commit message when I made this change years
ago. I may have been wrong at the time, or the kernel behavior may have
changed since then.

Let's ignore that for now, and think about how the kernel would behave when
specifying an ifindex value. Ignoring what the source address would be,
let's talk about sending the packet. If router2 receives the packet on
eth0 and records that if_index value, but its routing table back to the
manager points to eth1, then what will it do if we send the packet with
if_index pointing to eth0? The only reasonable thing it can do is to ARP
for the manager on eth0, and since it's not attached to eth0, it won't
reply. router1 might reply to the ARP if it's configured for proxy arp,
but that is not commonly configured and especially not by default. So, how
can router2 get the packet back to the manager if it has to specify an
if_index? It effectively needs to do a routing table lookup -- but that is
what is requested by if_index 0.

This looks like a case where the Linux socket API ends up putting the
application in a no-win situation. If you're a multi-homed router, do
this; if you're a router using VRFs do that; if you're a multi-homed router
using VRFs then...?

I wouldn't be opposed to having a configure arg to select which position
you want to be in. My product implements VRFs using network namespaces, so
does not have this problem; we can just select the existing behavior. Sam
can select the other option and then hope that he doesn't end up with the
multi-homed source address problem.

Bill

[David Ahern]

I understand the intent of net-snmp using IP_PKTINFO -- set source address but don't constrain the route lookup for the response to the interface the packet is received on.

For VRF, the same result is obtained by setting ifindex to the VRF device index. VRF, as a NOS feature, was designed for switches and routers so by definition works in multi-homed, multi-source address (duplicate source address, etc) deployments.

I submitted a bug fix to the kernel to prevent IP_PKTINFO with ifindex = 0 from overriding SO_BINDTODEVICE. This has no effect on your situation and allows one VRF configuration to work - namely, 'ip vrf exec <NAME> snmpd' which sets the VRF for snmpd. The full solution requires net-snmp to support VRF aware config statements. Sam is working on that.

[Bart Van Assche]

Support for SO_BINDTODEVICE has been checked in on the v5.8 and master branches. Is that sufficient to close this ticket?

[Sam Tannous]

Hello Bart,

For Linux, not quite. For the asymetric case (request comes in one interface, and response is forced out a different interface), we had to use sendto() instead of sendmsg() in snmpUDPBaseDomain.c.

This lets the kernel do what it should if there is an SO_BINDTODEVICE iface set. The chunk above
that (in the attached diff against V5-8-pathces) correctly sets the ipi.ipi_ifindex in case it
is needed further down.

Thanks,
Sam

[Bart Van Assche]

A modified version of this patch has been checked in on the v5.8 and master branches. Please retest.

[Sam Tannous]

Thanks, Bart. I merged in your patch and it works ok.

--Sam

[Gareth]

Will this be patched in the 5.7.3 version?

[Bart Van Assche]

The v5.7 branch is no longer supported. Please upgrade to v5.8.x.