[Keepalived-devel] Questions about use_vmac on bond interface...

[Sean <sma...@gm...>]

Hello,

I am building a pair of routers using keepalived-2.0.10-10.el8.x86_64
on CentOS/8.  This router pair is also running frr-7.0-5.el8.x86_64
and advertising our subnet upstream to the internet as an AS with BGP.
Each router has a different ISP interface, as well as a cross-connect
interface for iBGP peering.  I am currently modelling everything in a
Vagrant environment provisioned with ansible, so it's easy to scratch
and restart fresh to try different approaches.

The routers each have a bond interface called "core".  Core has the .2
(router 1) and .3 (router 2) IPs on the subnet, with the keepalived
VIP as the .1.  There is a client VM on this network with .100 as it's
IP.

The keepalived config seems very simple...
vrrp_instance VI_1 {
    interface core
    state MASTER
    use_vmac
    vmac_xmit_base
    virtual_router_id 51
    virtual_ipaddress {
        192.168.192.1
    }
    track_script {
        chk_frr
    }
    unicast_src_ip 192.168.192.2
    unicast_peer {
              192.168.192.3

    }
}

Relevant interface info -
# ip a show core
9: core: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc
noqueue state UP group default qlen 1000
    link/ether 08:00:27:81:27:3b brd ff:ff:ff:ff:ff:ff
    inet 192.168.192.2/22 brd 192.168.195.255 scope global noprefixroute core
       valid_lft forever preferred_lft forever
    inet6 fe80::7d4:ec78:9cd3:c02a/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc
fq_codel master core state UP group default qlen 1000
    link/ether 08:00:27:81:27:3b brd ff:ff:ff:ff:ff:ff
# ip a show eth3
5: eth3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc
fq_codel master core state UP group default qlen 1000
    link/ether 08:00:27:81:27:3b brd ff:ff:ff:ff:ff:ff
# ip a show vrrp.51
11: vrrp.51@core: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue state UP group default qlen 1000
    link/ether 00:00:5e:00:01:33 brd ff:ff:ff:ff:ff:ff
    inet 192.168.192.1/32 scope global vrrp.51
       valid_lft forever preferred_lft forever

Under this config, the .3 is BACKUP, and can ping the .1 when the .1
is on .2 (or .3 obviously).  But the .100 client can not ping the .1
at all, even though it can ping the .2 and .3 just fine.  Running arp
-env on the .100 client shows 192.168.192.1 Incomplete.  TCPDUMP on
either core or vrrp.51 show the arp request received, looks like it's
not sending the arp response though, maybe?  I don't understand why
the .100 can't ping the .1 or what needs to happen to make this work.
  Is it possible that FRR and Keepalived network kernel params or
other networking config conflict with each other?

I'm looking for suggestions and I'm open to alternatives to use_vmac
as long as the real client devices will reconnect quickly when the VIP
flips routers.

Thanks for listening!

--Sean