Loading...
Loading...
ARP replies are not being processed by VXLAN leaf with a failure reason of subnetMismatch N9k# show ip arp internal event-history errors 2023 Sep 27 16:40:35.889617: E_DEBUG arp [21993]: ARP process packet ok2Lesrn dropGlean=Disabled, Event=Validation, Result=subnetMismatch, IOD=5, Interface=Vlan2723, srcIP=172.20.130.6 2023 Sep 27 16:40:29.885363: E_DEBUG arp [21993]: ARP Send Request: Result=Failed, Resaon=UnAttachedIP, VRF=test-vrf, IP=172.20.130.6, Interface=Vlan2723, IOD=5 ARP incomplete will be seen on the ARP table No interface/ELAM drops ARP replies sent to other vPC peer will be processed & ARP entry created successfully.
Using "maximum-paths mixed" under the BGP/VRF configuration, this is a must to match the issue router bgp 65000 vrf test-vrf address-family ipv4 unicast redistribute direct route-map allow maximum-paths mixed 8 <<<<<<<<<<<<<<<<<<<<< Subnet route is received from BGP and as direct route: N9k2# show ip route 172.20.130.4 vrf test-vrf IP Route Table for VRF "test-vrf" '*' denotes best ucast next-hop '**' denotes best mcast next-hop '[x/y]' denotes [preference/metric] '%' in via output denotes VRF 172.20.130.4/30, ubest/mbest: 1/0, attached *via 172.20.130.5, Vlan2723, [0/0], 00:00:22, direct via 192.168.1.2%default, [200/0], 00:00:19, bgp-65000, internal, tag 65000, segid: 2214 tunnelid: 0xc0a80102 encap: VXLAN With "maximum-paths mixed" enabled the direct route might not be placed as the top in the list: output from the VPC peer: N9k# show ip route 172.20.130.4 vrf test-vrf IP Route Table for VRF "test-vrf" '*' denotes best ucast next-hop '**' denotes best mcast next-hop '[x/y]' denotes [preference/metric] '%' in via output denotes VRF 172.20.130.4/30, ubest/mbest: 2/0, attached, all-best (0xc0a80102) *via 192.168.1.1%default, [200/0], 00:00:26, bgp-65000, internal, tag 65000, segid: 2214 tunnelid: 0xc0a80101 encap: VXLAN *via 172.20.130.5, Vlan2723, [0/0], 00:00:24, direct <<<<<< it is not placed as the TOP ARP will not be processed if direct route for the subnet where ARP is received is not listed at the TOP on the URIB
Clear the subnet route Remove the "maximum-paths mixed" under the VRF the subnet route is in
If the all-best flags is not in sync between BGP and URIB, BGP does not resync the all-best flag if there are no updates to the path itself. With "maximum-paths mixed" enabled, best path selection is not done based on the AD/metrics. Instead this is done through sorting based on the VTEP IP - the lower IP will be sorted 1st. When an ARP reply is received, the ARP process performs an API call to the URIB to get the "best direct route". If value returned is the direct route, ARP will consider the subnet active and reply will be processed to create an ARP entry. But as URIB order/placement is no longer depending on the AD, if the direct route is not placed as 1st - the issue will be hit. "maximum-paths mixed" is only intended for routes either local to the device (static, iBGP, or eBGP) or remote (eBGP or iBGP learned over BGP-EVPN). It is not applicable to direct subnet/host routes. Transition of route learned only remotely to also directly causes break.
Cisco Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.