OPERATIONAL DEFECT DATABASE
...
...
L2RIB could fail to increment sequence number after host changes its MAC address resulting in traffic loss for up to 5 minutes when recovery occurs automatically.
Active-Active EVPN IRB Distributed Anycast Gateway solution. There is site with two PEs (PE1 and PE2). There is host with MAC1 which is learned (local arp entry) on PE1. Host changes its MAC address to MAC2 (IP is the same) and it is learned (local arp entry) again on PE1. PE1 must have multiple LCs, otherwise with a single LC the issue is not seen. Under some specific conditions the new route may fail to be preferred on PE2 which causes failure to update forwarding entry for this host to new MAC2 and eventually traffic mis-forwarding (to old MAC1). After 5 minutes the old route to MAC1 will be removed from all PEs and reachability will be recovered.
There is no workaround
The issue is dependent on PE1 having multiple LCs. The issue is also affected by timing so it may not happen every time. ######################### Last two MAC change events. 2023-06-01 07:02:20.236888: host reappeared with MAC 0011.0000.0002 and IP 10.47.177.225 2023-06-01 07:03:31.984150: host reappeared with MAC 0011.0000.0003 and IP 10.47.177.225 ######################### ######################### Outputs in problem state. ######################### PE1 RP/0/RSP0/CPU0:PE1#show arp vrf OM_BC location 0/3/cpu0 Thu Jun 1 07:03:48.627 CEST Address Age Hardware Addr State Type Interface 10.47.177.225 00:00:08 0011.0000.0003 Dynamic ARPA BVI1212 RP/0/RSP0/CPU0:PE1#show l2route evpn mac-ip 10.47.177.225 detail Thu Jun 1 07:03:52.837 CEST Topo ID Mac Address IP Address Producer Next Hop(s) Seq No Flags Opaque Data Type Opaque Data Len Opaque Data Value Opaque NH Type Opaque NH Len Opaque NH Value -------- -------------- --------------- ----------- ---------------------------------------- -------- -------- 157 0011.0000.0003 10.47.177.225 LOCAL Bundle-Ether8.1212, N/A 22 BL N/A N/A N/A N/A N/A N/A Last Update: Thu Jun 01 07:03:39.553 RP/0/RSP0/CPU0:PE1# PE2 RP/0/RSP0/CPU0:PE2#show arp vrf OM_BC location 0/0/cpu0 Thu Jun 1 07:03:48.164 CEST Address Age Hardware Addr State Type Interface 10.47.177.225 - 0011.0000.0002 EVPN_SYNC ARPA BVI1212 <<<<<<<<<<< still points to old MAC RP/0/RSP0/CPU0:PE2#show l2route evpn mac-ip 10.47.177.225 detail Thu Jun 1 07:03:50.968 CEST Topo ID Mac Address IP Address Producer Next Hop(s) Seq No Flags Opaque Data Type Opaque Data Len Opaque Data Value Opaque NH Type Opaque NH Len Opaque NH Value -------- -------------- --------------- ----------- ---------------------------------------- -------- -------- 27 0011.0000.0002 10.47.177.225 L2VPN Bundle-Ether8.1212, N/A 34 BS 0 12 0x06000000 0x35000080 0x00000000 N/A N/A N/A Last Update: Thu Jun 01 07:02:28.848 27 0011.0000.0003 10.47.177.225 L2VPN Bundle-Ether8.1212, N/A 22 BS 0 12 0x06000000 0x35000080 0x00000000 N/A N/A N/A Last Update: Thu Jun 01 07:03:39.594 RP/0/RSP0/CPU0:PE2# ######################### Outputs after recovery. ######################### PE1. same as before PE2. RP/0/RSP0/CPU0:PE2#show arp vrf OM_BC location 0/0/cpu0 Thu Jun 1 07:10:45.823 CEST Address Age Hardware Addr State Type Interface <<<<<<<<<<<<< there is no entry about 10.47.177.225 RP/0/RSP0/CPU0:PE2#show l2route evpn mac-ip 10.47.177.225 detail Thu Jun 1 07:10:48.771 CEST Flags: (Stt)=Static; (L)=Local; (Lp)=Local-Proxy; (R)=Remote; (N)=No Redistribution; (B)=Best Route; (P)=Probe; (S)=Peer Sync; (F)=Flush; (Dm)=Duplicate MAC; (Zm)=Frozen MAC; (Di)=Duplicate IP; (Zi)=Frozen IP; (Sfa)=Single Flow Active; (Spec)=Speculative Topo ID Mac Address IP Address Producer Next Hop(s) Seq No Flags Opaque Data Type Opaque Data Len Opaque Data Value Opaque NH Type Opaque NH Len Opaque NH Value -------- -------------- --------------- ----------- ---------------------------------------- -------- -------- 27 0011.0000.0003 10.47.177.225 L2VPN Bundle-Ether8.1212, N/A 22 BS 0 12 0x06000000 0x35000080 0x00000000 N/A N/A N/A Last Update: Thu Jun 01 07:03:39.594 RP/0/RSP0/CPU0:PE2# RP/0/RSP0/CPU0:PE2#show cef vrf OM_BC retry-db location 0/0/cpu0 Thu Jun 1 09:03:11.911 CEST -------------------- ---------- ----- ---------- --------------------- Obj-Type Retry Retry Scheduling Timestamp Flags Count Class -------------------- ---------- ----- ---------- --------------------- TI_RETRY 0x0 454 Slow Jun 1, 2023 07:09:34 Error code: 0x7faa, 'Subsystem(0)' detected the 'success' condition 'Code(63)' retry-type:ADJ-RIB-UPD addr:10.47.177.225, ifh:0x20002e0, pref:high RP/0/RSP0/CPU0:PE2# RP/0/RSP0/CPU0:PE2#show cef vrf OM_BC 10.47.177.225/32 detail location 0/0/cpu0 Thu Jun 1 07:10:48.213 CEST 10.47.177.225/32, version 171, internal 0x5000001 0xf0 (ptr 0x7e21c884) [1], 0x0 (0x0), 0x208 (0x7efc88a8) Updated Jun 1 07:03:39.594 Prefix Len 32, traffic index 0, precedence n/a, priority 15 gateway array (0x84175d68) reference count 34, flags 0x2038, source rib (7), 0 backups [1 type 1 flags 0x48441 (0x7f019b60) ext 0x0 (0x0)] LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0] gateway array update type-time 1 May 31 16:48:27.016 LDI Update time May 31 16:48:27.016 via 10.0.0.25/32, 7 dependencies, recursive [flags 0x6000] <<<<<<< traffic is sent to PE1, not forwarded locally path-idx 0 NHID 0x0 [0x7f1079c8 0x0] recursion-via-/32 next hop VRF - 'default', table - 0xe0000000 next hop 10.0.0.25/32 via 16125/0/21 next hop 10.0.48.41/32 Te0/0/0/1 labels imposed {16125 25147} Load distribution: 0 (refcount 1) Hash OK Interface Address 0 Y recursive 16125/0 RP/0/RSP0/CPU0:PE2#
Click on a version to see all relevant bugs
Cisco Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.
