Symptom

Traffic from endpoint A(EPG_A) to endpoint B(EPG_B) is disrupted when EPG_B is removed from the ESG

Conditions

This can be reproduced systematically. This scenario was found while testing a possible rollback from an EPG to ESG migration Overlay: EPG_A with endpoint EP_A(172.16.11.100) attached to Leaf_A EPG_B with endpoint EP_B(172.16.1.100) attached to Leaf_B Traffic is allowed from EPG_A to EPG_B. Customer associates EPG_A to ESG_A and EPG_B to ESG_B All the traffic continues to works as EPG contracts get inherited by ESGs. The new global pctags are: ESG_A pctag 5476 ESG_B pctag 5477 From Leaf_B: show system internal epm endpoint ip 172.16.11.100 MAC : 0050.56b6.db06 ::: Num IPs : 1 IP# 0 : 172.16.11.100 ::: IP# 0 flags : ::: l3-sw-hit: No Vlan id : 26 ::: Vlan vnid : 9995 ::: VRF name : marvel_tn:prod_vrf BD vnid : 16711542 ::: VRF vnid : 2555905 Phy If : 0x1a000000 ::: Tunnel If : 0 Interface : Ethernet1/1 Flags : 0x82004c04 ::: sclass : 5477 ::: Ref count : 5 EP Create Timestamp : 12/04/2024 11:43:12.209132 EP Update Timestamp : 12/09/2024 13:54:04.036423 EP Flags : local|IP|MAC|sclass|timer|mac-ckt| :::: show system internal epm endpoint ip 172.16.1.100 MAC : 0000.0000.0000 ::: Num IPs : 1 IP# 0 : 172.16.1.100 ::: IP# 0 flags : ::: l3-sw-hit: No Vlan id : 0 ::: Vlan vnid : 0 ::: VRF name : marvel_tn:prod_vrf BD vnid : 0 ::: VRF vnid : 2555905 Phy If : 0 ::: Tunnel If : 0x18010003 Interface : Tunnel3 Flags : 0x80004408 ::: sclass : 5476 ::: Ref count : 3 EP Create Timestamp : 12/09/2024 13:54:34.761754 EP Update Timestamp : 12/09/2024 13:54:35.061827 EP Flags : bounce|IP|sclass|timer| show zoning-rule scope 2555905 dst-epg 5476 src-epg 5477 +---------+--------+--------+----------+----------------+---------+---------+---------------------------------------+--------+---------------+ | Rule ID | SrcEPG | DstEPG | FilterID | Dir | operSt | Scope | Name | Action | Priority | +---------+--------+--------+----------+----------------+---------+---------+---------------------------------------+--------+---------------+ | 4121 | 5477 | 5476 | 9 | uni-dir-ignore | enabled | 2555905 | marvel_tn:marvel_apache_to_tomcat_con | permit | fully_qual(7) | +---------+--------+--------+----------+----------------+---------+---------+---------------------------------------+--------+---------------+ At some point administrator wants to removes EPG_A as EPG Selector for ESG_A. This action is allowed from the APIC. Leaf_B re-programs the zoning-rule table but it doesn't flush the remote EPG_A entry which remains in the table with the previous global pctag. If traffic is not received by EP_A then Leaf_B will never allow traffic from EP_B to EP_A as that will not match any zoning-rule entry.

Workaround

Flush the remote EP from the leaf.

Further Problem Description