Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
N9K switch Fails to restore macsec policy from PSS after the switch is upgarded from 9.3.9 to 9.3.13 and then to 10.3.4a. Link with macsec comes up , however there is no traffic passing through it. CDP/ARP/OSPF hello will not pass. L3 reachability will also be broken. SW1# sh macsec mka session Interface Local-TxSCI # Peers Status Key-Server Auth Mode ------------------ -------------------------------- ------------------ ------------------ ------------------ ------------------ Ethernet1/49 003a.9c67.5a18/0001 1 Secured No PRIMARY-PSK ------------------ -------------------------------- ------------------ ------------------ ------------------ ------------------ Total Number of Sessions : 1 Secured Sessions : 1 Pending Sessions : 0 SW2# sh macsec mka session Interface Local-TxSCI # Peers Status Key-Server Auth Mode ------------------ -------------------------------- ------------------ ------------------ ------------------ ------------------ Ethernet1/49 003a.9c67.5898/0001 1 Secured Yes PRIMARY-PSK ------------------ -------------------------------- ------------------ ------------------ ------------------ ------------------ Total Number of Sessions : 1 Secured Sessions : 1 Pending Sessions : 0 Pending Sessions : 0 SW2# sh macsec policy MACSec Policy Cipher Pri Window Offset Security SAK Rekey time ICV Indicator Include-SCI -------------------------------- ---------------- ---- ------------ -------- -------------- -------------- ------------- ------------- ISN-MACSEC_pol GCM-AES-256 10 148809600 0 must-secure pn-rollover FALSE FALSE system-default-macsec-policy GCM-AES-XPN-256 16 148809600 0 should-secure pn-rollover FALSE TRUE MACSec Policy Cipher-Suite Enforce-Peer -------------------------------- ----------------------------------------------------- SW2# sh macsec policy MACSec Policy Cipher Pri Window Offset Security SAK Rekey time ICV Indicator Include-SCI -------------------------------- ---------------- ---- ------------ -------- -------------- -------------- ------------- ------------- ISN-MACSEC_pol GCM-AES-256 10 148809600 0 must-secure pn-rollover FALSE TRUE system-default-macsec-policy GCM-AES-XPN-256 16 148809600 0 should-secure pn-rollover FALSE TRUE MACSec Policy Cipher-Suite Enforce-Peer -------------------------------- ----------------------------------------------------- SW1# sh run interface e1/49 !Command: show running-config interface Ethernet1/49 !Running configuration last done at: Wed Jan 31 12:24:39 2024 !Time: Wed Jan 31 14:04:27 2024 version 10.3(4a) Bios:version 05.51 interface Ethernet1/49 macsec keychain ISN-MACSEC-kc policy ISN-MACSEC_pol fallback-keychain SN-MACSEC-FALLBACK-kc mtu 9150 ip address 10.255.8.105/30 ip ospf network point-to-point no ip ospf passive-interface ip router ospf 1 area 0.0.0.0 ip pim sparse-mode no shutdown SW2# sh run interface e1/49 !Command: show running-config interface Ethernet1/49 !Running configuration last done at: Wed Jan 31 11:36:22 2024 !Time: Wed Jan 31 14:04:47 2024 version 10.3(4a) Bios:version 05.51 interface Ethernet1/49 macsec keychain ISN-MACSEC-kc policy ISN-MACSEC_pol fallback-keychain ISN-MACSEC-FALLBACK-kc mtu 9150 ip address 10.255.8.106/30 ip ospf network point-to-point no ip ospf passive-interface ip router ospf 1 area 0.0.0.0 ip pim sparse-mode no shutdown SW2# ping 10.255.8.105 PING 10.255.8.105 (10.255.8.105): 56 data bytes 36 bytes from 10.255.8.106: Destination Host Unreachable Request 0 timed out 36 bytes from 10.255.8.106: Destination Host Unreachable Request 1 timed out 36 bytes from 10.255.8.106: Destination Host Unreachable Request 2 timed out 36 bytes from 10.255.8.106: Destination Host Unreachable Request 3 timed out 36 bytes from 10.255.8.106: Destination Host Unreachable --- 10.255.8.105 ping statistics --- 5 packets transmitted, 0 packets received, 100.00% packet loss SW2# sh cdp neighbors >>>>>no e1/49 in cdp list Capability Codes: R - Router, T - Trans-Bridge, B - Source-Route-Bridge S - Switch, H - Host, I - IGMP, r - Repeater, V - VoIP-Phone, D - Remotely-Managed-Device, s - Supports-STP-Dispute Device-ID Local Intrfce Hldtme Capability Platform Port ID 513e-a-17-pod-sw.cisco.com mgmt0 141 S I WS-C2960X-48FPD- Gig1/0/21 SW1.be.net.sys(FDO23041G4X) Eth1/17 136 R S s N9K-C93180YC-FX Eth1/17 Total entries displayed: 2 SW2# sh ip ospf neighbors >>>>>>>>>>>>>>>>No neigh
interface configured with macsec on both the switch and upgrade is performed from 9.3.13 to 10.3.4a Sample example for one of the switch below SW2# sh run macsec !Command: show running-config macsec !Running configuration last done at: Wed Jan 31 11:36:22 2024 !Time: Wed Jan 31 14:07:25 2024 version 10.3(4a) Bios:version 05.51 feature macsec macsec policy ISN-MACSEC_pol cipher-suite GCM-AES-256 key-server-priority 10 security-policy must-secure interface Ethernet1/49 macsec keychain ISN-MACSEC-kc policy ISN-MACSEC_pol fallback-keychain ISN-MACSEC-FALLBACK-kc SW2# sh run | sec key key-chain macsec-psk no-show key chain ISN-MACSEC-FALLBACK-kc macsec key 1000 key-octet-string 7 ****** cryptographic-algorithm AES_256_CMAC send-lifetime 00:00:00 Mar 25 2021 infinite key chain ISN-MACSEC-kc macsec key 10 key-octet-string 7 ****** cryptographic-algorithm AES_256_CMAC send-lifetime 00:00:00 Mar 25 2021 infinite key-server-priority 10 macsec keychain ISN-MACSEC-kc policy ISN-MACSEC_pol fallback-keychain ISN-MACSEC-FALLBACK-kc
Once upgrade from 9.2.x to 9.3.x, change the macsec policy cfg will trigger creation of ext policy pss, which will make it work after upgrade to 10.3.. I would suggest to do the following change which should have no impact: SW2(config-macsec-policy)# macsec policy p2 SW2(config-macsec-policy)# sak-expiry-time ? Time in seconds SW2(config-macsec-policy)# sak-expiry-time 2592000 SW2(config-macsec-policy)# no sak-expiry-time 2592000 Basically sent sak expire time to a large number and remove that. This will trigger creation of ext policy pss to be created. Other policy change will also trigger that, but some of them may also trigger rekey or port flap. After this upgrade to 10.x . We should not see the issue. sci cfg should works fine. You can check that either from "show cts inter debug policy" or "show macsec policy" If you already in 10.x and see the issue, the following cfg change should make it work: Toggle the sci cfg and make sure include-sci is enabled. SW2(config-macsec-policy)# macsec policy p2 SW2(config-macsec-policy)# no include-sci SW2(config-macsec-policy)# include-sci SW2(config-macsec-policy)#