Loading...
Loading...
1. The fibre interface not coming up on N9K-C9336C-FX2 post upgrading to NXOS 10.3.6. 2. we need to set speed manually to 40000 on interface to bring up. 3. Its replicable in lab.
1. The fibre interface not coming up on N9K-C9336C-FX2 post upgrading to NXOS 10.3.6. 2. we need to set speed manually to 40000 on interface to bring up. 3. Its replicable in lab.
1. Device was running on software codes 9.3(13). 2. Customer upgraded device to NXOS 10.3.6. 3. SFP QSFP-H40G-AOC5M is used on interface. 4. Post upgrade interface didn't come up. 5. After setting the speed manually interfaces came up. 6. we replicated in lab with same software and hardware, and its reproduced. The following Lab tests were performed: 1/The version before the upgrade is 9.3(13), using the fiber module QSFP-H40G-AOC5M that is consistent with the customer environment N9K-1(config)# sh version Software BIOS: version 05.51 NXOS: version 9.3(13) BIOS compile time: 11/29/2023 NXOS image file is: bootflash:///nxos.9.3.13.bin NXOS compile time: 1/31/2024 12:00:00 [12/13/2023 14:06:50] Hardware cisco Nexus9000 C9336C-FX2 Chassis Intel(R) Xeon(R) CPU D-1526 @ 1.80GHz with 24569944 kB of memory. Processor Board ID FDO233011BV Device name: N9K-1 bootflash: 12503040 kB Kernel uptime is 0 day(s), 0 hour(s), 38 minute(s), 37 second(s) Last reset at 154312 usecs after Sun Mar 2 11:20:06 2025 Reason: Reset Requested by CLI command reload System version: 9.3(13) Service: plugin Core Plugin, Ethernet Plugin Active Package(s): N9K-1(config)# sh vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 1 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled Delay-restore status : Timer is off.(timeout = 30s) Delay-restore SVI status : Timer is off.(timeout = 10s) Operational Layer3 Peer-router : Disabled Virtual-peerlink mode : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ ------------------------------------------------- 1 Po1 up 1 N9K-1(config)# show int eth1/34-36 transceiver | i type type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M version 9.3(13) Bios:version 05.51 interface Ethernet1/34 vrf member vPC-keepalive ip address 192.168.10.1/30 no shutdown interface Ethernet1/35 switchport switchport mode trunk channel-group 1 mode active no shutdown interface Ethernet1/36 switchport switchport mode trunk channel-group 1 mode active no shutdown ##### N9K-2(config)# sh ver Software BIOS: version 05.51 NXOS: version 9.3(13) BIOS compile time: 11/29/2023 NXOS image file is: bootflash:///nxos.9.3.13.bin NXOS compile time: 1/31/2024 12:00:00 [12/13/2023 14:06:50] Hardware cisco Nexus9000 C9336C-FX2 Chassis Intel(R) Xeon(R) CPU D-1526 @ 1.80GHz with 24569944 kB of memory. Processor Board ID FDO2148050L Device name: N9K-2 bootflash: 25006080 kB Kernel uptime is 0 day(s), 0 hour(s), 39 minute(s), 48 second(s) Last reset at 560973 usecs after Sun Mar 2 11:14:48 2025 Reason: Reset Requested by CLI command reload System version: 9.3(13) Service: plugin Core Plugin, Ethernet Plugin Active Package(s): N9K-2(config)# sh vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 1 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : secondary Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled Delay-restore status : Timer is off.(timeout = 30s) Delay-restore SVI status : Timer is off.(timeout = 10s) Operational Layer3 Peer-router : Disabled Virtual-peerlink mode : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ ------------------------------------------------- 1 Po1 up 1 N9K-2(config)# show int eth1/34-36 transceiver | i type type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M version 9.3(13) Bios:version 05.51 interface Ethernet1/34 vrf member vPC-keepalive ip address 192.168.10.2/30 no shutdown interface Ethernet1/35 switchport switchport mode trunk channel-group 1 mode active no shutdown interface Ethernet1/36 switchport switchport mode trunk channel-group 1 mode active no shutdown ### -------------------------------------------------- #### 2/After upgrading to version 10.3(6), the Link not connected issue does occur after rebooting. The speed must be manually set to 40000 for the port to come up. N9K-1# sh version Software BIOS: version 05.51 NXOS: version 10.3(6) [Maintenance Release] <<<<<<<<<<<<<<<<<<<<<<<<< BIOS compile time: 11/29/2023 NXOS image file is: bootflash:///nxos64-cs.10.3.6.M.bin NXOS compile time: 8/20/2024 12:00:00 [08/20/2024 19:31:13] Hardware cisco Nexus9000 C9336C-FX2 Chassis Intel(R) Xeon(R) CPU D-1526 @ 1.80GHz with 24545588 kB of memory. Processor Board ID FDO233011BV Device name: N9K-1 bootflash: 12503040 kB Kernel uptime is 0 day(s), 0 hour(s), 5 minute(s), 16 second(s) Last reset at 950836 usecs after Mon Mar 3 15:17:24 2025 Reason: Reset due to upgrade System version: 9.3(13) Service: plugin Core Plugin, Ethernet Plugin Active Package(s): N9K-1# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 1 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary, operational secondary Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled Delay-restore status : Timer is off.(timeout = 30s) Delay-restore SVI status : Timer is off.(timeout = 10s) Delay-restore Orphan-port status : Timer is off.(timeout = 0s) Operational Layer3 Peer-router : Disabled Virtual-peerlink mode : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ ------------------------------------------------- 1 Po1 up 1 N9K-1# show int eth1/34-36 transceiver | i type type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M version 10.3(6) Bios:version 05.51 interface Ethernet1/34 vrf member vPC-keepalive ip address 192.168.10.1/30 no shutdown interface Ethernet1/35 switchport switchport mode trunk channel-group 1 mode active no shutdown interface Ethernet1/36 switchport switchport mode trunk channel-group 1 mode active no shutdown Eth1/34 -- eth routed up none 40G(D) -- Eth1/35 1 eth trunk up none 40G(D) 1 Eth1/36 1 eth trunk up none 40G(D) 1 #### --------------- #### N9K-2# sh ver Software BIOS: version 05.51 NXOS: version 10.3(6) [Maintenance Release] BIOS compile time: 11/29/2023 NXOS image file is: bootflash:///nxos64-cs.10.3.6.M.bin NXOS compile time: 8/20/2024 12:00:00 [08/20/2024 19:31:13] Hardware cisco Nexus9000 C9336C-FX2 Chassis Intel(R) Xeon(R) CPU D-1526 @ 1.80GHz with 24545588 kB of memory. Processor Board ID FDO2148050L Device name: N9K-2 bootflash: 25006080 kB Kernel uptime is 0 day(s), 0 hour(s), 10 minute(s), 49 second(s) Last reset at 545181 usecs after Mon Mar 3 15:24:45 2025 Reason: Reset due to upgrade System version: 9.3(13) Service: plugin Core Plugin, Ethernet Plugin Active Package(s): N9K-2# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 1 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : secondary Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled Delay-restore status : Timer is off.(timeout = 30s) Delay-restore SVI status : Timer is off.(timeout = 10s) Delay-restore Orphan-port status : Timer is off.(timeout = 0s) Operational Layer3 Peer-router : Disabled Virtual-peerlink mode : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ ------------------------------------------------- 1 Po1 up 1 N9K-2# show int eth1/34-36 transceiver | i type type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M version 10.3(6) Bios:version 05.51 interface Ethernet1/34 vrf member vPC-keepalive ip address 192.168.10.2/30 no shutdown interface Ethernet1/35 switchport switchport mode trunk channel-group 1 mode active no shutdown interface Ethernet1/36 switchport switchport mode trunk channel-group 1 mode active no shutdown Eth1/34 -- eth routed up none 40G(D) -- Eth1/35 1 eth trunk up none 40G(D) 1 Eth1/36 1 eth trunk down Link not connected auto(D) 1 <<<<<<<<< ***** Happen issue: N9K-1 Eth1/34 -- eth routed up none 40G(D) -- Eth1/35 1 eth trunk up none 40G(D) 1 Eth1/36 1 eth trunk down Link not connected auto(D) 1 <<<<<<<<<<<<< N9K-2 Eth1/34 -- eth routed up none 40G(D) -- Eth1/35 1 eth trunk up none 40G(D) 1 Eth1/36 1 eth trunk down Link not connected auto(D) 1 <<<<<<<<<<<<<<< **** Need to config speed 40000 both switch's interface eth1/36 or reload after up . interface eth1/36 speed 40000 N9K-1 Eth1/34 -- eth routed up none 40G(D) -- Eth1/35 1 eth trunk up none 40G(D) 1 Eth1/36 1 eth trunk up none 40G(D) 1 N9K-2 Eth1/34 -- eth routed up none 40G(D) -- Eth1/35 1 eth trunk up none 40G(D) 1 Eth1/36 1 eth trunk up none 40G(D) 1
Need to set speed manually to 40000 on interface to bring up
LAB TESTING RESULTS 1/The version before the upgrade is 9.3(13), using the fiber module QSFP-H40G-AOC5M that is consistent with the customer environment N9K-1(config)# sh version Software BIOS: version 05.51 NXOS: version 9.3(13) BIOS compile time: 11/29/2023 NXOS image file is: bootflash:///nxos.9.3.13.bin NXOS compile time: 1/31/2024 12:00:00 [12/13/2023 14:06:50] Hardware cisco Nexus9000 C9336C-FX2 Chassis Intel(R) Xeon(R) CPU D-1526 @ 1.80GHz with 24569944 kB of memory. Processor Board ID FDO233011BV Device name: N9K-1 bootflash: 12503040 kB Kernel uptime is 0 day(s), 0 hour(s), 38 minute(s), 37 second(s) Last reset at 154312 usecs after Sun Mar 2 11:20:06 2025 Reason: Reset Requested by CLI command reload System version: 9.3(13) Service: plugin Core Plugin, Ethernet Plugin Active Package(s): N9K-1(config)# sh vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 1 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled Delay-restore status : Timer is off.(timeout = 30s) Delay-restore SVI status : Timer is off.(timeout = 10s) Operational Layer3 Peer-router : Disabled Virtual-peerlink mode : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ ------------------------------------------------- 1 Po1 up 1 N9K-1(config)# show int eth1/34-36 transceiver | i type type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M version 9.3(13) Bios:version 05.51 interface Ethernet1/34 vrf member vPC-keepalive ip address 192.168.10.1/30 no shutdown interface Ethernet1/35 switchport switchport mode trunk channel-group 1 mode active no shutdown interface Ethernet1/36 switchport switchport mode trunk channel-group 1 mode active no shutdown ##### N9K-2(config)# sh ver Software BIOS: version 05.51 NXOS: version 9.3(13) BIOS compile time: 11/29/2023 NXOS image file is: bootflash:///nxos.9.3.13.bin NXOS compile time: 1/31/2024 12:00:00 [12/13/2023 14:06:50] Hardware cisco Nexus9000 C9336C-FX2 Chassis Intel(R) Xeon(R) CPU D-1526 @ 1.80GHz with 24569944 kB of memory. Processor Board ID FDO2148050L Device name: N9K-2 bootflash: 25006080 kB Kernel uptime is 0 day(s), 0 hour(s), 39 minute(s), 48 second(s) Last reset at 560973 usecs after Sun Mar 2 11:14:48 2025 Reason: Reset Requested by CLI command reload System version: 9.3(13) Service: plugin Core Plugin, Ethernet Plugin Active Package(s): N9K-2(config)# sh vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 1 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : secondary Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled Delay-restore status : Timer is off.(timeout = 30s) Delay-restore SVI status : Timer is off.(timeout = 10s) Operational Layer3 Peer-router : Disabled Virtual-peerlink mode : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ ------------------------------------------------- 1 Po1 up 1 N9K-2(config)# show int eth1/34-36 transceiver | i type type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M version 9.3(13) Bios:version 05.51 interface Ethernet1/34 vrf member vPC-keepalive ip address 192.168.10.2/30 no shutdown interface Ethernet1/35 switchport switchport mode trunk channel-group 1 mode active no shutdown interface Ethernet1/36 switchport switchport mode trunk channel-group 1 mode active no shutdown ### -------------------------------------------------- #### 2/After upgrading to version 10.3(6), the Link not connected issue does occur after rebooting. The speed must be manually set to 40000 for the port to come up. N9K-1# sh version Software BIOS: version 05.51 NXOS: version 10.3(6) [Maintenance Release] <<<<<<<<<<<<<<<<<<<<<<<<< BIOS compile time: 11/29/2023 NXOS image file is: bootflash:///nxos64-cs.10.3.6.M.bin NXOS compile time: 8/20/2024 12:00:00 [08/20/2024 19:31:13] Hardware cisco Nexus9000 C9336C-FX2 Chassis Intel(R) Xeon(R) CPU D-1526 @ 1.80GHz with 24545588 kB of memory. Processor Board ID FDO233011BV Device name: N9K-1 bootflash: 12503040 kB Kernel uptime is 0 day(s), 0 hour(s), 5 minute(s), 16 second(s) Last reset at 950836 usecs after Mon Mar 3 15:17:24 2025 Reason: Reset due to upgrade System version: 9.3(13) Service: plugin Core Plugin, Ethernet Plugin Active Package(s): N9K-1# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 1 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary, operational secondary Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled Delay-restore status : Timer is off.(timeout = 30s) Delay-restore SVI status : Timer is off.(timeout = 10s) Delay-restore Orphan-port status : Timer is off.(timeout = 0s) Operational Layer3 Peer-router : Disabled Virtual-peerlink mode : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ ------------------------------------------------- 1 Po1 up 1 N9K-1# show int eth1/34-36 transceiver | i type type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M version 10.3(6) Bios:version 05.51 interface Ethernet1/34 vrf member vPC-keepalive ip address 192.168.10.1/30 no shutdown interface Ethernet1/35 switchport switchport mode trunk channel-group 1 mode active no shutdown interface Ethernet1/36 switchport switchport mode trunk channel-group 1 mode active no shutdown Eth1/34 -- eth routed up none 40G(D) -- Eth1/35 1 eth trunk up none 40G(D) 1 Eth1/36 1 eth trunk up none 40G(D) 1 #### --------------- #### N9K-2# sh ver Software BIOS: version 05.51 NXOS: version 10.3(6) [Maintenance Release] BIOS compile time: 11/29/2023 NXOS image file is: bootflash:///nxos64-cs.10.3.6.M.bin NXOS compile time: 8/20/2024 12:00:00 [08/20/2024 19:31:13] Hardware cisco Nexus9000 C9336C-FX2 Chassis Intel(R) Xeon(R) CPU D-1526 @ 1.80GHz with 24545588 kB of memory. Processor Board ID FDO2148050L Device name: N9K-2 bootflash: 25006080 kB Kernel uptime is 0 day(s), 0 hour(s), 10 minute(s), 49 second(s) Last reset at 545181 usecs after Mon Mar 3 15:24:45 2025 Reason: Reset due to upgrade System version: 9.3(13) Service: plugin Core Plugin, Ethernet Plugin Active Package(s): N9K-2# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 1 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : secondary Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled Delay-restore status : Timer is off.(timeout = 30s) Delay-restore SVI status : Timer is off.(timeout = 10s) Delay-restore Orphan-port status : Timer is off.(timeout = 0s) Operational Layer3 Peer-router : Disabled Virtual-peerlink mode : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ ------------------------------------------------- 1 Po1 up 1 N9K-2# show int eth1/34-36 transceiver | i type type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M type is QSFP-H40G-AOC5M version 10.3(6) Bios:version 05.51 interface Ethernet1/34 vrf member vPC-keepalive ip address 192.168.10.2/30 no shutdown interface Ethernet1/35 switchport switchport mode trunk channel-group 1 mode active no shutdown interface Ethernet1/36 switchport switchport mode trunk channel-group 1 mode active no shutdown Eth1/34 -- eth routed up none 40G(D) -- Eth1/35 1 eth trunk up none 40G(D) 1 Eth1/36 1 eth trunk down Link not connected auto(D) 1 <<<<<<<<< ***** Happen issue: N9K-1 Eth1/34 -- eth routed up none 40G(D) -- Eth1/35 1 eth trunk up none 40G(D) 1 Eth1/36 1 eth trunk down Link not connected auto(D) 1 <<<<<<<<<<<<< N9K-2 Eth1/34 -- eth routed up none 40G(D) -- Eth1/35 1 eth trunk up none 40G(D) 1 Eth1/36 1 eth trunk down Link not connected auto(D) 1 <<<<<<<<<<<<<<< **** Need to config speed 40000 both switch's interface eth1/36 or reload after up . interface eth1/36 speed 40000 N9K-1 Eth1/34 -- eth routed up none 40G(D) -- Eth1/35 1 eth trunk up none 40G(D) 1 Eth1/36 1 eth trunk up none 40G(D) 1 N9K-2 Eth1/34 -- eth routed up none 40G(D) -- Eth1/35 1 eth trunk up none 40G(D) 1 Eth1/36 1 eth trunk up none 40G(D) 1
Cisco Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.