OPERATIONAL DEFECT DATABASE
...
...
Version Release Date Details 2 06/30/2021 Added a note to the Resolution section. 1 06/22/2021 Original Document Release. An issue has been found in M-series environments that utilize Multi-Chassis Link Aggregation (MLAG) where switches in an MLAG pair remain in a peering state for an extended period of time after being rebooted or physically power-cycled. The source of the issue is a default MTU change to 9216 bytes included in ONYX-3.9.2110 M-Series switch firmware. This issue is exacerbated after upgrading from an earlier version than ONYX-3.9.2110 to ONYX-3.9.2110; however later versions of ONYX have corrected this issue. The original intent was to change the default L2/L3 value of all switch ports from the MTU default 1500 bytes to the maximum allowed value of 9216 bytes to allow for better performance in the event of data flow through the IPL. It has been found that a conflict affecting MLAG operation arises when there is a MTU 1500 force configuration statement visible in the running configuration file. The MTU of the port channel and ports of the inter-peer link (IPL) will not automatically be configured to 9216 bytes if they are configured with the force command prior to the upgrade. This will lead to an inconsistency between the ports and the port channel and prevent the MLAG pair from synchronizing, which will furthermore remove any hardware redundancy in the event of a failure.
The following HPE Storage Networking M-Series ONYX switches are affected by this issue: HPE M-series SN2010M Switch Q9E63A HPE M-series SH2200M Switch for Synergy HPE M-series SN2100M Switch Q2F23A and Q2F24A HPE M-series SN2410M Switch Q2F22A and Q6M27A HPE M-series SN2410bM Switch Q6M28A and Q6M29A HPE M-series SN2700M Switch Q2F21A and Q6M26A HPE M-series SN3700M and SN3700cM Switch R5Z74A and R3B14A HPEM-series SN4600cM Switch R6R24A
The way to pinpoint an IPL MTU mismatch is to verify whether packets on the IPL ports are getting discarded for any reason. In the following example, ports 30 and 31 are part of port-channel 1 (Po1) which is assigned to the IPL: Run these CLI commands on both switches in the MLAG pair. SN3700C-ONYX-ODD1 [vip-4243: master] (config) # show interfaces port-channel summary Flags D: Down U: Up P: Up in port-channel (members) S: Suspend in port-channel (members) I: Individual ------------------------------------------------------------------------ Group Port-Channel Type Member Ports ------------------------------------------------------------------------ 1 Po1(U) LACP Eth1/30(P) Eth1/31(P) 2 Po3(U) LACP Eth1/11(P) Eth1/12(P)Eth1/13(P) Eth1/14(P) HPE now initiates a command to verify the number of discards seen and the reasons. A correctly configured MLAG IPL should not be experiencing large numbers of discards under any circumstance. SN3700C-ONYX-ODD1 [vip-4243: master](config) # show interfaces ethernet 1/30 counters discard Interface Eth1/30: Rx: 50000 discard packets 22110 error packets 14211 fcs errors 0 undersize packets 12002 oversize packets Tx: 47833 discard packets 20211 error packets 0 hoq discard packets 14321 oversize packets When HPE examines the system configuration with the show running-config command and compare it to the output of show interfaces port-channel 1, then the source of the MTU mismatch can be found: ## Interface Ethernet configuration ## interface ethernet 1/30-1/31 mtu 1500 force interface ethernet 1/30-1/31channel-group 1 mode active SN3700C-ONYX-ODD1 [vip-4243: master] (config) # show interfaces port-channel 1 Po1: Admin state : Enabled Operational state : Up Description : IPL to other switch Mac address : N/A MTU : 9216 bytes (Maximum packet size 9238 bytes) The ports comprising Port Channel 1 (1/30-1/31) are force set to MTU 1500 while the port channel itself is set to the default MTU value of 9216, which means that the maximum MTU is going to be the lower of the two. If you have not upgraded to 3.9.2110 then this issue can be prevented by making the following changes before upgrading. If you are currently running ONYX 3.9.2110 and experience an issue where the two switches in the MLAG pair is stuck in a peering state, then you will need to do the following to resolve the issue. If you are not comfortable performing this recommended action for the production environment, then HPE recommends that you contact HPE Customer Support and reference this Customer Advisory. The first step is to identify the master switch in the MLAG configuration, which is the switch with a higher IPL port-channel IP address (1.1.1.2 in this case): SN3700C-ONYX-ODD1 [vip-4243: master](config) # show mlag The switch with the lower IPL port-channel IP address (in this case the IP address is 1.1.1.1) is the "standby" switch and should be modified first. Please note that DHCI configurations should have their existing network uplink ports on the standby switch disabled first to prevent any loops before moving forward with the following set of commands. Disable the IPL ports between the two switches (1/30 and 1/31 in this case) from the standby switch: interface ethernet 1/30-1/31 shutdown Next remove the IPL ports from the port channel: no interface ethernet 1/30-1/31 channel-group Change the MTU of the IPL ports to be 9216: interface ethernet 1/30-1/31 mtu 9216 If your existing port channel MTU is not set to 9216 then you can do that as well: interface port-channel 1 mtu 9216 Ensure that your VLAN interface is also set to 9216: interface vlan 3000 mtu 9216 Finally add the ports back to the IPL: interface ethernet 1/30-1/31 channel-group 1 mode active interface port-channel 1 ipl 1 Repeat the above commands on the master switch. Once the IPL ports, the IPL port channels, and VLAN interfaces are confirmed to be set to 9216 by the commands below, then you can re-enable the IPL ports on the standby switch and write the changes to memory on both switches. show interface ethernet 1/30-1/31 show interfaces port-channel 1 show interfaces vlan 3000 interface ethernet 1/30-1/31 no shutdown write memory Issue a show mlag command on both switches to ensure that the members are in an "UP" state: NOTE : If the MLAG status does not display "UP," reboot one switch at a time and check the MLAG status again. RECEIVE PROACTIVE UPDATES : Receive support alerts (such as Customer Advisories), as well as updates on drivers, software, firmware, and customer replaceable components, proactively in your e-mail through HPE Support Alerts. Sign up for Support Alerts at the following URL: Proactive Updates Subscription Form.
Click on a version to see all relevant bugs
Hewlett Packard Enterprise Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.
