Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
Document Version Release Date Details 7 September 28, 2021 Added link to Red Hat KB article 6 August 13, 2021 Updated Linux information in this document 5 June 8, 2021 Added link to permanent fix and additional information under Resolution section. 4 June 2, 2021 Added additional impacted VMware ESXi Operating Systems and corresponding OS information/links. 3 February 23, 2021 Updated document with the latest information 2 December 11, 2021 Updated with SUSE Resolution 1 December 8, 2020 Original document release HPE Superdome Flex servers with some Cascade Lake processors (Intel Xeon Scalable processors 62XX/82XX) or with some Skylake processors (Intel Xeon Scalable processors 61XX/81XX) may stop responding or fail to boot when running with OS versions containing certain Intel microcode versions. Note : Another rare but possible side effect of this issue is the random de-configuration of processors and/or processor cores. The faulty microcode (02006a08 for Skylake, 04003003 or 05003003 for Cascade Lake) is found in recent SUSE and RHEL microcode-intel updates as well as some recent Windows and VMware ESXi KB updates. Systems with the following processors may encounter the issue: Note: "New Version" indicates microcode versions with the issue that should not be used. "Old Version" indicates microcode versions without the issue and OK to use How to check the microcode version: On the serial console or /var/log/messages, look for "microcode:" lines. Examples: [ 0.000000] microcode: microcode updated early to revision 0x5003003, date = 2020-06-18 [ 35.012898] microcode: sig=0x50657, pf=0x80, revision= 0x5003003
HPE Superdome Flex servers with the following processors may encounter the issue: Note: "New Version" indicates microcode versions with the issue that should not be used. "Old Version" indicates microcode versions without the issue and OK to use On Linux, the issue currently exists when running with the following OS versions: On SLES, RPMs with the microcode issue are: sles15sp2: ucode-intel-20201110-2.10.1.x86_64, ucode-intel-20201118-2.13.1.x86_64 sles12sp5: ucode-intel-20201110-3.23.1.x86_64, ucode-intel-20201118-2.13.1.x86_64 On RHEL, RHEL RPMs with the microcode issue are: microcode_ctl-2.1-73.2.el7_9.x86_64.rpm microcode_ctl-20200609-2.el8.x86_64.rpm On Windows, the issue exists when running with the following KBs: KB4589210 on Windows Server 2016: https://support.microsoft.com/en-us/help/4589210/intel-microcode-updates KB4589208 on Windows Server 2019: https://support.microsoft.com/en-us/help/4589208/intel-microcode-updates On VMware, the issue exists with the following versions (see KB83917 for details): -ESXi 6.7p05 (ESXi670-202103001) -ESXi 7.0 Update 2 -ESXi 7.0 Update 2a
The issue is fixed in server firmware version 3.30.140 (available at this link ) as that firmware contains newer microcode. Possible workaround if the firmware cannot be updated: Linux: If a system has the faulty microcode installed and experiences this issue, adding dis_ucode_ldr to the kernel command line via grub will allow the system to boot using the older microcode provided by BIOS. NOTE: An alternative is to update to HPE Foundation Software version 2.4.1 at this link as it implements a new UV_DISABLE_MICROCODE_UPDATE sysconfig variable allowing to enable/disable automatic microcode updates; by default, loading is disabled. After the system boots, update the ucode-intel RPM to the below newly released version using zypper. The affected CPU/steppings will no longer be subject to the issues described in this advisory after this is done. NOTE: Even though the issue only affects the Linux versions listed under SCOPE, below is the list of all the known good microcode to be used for each Linux version supported on Superdome Flex servers (later versions are not affected): a. SLES known good microcode: SLES12 SP2: ucode-intel-20201118-13.84.1.x86_64.rpm SLES12 SP3: ucode-intel-20201118-13.84.1.x86_64.rpm SLES12 SP4: ucode-intel-20201118-13.84.1.x86_64.rpm SLES12 SP5: ucode-intel-20210216-3.32.1.x86_64.rpm SLES15 SP1: ucode-intel-20201118-3.45.1.x86_64.rpm SLES 15 SP2 : ucode-intel-20210216-2.19.1 Note: The rpms for SLES 12 SP2, SP3, and SP4 require additional entitlements to access (LTSS or you must be running SLES for SAP). See also SUSE related communication: https://www.suse.com/support/kb/doc/?id=000019800 b. RHEL, good known microcodes: (for a complete list, see RHEL article 5639591 ) rhel-7.2.z: microcode_ctl-2.1-12.36.el7_2 rhel-7.3.z: microcode_ctl-2.1-16.39.el7_3 rhel-7.4.z: microcode_ctl-2.1-22.38.el7_4 rhel-7.6.z: microcode_ctl-2.1-47.20.el7_6 rhel-7.7.z: microcode_ctl-2.1-53.15.el7_7 rhel-7.9.z: microcode_ctl-2.1-73.8.el7_9 rhel-8.0.z: microcode_ctl-20180807a-2.20210216.1.el8_0 rhel-8.1.0.z:microcode_ctl-20190618-1.20210216.1.el8_1 rhel-8.2.0.z: microcode_ctl-20191115-4.20210216.1.el8_2 rhel-8.3.0.z: microcode_ctl-20200609-2.20210216.1.el8_3 rhel-8.4: microcode_ctl-20210216-1.el8 Windows: If the microcode was installed from a KB, try booting in safe mode and remove the faulty microcode update package with dism. VMware: see workaround documented in KB83917 at https://kb.vmware.com/s/article/83917 RECEIVE PROACTIVE UPDATES : Receive support alerts (such as Customer Advisories), as well as updates on drivers, software, firmware, and customer replaceable components, proactively in your e-mail through HPE Subscriber's Choice. Sign up for Subscriber's Choice at the following URL: Proactive Updates Subscription Form. NAVIGATION TIP: For hints on navigating HPE.com to locate the latest drivers, patches and other support software downloads ,refer to the Navigation Tips document. SEARCH TIP: For hints on locating similar documents on HPE.com, refer to the Search Tips document.