Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
Certain NVMe drive models with firmware version MPK77H5Q, MPK7725Q or HPK5 may cause an Uncorrectable Machine Check Exception (UMCE) to occur. An uncorrectable PCIe bus error may also be logged in the case of Intel-based servers. These errors will be logged to the HPE Integrated Management Log (IML) and will cause the operating system to crash. The error may also cause an unexpected server reboot event. Example of UMCE in the IML from AMD-based server: Uncorrectable Machine Check Exception (Processor 1, APIC ID 0x0000003B, Bank 0x00000005, Status 0xBEA00000'00000108, Address 0x0101FFFF'B9137993, Misc 0xD01C0000'00000000). ACTION: Update the system firmware. If the issue persists, contact support. Example of UMCE and PCIe bus error in the IML from Intel-based server: Critical,1240,172348,0x0005,CPU,0x0003,Hardware,06/12/2024 12:05:43,40118: Uncorrectable Machine Check Exception (Processor 2, APIC ID 0x00000020, Bank 0x00000006, Status 0xBB800000'00000E0B, Address 0x00000000'00000000, Misc 0x00000000'AE100000). ACTION: Update the system firmware. If the issue persists, contact support. Critical,1240,172355,0x0014,System Error,0x0005,Hardware,06/12/2024 12:05:43,40119: Unrecoverable I/O Error has occurred. System Firmware will log additional details in a separate IML message entry if possible. Critical,1240,172360,0x0008,PCI Bus,0x0003,Hardware,06/12/2024 12:05:46,40120: Uncorrectable PCI Express Error Detected. Embedded [UNKNOWN] (Segment 0x0, Bus 0xAE, Device 0x2, Function 0x0). Uncorrectable Error Status: 0x00004000 ACTION: Update the firmware of the device. If the issue persists, replace the failing device.
Any of the HPE Gen9/Gen10/Gen10 Plus/Gen11 servers in the "Products Affected" list that support the following NVMe Solid State Drives "MZXL5," "MZXLR," "VO" and "MO" models: Models beginning with "MZXL5" Model Numbers Descriptions Part Numbers MZXL5800HBHQ-000H3 HPE 800GB NVMe x4 MU SFF SC U.3 SSD P26543-B21 MZXL5960HBHQ-000H3 HPE 960GB NVMe x4 RI SFF SC U.3 SSD P26538-B21 MZXL51T6HBJR-000H3 HPE 1.6TB NVMe x4 MU SFF SCN U.3 SPL SSD P56239-B21 MZXL51T9HBJR-000H3 HPE 1.92TB NVMe x4 RI SFF SC U.3 DS SSD P16501-B21 MZXL53T2HBLS-000H3 HPE 3.2TB NVMe x4 MU SFF SC U.3 DS SSD P16499-B21 MZXL53T8HBLS-000H3 HPE 3.84TB NVMe x4 RI SFF SCN U.3 SSD P22278-B21 MZXL56T4HALA-000H3 HPE 6.4TB NVMe x4 MU SFF SCN U.3 SSD P22272-B21 MZXL57T6HALA-000H3 HPE 7.68TB NVMe x4 RI SFF SCN U.3 SSD P22280-B21 MZXL512THALA-000H3 HPE 12.8TB NVMe x4 MU SFF SC U.3 SSD P26129-B21 MZXL515THALA-000H3 HPE 15.36TB NVMe x4 RI SFF SCN U.3 SSD P22282-B21 Models beginning with "MZXLR" Model Numbers Descriptions Part Numbers MZXLR800HBHQ-000H3 HPE 800GB NVMe x4 MU SFF SC U.3 SSD P26543-B21 MZXLR960HBHQ-000H3 HPE 960GB NVMe RI SFF BC U.3 PM1733 SSD P40564-B21 MZXLR1T6HBJR-000H3 HPE 1.6TB NVMe MU SFF BC U.3 PM1735 SSD P40570-B21 MZXLR1T9HBJR-000H3 HPE 1.92TB NVMe x4 RI SFF SC U.3 DS SSD P16501-B21 MZXLR3T2HBLS-000H3 HPE 3.2TB NVMe x4 MU SFF SCN U.3 SSD P22270-B21 MZXLR3T8HBLS-000H3 HPE 3.84TB NVMe x4 RI SFF SCN U.3 SSD P22278-B21 MZXLR6T4HALA-000H3 HPE 6.4TB NVMe x4 MU SFF SC U.3 SSD P26124-B21 MZXLR7T6HALA-000H3 HPE 7.68TB NVMe RI BC U.3 PM1733 SSD P40567-B21 MZXLR12THALA-000H3 HPE 12.8TB NVMe x4 MU SFF SC U.3 Spl SSD P48209-B21 MZXLR15THALA-000H3 HPE 15.36TB NVMe x4 RI SFF SCN U.3 SSD P22282-B21 Models beginning with "VO" Model Numbers Descriptions Part Numbers VO001920KYDMT HPE 1.92TB NVMe RI SCN U.3 PM1733a SSD HPE 1.92TB NVMe RI SC U.3 PM1733a SSD HPE 1.92TB NVMe RI BC U.3 PM1733a SSD P50214-B21 P50215-B21 P50216-B21 VO003840KYDMV HPE 3.84TB NVMe RI SCN U.3 PM1733a SSD HPE 3.84TB NVMe RI SC U.3 PM1733a SSD HPE 3.84TB NVMe RI BC U.3 PM1733a SSD P50217-B21 P50218-B21 P50219-B21 VO007680KYDNA HPE 7.68TB NVMe RI SCN U.3 PM1733a SSD HPE 7.68TB NVMe RI SC U.3 PM1733a SSD HPE 7.68TB NVMe RI BC U.3 PM1733a SSD P50220-B21 P50221-B21 P50222-B21 VO015360KYDNB HPE 15.36TB NVMe RI SC U.3 PM1733a SSD HPE 15.36TB NVMe RI BC U.3 PM1733a SSD P50223-B21 P50224-B21 Models beginning with "MO" Model Numbers Descriptions Part Numbers MO001600KYDMU HPE 1.6TB NVMe MU SCN U.3 PM1735a SSD HPE 1.6TB NVMe MU SFF SC U.3 PM1735a SSD HPE 1.6TB NVMe MU SFF BC U.3 PM1735a SSD P50225-B21 P50225-B21 P50227-B21 MO003200KYDNC HPE 3.2TB NVMe MU SCN U.3 PM1735a SSD HPE 3.2TB NVMe MU SFF SC U.3 PM1735a SSD HPE 3.2TB NVMe MU SFF BC U.3 PM1735a SSD P50228-B21 P50229-B21 P50230-B21 MO006400KYDND HPE 6.4TB NVMe MU SCN U.3 PM1735a SSD HPE 6.4TB NVMe MU SFF SC U.3 PM1735a SSD HPE 6.4TB NVMe MU SFF BC U.3 PM1735a SSD P50231-B21 P50232-B21 P50233-B21
This issue is resolved in firmware versions MPK7825Q, MPK78H5Q, HPK6 (or later versions). Users should upgrade to the latest firmware version as soon as possible to avoid the failure described above. Upgrade Path Examples : MPK7725Q -> MPK7825Q MPK77H5Q -> MPK78H5Q HPK5 --> HPK6 Firmware update components can be downloaded from HPE Support Center by performing the following steps: To locate the firmware, enter the model number of the NVMe drive to be updated in the empty Search field and press the Search icon next to this field. In the list of results, find and select the firmware update smart component that matches the operating system installed on the server. Download and install the online flash components. Document Version Release Date Details 4 November 27, 2024 Updated the advisory with additional drive models and firmware resolution information. 3 October 22, 2024 Updated the Scope section with drive models, MZXL51T9HBJR-000H3 and MZXL53T8HBLS-000H3 2 September 12, 2024 Updated the Resolution section with the permanent fix. 1 August 15, 2024 Original Document Release.