Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
Document Version Release Date Details 3 October 5, 2021 Updated Resolution, issue has been addressed with System ROM 2.54. 2 August 30, 2021 Updated Resolution, it is recommended to disable Extended Memory Test and added steps to accomplish this. 1 August 25, 2021 Original Document Release. Due to a firmware fault found in the Extended Memory Test algorithm after a System ROM update to version 2.50 and rebooting to take effect, HPE Intel-based Gen10 servers may suddenly experience unexpected Uncorrectable Machine Check Exception (UMCE) errors and memory may be called out. In addition, the memory may be mapped out because the uncorrectable memory error threshold is exceeded. When this occurs, the following error will be seen in the Integrated Management Log (IML): Uncorrectable Machine Check Exception (Processor X, APIC ID 0x000000xx, Bank 0x0000000x,... DIMM Failure - Uncorrectable Memory Error (Processor X, DIMM Y) ACTION: Replace Failed DIMM. Or be added to the following: Uncorrectable Memory Error Threshold Exceeded (Processor X, DIMM Y). The DIMM is mapped out and is currently not available. ACTION: Take corrective action for the failing DIMM. Re-map all DIMMs back into the memory map in RBSU. If the issue persists, contact support. For understanding the feature of Extended Memory Test options, please refer to the following link: Enabling or disabling Extended Memory Test Note: If there is no need to use the Extended Memory Test and it is disabled, the current v2.5x can continue to be used.
Any HPE Intel-based ProLiant, Synergy or Apollo Gen10 Servers.
This issue has been addressed with System ROM version 2.54, and the Extended Memory Test algorithm has returned to normal. To enable the Extended Memory Test and make the function work properly, HPE recommends that earlier versions of v2.5x or versions that plan to update from an older version to v2.5x be updated to v2.54 to avoid this issue. The new System ROM v2.54 can also be used to prevent further failures if the server encounters the issue after updating to System ROM v2.50. To download the System ROM 2.54, perform the following steps: 1.Click the following link: Hewlett Packard Enterprise Support Center . 2.Enter a product name (e.g., " DL380 Gen10") in the text search field and wait for a list of Suggested Products to display. From the Suggested Products list displayed, identify the desired product and select it. 3.The page should refresh to display the "DRIVERS AND SOFTWARE" tab and the components that support the selected product. From the "DRIVERS AND SOFTWARE" expandable filter menus on the left side of the page: -Select the Software Type "BIOS" -Select the Subtype "System ROM" -For further filtering if needed select the specific Operating System from the Operating Environment Type. 4.Locate and select the System ROM 2.54 (or later). Note: To ensure that you have selected the latest version of the firmware/driver, click the Revision History tab to check if a new version of the firmware/driver is available. For more important information, review the Release Notes tab. 5.Click Download. IMPORTANT : the DIMM is not faulty and there is no need for a replacement. If the DIMM was mapped out, use the Memory Remap option to remap the system memory that may be disabled due to a failure event, such as an uncorrectable memory error: From the System Utilities screen, select System Configuration > BIOS/Platform Configuration (RBSU) > Memory Options > Memory Remap. Select a setting > Remap All Memory - Makes all memory in the system available again on the next boot. Go back to System Configuration > BIOS/Platform Configuration (RBSU). Press Ctrl + A > Clear DIMM Mapout Records. Select a setting > Enabled - After reboot the value in this option and Step 2 are restored to default value. Save the settings and reboot the system to take effect. The workaround above can be implemented by the RESTful Interface tool for mass deployment: 1. Create a xxxx.json file with the following, and save the file in C:\Users\Administrator\Documents directory { "/redfish/v1/systems/1/bios/settings/": { "Attributes": { "MemoryRemap": "AllMemory", "ExtendedMemTest": "Disabled" } } , "/redfish/v1/systems/1/bios/service/settings/": { "Attributes": { "ClrDimmMapoutRecords": "Enabled" } } } 2. Install the RESTful Interface tool . 3. Click Windows logo > Hewlett Packard Enterprise >RESTful Interface Tool. 4. Type ilorest at the prompt, to get the iLOrest> C:\Users\Administrator\Documents>ilorest iLOrest : RESTful Interface Tool version 3.2.2 Copyright (c) 2014-2021 Hewlett Packard Enterprise Development LP -------------------------------------------------------------------------------- 5. Follow the example below to enter the process. iLOrest >login 15.100.XXX.XX -u Administrator -p password Discovering data...Done iLOrest >rawpost xxxx.json The operation completed successfully. The operation completed successfully. Note: Before executing the instruction, the xxxx.json file must be saved in the C:\Users\Administrator\Documents directory. User name will be the current logged in User Name, instead of Administrator. The above iLO IP address is an example, modify it accordingly. The settings of "MemoryRemap" and "ClrDimmMapoutRecords" will return back to the default value "No Action" and "Disabled" respectively after execution, regardless of whether there was DIMM mapped out already or not. 6. Reboot the server for configuration changes to take effect. RECEIVE PROACTIVE UPDATES : Receive support alerts (such as Customer Advisories), as well as updates on drivers, software, firmware, and customer replaceable components, proactively in your e-mail through HPE Support Alerts. Sign up for Support Alerts at the following URL: Proactive Updates Subscription Form. NAVIGATION TIP: For hints on navigating HPE.com to locate the latest drivers, patches and other support software downloads, refer to the Navigation Tips document. SEARCH TIP: For hints on locating similar documents on HPE.com, refer to the Search Tips document.