Loading...
Loading...
On any HPE Compute XD690 system configured with eight NVIDIA Blackwell Ultra (B300) GPUs and configured with NVIDIA ConnectX‑8 (CX8) network adapters; enabling ConnectX‑8 (CX8) InfiniBand mode may cause the system to appear to stop responding during the Power-On Self-Test (POST) for approximately 40 minutes before completing. During this time, the system may appear unresponsive even though Power-On Self-Test (POST) is still progressing.This condition is consistent and occurs under specific BIOS/Platform Configuration (RBSU) and specific network adapter configuration conditions.The extended Power-On Self-Test (POST) delay occurs due to an interaction between the BIOS DMA Protection, and UEFI PXE boot options enabled on CX8 InfiniBand network adapters. This combination causes the CX8 initialization routine to appear frozen during the system Power-On Self-Test (POST).
Any HPE Compute XD690 system configured with eight NVIDIA Blackwell Ultra (B300) GPUs, and configured with NVIDIA ConnectX‑8 (CX8) network adapters with CX8 UEFI PXE boot options enabled, and the BIOS/Platform Configuration (RBSU) with DMA Protection enabled.
NVIDIA has confirmed that no firmware, BIOS, or software fix will be provided for this condition.This is considered a permanent platform limitation when CX8 UEFI PXE boot is enabled.WorkaroundGo to the "BIOS menu -> Advanced -> Option ROM Dispatch Policy", and make sure to disable the Option ROM for CX8 devices.Note:When this is enabled, the BIOS Power-On Self-Test (POST) time up will be increased by approximately one hour.The following Redfish method can be used to check the Option ROMs for CX8 devices:Find the AttributeName names:curl -ksu $login:$password https://$bmc/redfish/v1/Registries/BiosAttributeRegistry3.en-US.1.6.0.json | gunzip | jq | grep -B 2 "CX8 OROM" | grep AttributeName"AttributeName": "OROM999","AttributeName": "OROM998","AttributeName": "OROM996","AttributeName": "OROM995","AttributeName": "OROM994","AttributeName": "OROM993","AttributeName": "OROM991","AttributeName": "OROM990",Check the current BIOS settings:# curl -ksu $login:$password https://$bmc/redfish/v1/Systems/DGX/Bios | jq | egrep "OROM990|OROM991|OROM993|OROM994|OROM995|OROM996|OROM998|OROM999" | grep -v MAP"OROM990": "Disabled","OROM991": "Disabled","OROM993": "Disabled","OROM994": "Disabled","OROM995": "Disabled","OROM996": "Disabled","OROM998": "Disabled","OROM999": "Disabled",When the Option ROMs are enabled, use the following curl command and reboot the node for the new BIOS settings:curl -ksu $login:$password -H 'content-type: application/json' -H 'If-Match: *' -X PATCH https://$bmc/redfish/v1/Systems/DGX/Bios/SD -d '{"Attributes":{"OROM990": "Disabled","OROM991": "Disabled","OROM993": "Disabled","OROM994": "Disabled","OROM995": "Disabled","OROM996": "Disabled","OROM998": "Disabled","OROM999": "Disabled"}}' | jqRecommended actions:Disable PXE boot on all CX8 InfiniBand network adapters.Review provisioning workflows that formerly used CX8 Unified Extensible Firmware Interface (UEFI) Preboot Execution Environment (PXE).Update the automation and build procedures to enforce the corrected configuration.Ensure Redfish integrations use the correct GPU power URI paths.
Operating Systems Affected:Not Applicable
Click on a version to see all relevant bugs
Hewlett Packard Enterprise Integration
Learn more about where this data comes from
BugZero Plan
Streamline upgrades with automated vendor bug scrubs
BugZero Prevent
Wish you caught this bug sooner? Get proactive today.