Loading...
Loading...
Option NVIDIA GPUs and PCIe network adapters may not communicate with the appropriate driver on an HPE Apollo 6500 Gen10 Plus system (XL675d and XL645d) running Ubuntu 20.04 LTS.After the Ubuntu 20.04 LTS installation, driver communication failure messages are displayed after the GPU/CUDA driver is installed. For example, the following message is displayed after the HPE NVIDIA driver installation:root@xl675d:~#nvidia-smiNVIDIA-SMI has failed because it couldn't communicate with the NVIDIAdriver. Make sure that the latest NVIDIA driver is installed and running.The NVIDIA driver is installed however, is unable to communicate with the GPUs properly.This occurs because the PCIe memory BAR for the GPUs is not assigned properly during OS initialization as follows:[ 22.345395] kernel: NVRM: This PCI I/O region assigned to your NVIDIAdevice is invalid:NVRM: BAR0 is 0M @ 0x0 (PCI:0000:48:00.0)[ 22.348443] kernel: NVRM: The system BIOS may have misconfigured your GPU.During boot, Ubuntu 20.04 attempts to re-allocate I/O and memory space to specific unused BAR registers that were not previously assigned by the BIOS. During this process, Ubuntu removes the memory range that the BIOS had previously allocated to the GPUs and NICs, thereby preventing the appropriate device driver from communicating with the endpoint (i.e. NIC or GPU).
Any HPE Apollo 6500 Gen10 Plus system (HPE ProLiant XL645d Gen10 Plus or HPE ProLiant XL675d Gen10 Plus), or HPE ProLiant DL385 Gen11 running Ubuntu 20.04 LTS configured with NVIDIA GPUs or PCIe network adapters.
To ensure option NVIDIA GPUs and PCIe network adapters communicate with the appropriate driver, add the kernel parameter at boot time: "pci=realloc=off" as shown below:Open /etc/default/grub using a text editor (for example, vi /etc/default/grub)After the "GRUB_CMDLINE_LINUX_DEFAULT=", add "pci=realloc=off" in quotation marks as shown below:Save and close the file, and run the command "update-grub" as follows:Reboot the system. The driver should be able to communicate with all GPUs on the next boot.Note: Red Hat Enterprise Linux and CentOS have this parameter set to "off" by default, whereas Ubuntu has this parameter set to "on" by default.Document VersionRelease DateDetails2July 24, 2025Added HPE ProLiant DL385 Gen11 as affected.1March 23, 2021Original Document Release.
Operating Systems Affected:Ubuntu 20.04 LTS
Click on a version to see all relevant bugs
Hewlett Packard Enterprise Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.