Symptoms
Affected products:PowerEdge servers with BIOS version 2.4.8 and Intel Xeon Scalable Processors (Skylake) or 2nd Generation Intel Xeon Scalable Processors (Cascade Lake)Summary:PowerEdge servers with BIOS version 2.4.8 and Intel Xeon Scalable Processors (Skylake) or 2nd Generation Intel Xeon Scalable Processors (Cascade Lake) installed, may encounter a server halt during POST at "Loading BIOS Drivers ..." when attempting to downgrade BIOS to an older 2.x.x BIOS version.Details:PowerEdge servers with BIOS version 2.4.8 and Intel Xeon Scalable Processors (Skylake) or 2nd Generation Intel Xeon Scalable Processors (Cascade Lake) installed, may encounter a server halt during POST at "Loading BIOS Drivers ..." when attempting to downgrade BIOS to an older 2.x.x BIOS version. In the iDRAC job queue, the BIOS downgrade process will show at ~10% complete.This issue does not occur in every configuration thus not every server will experience it.
NOTE: BIOS version 2.1.x or newer is required for PowerEdge servers with 2nd Generation Intel Xeon Scalable Processors (Cascade Lake). BIOS versions 1.x.x only supports Intel Xeon Scalable Processors (Skylake).
Once a server is in this state, neither a warm or a cold reset will recover the server.
Cause
PowerEdge servers with BIOS version 2.4.8 and Intel Xeon Scalable Processors (Skylake) or 2nd Generation Intel Xeon Scalable Processors (Cascade Lake) installed, may encounter a server halt during POST at "Loading BIOS Drivers ..." when attempting to downgrade BIOS to an older 2.x.x BIOS version.
Resolution
Resolution:Dell EMC Engineering is aware of the issue and has fixed this downgrade issue in BIOS release version 2.5.4 (or higher). A BIOS downgrade from 2.5.4 (or higher) to older 2.x.x BIOS versions will not encounter this issue.To recover a server that is already in this state as the result of attempting to downgrade BIOS, perform the following steps to restore the previously saved version of BIOS:
ssh to the server idrac and log in.Run racadm command and subcommands as shown below:
For example:
If run remotely (with the correct IP, user and passwords):
racadm -r 192.168.1.1 -u root -p abc123 recover BIOS.Setup.1-1
If run in idrac serial console:
racadm recover BIOS.Setup.1-1
This article will be updated as new information becomes available.