Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
On an HPE ProLiant Server or HPE Synergy Compute Module running VMware ESXi 6.7 or VMware ESXi 7.0 and configured with any of the HPE QLogic NX2-based network adapters with VMware ESXi 6.7 qfle3 driver version 1.1.14.2 (or earlier) or VMware ESXi 7.0 qfle3 driver version 1.4.12.0 (or earlier) listed in the Scope section below, a Purple Screen of Death (PSOD) may occur when there is a vmkernel delay and the qfle3 driver times out. The driver times out when the VMware kernel is not responding within the expected timeframe of the qfle3 driver. Some delays in the VMware kernel are not easily recognizable when looking at the logs, but some delays are easier to detect as in a pcpu locked up and heartbeat delays. Example of the Purple Screen of Death (PSOD) screen: #PF Exception 14 in world 2098118:vmnic3-pollW IP 0x41802ac2e572 addr 0x28 Code start: 0x41802aa00000 VMK uptime: 26:04:52:03.865 PktList_SplitBySrcPort@vmkernel#nover+0x6 stack: 0x100000000 PktList_DoIOComplete@vmkernel#nover+0x39 stack: 0x1 PktListIOCompleteInt@vmkernel#nover+0x165 stack: 0x0 Net_AcceptRxList@vmkernel#nover+0x6e stack: 0x4305c3799a00 NetPollWorldCallback@vmkernel#nover+0xb6 stack: 0x15 CpuSched_StartWorld@vmkernel#nover+0x77 stack: 0x0 base fs=0x0 gs=0x418040000000 Kgs=0x0 Example of a pcpu locked up and heartbeat delays: Note: The pcpu locked up creates a kernel delay and eventually the qfle3 driver times out. This occurs when the Operating System is under extreme stress. The following logs show 8 seconds, 21 seconds and 49 seconds heartbeat delays. qfle3: ecore_queue_state_change:4966: [vmnic0] +++ Ecore Wait Complete Timed out, manually polling WARNING: Heartbeat: 760: PCPU 63 didn't have a heartbeat for 8 seconds; *may* be locked up. ALERT: NMI: 696: NMI IPI: RIPOFF(base):RBP:CS [0xaa2815(0x418022c00000):0x430fbb7a0940:0xfc8] (Src 0x1, CPU63) ecore_queue_state_change@(qfle3)#<None>+0x271 stack: 0x4 qfle3_remove_queue_filter@(qfle3)#<None>+0x1b0 stack: 0x451b29fa3000 UplinkNetq_RemoveHwRxFilter@vmkernel#nover+0x7e stack: 0x451b00000016 NetqueueBalDeactivatePendingFilters@vmkernel#nover+0x1bd stack: 0x451b0a61bcf8 NetqueueBalRxQueueCommitChanges@vmkernel#nover+0x49 stack: 0x0 UplinkNetqueueBal_BalanceCB@vmkernel#nover+0x19fc stack: 0x4307072a69a0 UplinkAsyncProcessCallsHelperCB@vmkernel#nover+0x116 stack: 0x43089557a570 HelperQueueFunc@vmkernel#nover+0x157 stack: 0x43089557a0b8 CpuSched_StartWorld@vmkernel#nover+0x77 stack: 0x0 WARNING: Heartbeat: 760: PCPU 63 didn't have a heartbeat for 21 seconds; *may* be locked up. . . . WARNING: Heartbeat: 760: PCPU 63 didn't have a heartbeat for 49 seconds; *may* be locked up.
Any HPE ProLiant Server or HPE Synergy Compute Module running VMware ESXi 6.7 or VMware ESXi 7.0 and configured with any of the following HPE QLogic NX2 based network adapters with VMware ESXi 6.7 qfle3 driver version 1.1.14.2 (or earlier)or VMware ESXi 7.0 qfle3 driver version 1.4.12.0 (or earlier): HPE QLogic NX2 Based Network Adapters for HPE ProLiant Rack and Tower servers HPE Ethernet 10Gb 2-port BASE-T 57810S Adapter (656596-B21) HPE Ethernet 10Gb 2-port SFP+ 57810S Adapter (652503-B21) HPE FlexFabric 10Gb 4-port FLR-T 57840S Adapter (764302-B21) HPE FlexFabric 10Gb 2-port FLR-SFP+ 57810S Adapter (700751-B21) HPE FlexFabric 10Gb 2-port FLR-T 57810S Adapter (700759-B21) HPE CN1100R Dual Port Converged Network Adapter (QW990A) HPE CN1100R 10GBASE-T Dual Port Converged Network Adapter (N3U52A) HPE QLogic NX2 Based Network Adapters for HPE ProLiant Server Blades (c-Class BladeSystems) HPE FlexFabric 20Gb 2-port 630FLB Adapter (700065-B21) HPE FlexFabric 20Gb 2-port 630M Adapter (700076-B21) HPE FlexFabric 10Gb 2-port 536FLB Adapter (766490-B21) HPE FlexFabric 10Gb 2-port 534M Adapter (700748-B21) HPE QLogic NX2 Based Network Adapters for HPE Synergy Compute Modules HPE Synergy 2820C 10Gb Converged Network Adapter (794538-B21) HPE Synergy 3820C 10/20Gb Converged Network Adapter (777430-B21)
To resolve this issue, download and install the following driver and firmware components: HPE ProLiant Rack and Tower servers HPE QLogic NX2 10/20 GbE Multifunction Driver for VMware vSphere 6.7 Version 2021.09.04 HPE QLogic NX2 10/20 GbE Multifunction Driver for VMware vSphere 7.0 Version 2021.09.04 HPE QLogic NX2 Online Firmware Upgrade Utility for VMware Version 1.29.3 For HPE ProLiant Server Blades (c-Class BladeSystems) HPE Blade QLogic NX2 10/20 GbE Multifunction Driver for VMware vSphere 6.7 Version 2021.09.01 HPE Blade QLogic NX2 10/20 GbE Multifunction Driver for VMware vSphere 7.0 Version 2021.09.01 HPE Blade QLogic NX2 Online Firmware Upgrade Utility for VMware Version 1.5.2 For HPE Synergy Compute Modules HPE Synergy QLogic NX2 10/20 GbE Multifunction Driver for VMware vSphere 6.7 Version 2021.09.01 HPE Synergy QLogic NX2 10/20 GbE Multifunction Driver for VMware vSphere 7.0 Version 2021.09.01 HPE Synergy QLogic NX2 Online Firmware Upgrade Utility for VMware Version 1.6.4 RECEIVE PROACTIVE UPDATES : Receive support alerts (such as Customer Advisories), as well as updates on drivers, software, firmware, and customer replaceable components, proactively in your e-mail through HPE Support Alerts. Sign up for Support Alerts at the following URL: Proactive Updates Subscription Form. NAVIGATION TIP: For hints on navigating HPE.com to locate the latest drivers, patches and other support software downloads, refer to the Navigation Tips document. SEARCH TIP: For hints on locating similar documents on HPE.com, refer to the Search Tips document.