Loading...
Loading...
Windows Server 2022 (and newer) with Modern Standby-capable hardware can cause sporadic SDC disconnects and IO latency spikes (800-1100ms) on PowerFlex volumes. The root cause is Windows Modern Standby (S0 Low Power Idle / Connected Standby) periodically putting storage NICs into a low-power state, which drops all SDC-to-SDS TCP connections. Affected configurations: Physical Windows Server 2022+ hosts running the PowerFlex SDC (scini driver)Server hardware with UEFI S0 Low Power Idle (Modern Standby) supportNIC firmware that fully complies with Windows Modern Standby power transitions (e.g., Broadcom NetXtreme E-Series 25Gb with firmware 23.3x+) Symptoms Sporadic, unpredictable IO latency spikes (~1000ms) on PowerFlex volumes presented to Windows SDC hosts. Cannot be correlated with any specific application workload.Windows Event 153 (disk IO retry) in the System log: LOG: System 03/21/2026 09:44:07 AM Warning DESTURMZRELSQ01 153 disk N/A N/A The IO operation at logical block address 0x0 for Disk 16 (PDO name: \Device\00000094) was retried. Windows StorPort Event 549 with SCSI sense code (0x6, 0x29, 0x2) — "Power on, reset, or bus device reset occurred": LOG: StorageStorport Operational 03/21/2026 09:44:07 AM Error DESTURMZRELSQ01 549 Microsoft-Windows-StorPort Port N/A Storport Device (Port = 4, Path = 0, Target = 21, Lun = 21) The request opcode was 0x28 and completed with SrbStatus 0x4 and ScsiStatus 0x2. The sense code was (0x6,0x29,0x2). The io latency was 1006 ms. Windows Kernel-Power Event ID 172 indicating NIC compliance with standby power request: Connectivity state in standby: Disconnected, Reason: NIC compliance MDM events showing SDC disconnect/reconnect cycles, each lasting ~2 minutes: 2026-03-20T15:23:04.746 SDC_DISCONNECTED WARNING SDC disconnected. ID: 1f40c33e00000006 2026-03-20T15:25:21.972 SDC_CONNECTED INFO SDC connected. ID: 1f40c33e00000006; IP: 172.16.10.190 2026-03-20T15:25:22.736 SDC_CONNECTED INFO SDC connected. ID: 1f40c33e00000006; IP: 172.16.20.190,172.16.10.190 2026-03-20T15:44:03.746 SDC_DISCONNECTED WARNING SDC disconnected. ID: 1f40c33f00000007 2026-03-20T15:46:19.077 SDC_CONNECTED INFO SDC connected. ID: 1f40c33f00000007; IP: 172.16.10.191 2026-03-20T15:46:20.713 SDC_CONNECTED INFO SDC connected. ID: 1f40c33f00000007; IP: 172.16.20.191,172.16.10.191 2026-03-21T06:44:09.230 SDC_DISCONNECTED WARNING SDC disconnected. ID: 1f40c33e00000006 2026-03-21T06:46:23.125 SDC_CONNECTED INFO SDC connected. ID: 1f40c33e00000006; IP: 172.16.10.190 2026-03-21T07:20:03.266 SDC_DISCONNECTED WARNING SDC disconnected. ID: 1f40c33f00000007 2026-03-21T07:22:17.388 SDC_CONNECTED INFO SDC connected. ID: 1f40c33f00000007; IP: 172.16.10.191 Rapid disconnect storms may trigger PORT_FLAP detection on the MDM: 15:08:40 — SDC 0006 disconnects from ALL 15 SDS nodes simultaneously 15:08:58 — PORT_FLAP_COMP_BECAME_UNSTABLE (SDC 0006) 15:09:42 — SDC_SDS_MULTIPLE_DISCONNECTIONS WARNING 15:09:48 — PORT_FLAP_COMP_IN_TO_TME (SDC 0006 enters Too Many Events state) 15:10:08 — PORT_FLAP_COMP_OUT_OF_TME (recovery) 15:12:42 — SDC_SDS_ALL_CONNECTED 15:13:04 — PORT_FLAP_COMP_BECAME_STABLE Key Diagnostic Correlation The definitive indicator is a 1:1 match between MDM SDC disconnect events and Windows Kernel-Power Event 172 entries. In the reference case, every single disconnect correlated: MDM Event (local time) Windows Kernel-Power 172 Host 03/20 15:23:04 SDC_DISCONNECTED (0006) 03/20 15:25:15 NIC compliance disconnect SQ01 03/20 15:44:03 SDC_DISCONNECTED (0007) 03/20 15:46:13 NIC compliance disconnect SQ02 03/21 06:44:09 SDC_DISCONNECTED (0006) 03/21 06:46:17 NIC compliance disconnect SQ01 03/21 07:01:55 (0006 reconnect window) 03/21 07:01:55 NIC compliance disconnect SQ01 03/21 07:22:17 (0007 reconnect) 03/21 07:22:12 NIC compliance disconnect SQ02 03/21 07:32:56 (0007 reconnect) 03/21 07:32:52 NIC compliance disconnect SQ02 03/21 08:16:56 (0007 reconnect) 03/21 08:16:50 NIC compliance disconnect SQ02 03/21 10:28:20 (0007 disconnect) 03/21 10:30:38 NIC compliance disconnect SQ02 03/21 10:38:35 (0006 disconnect) 03/21 10:40:43 NIC compliance disconnect SQ01 Note: The Kernel-Power Event 172 fires when the NIC wakes up (exits standby), not when it enters standby. It consistently appears ~5-7 seconds before the MDM sees the SDC reconnect, or ~2 minutes after the initial disconnect.
What is Windows Modern Standby? Modern Standby (also called Connected Standby or S0 Low Power Idle) is a power management feature introduced in Windows 10/Server 2016+ that replaces traditional S3 sleep. Unlike S3, Modern Standby allows the system to enter low-power states while the OS appears to be running. The system transitions between active (S0) and idle (S0 Low Power Idle) states opportunistically based on system activity. Why does this affect PowerFlex? NIC enters low-power state: When Windows enters S0 Low Power Idle, it instructs compliant NIC firmware to enter a low-power ACPI D-state. Newer Broadcom firmware (23.3x+) fully implements S0ix compliance and honors this request, dropping link-level connectivity.All SDC-to-SDS TCP connections drop simultaneously: The PowerFlex SDC maintains persistent TCP connections to every SDS that holds data for its mapped volumes. When the NIC goes low-power, all connections on all VLANs drop at once — this is an OS-level event, not a network event.SDS detects stale connections: The SDS keepalive subsystem (kalive/) detects the dead TCP connections within seconds and marks the SDC as disconnected. The MDM logs SDC_DISCONNECTED.IO stalls during reconnect: When the NIC wakes up, the SDC must re-establish TCP connections, re-authenticate with each SDS, and re-sync mapping state with the MDM (generation numbers). Any IO in flight or issued during this window returns SCSI sense (0x6, 0x29, 0x2) with ~1000ms latency (the TCP reconnect + auth cycle time).Sporadic and unpredictable: Modern Standby enters idle opportunistically. Bursty workloads (like SQL) have quiet periods where Windows sees low activity and triggers the idle transition. The latency correlates with the absence of activity, not with any specific operation. Why BIOS settings don't prevent this Even with correct BIOS power settings (CPU Power Management = MaxPerf, C-States = Disabled, Energy Efficient Policy = MaxPower), Windows Modern Standby operates at the OS driver level, bypassing BIOS-level power management. The BIOS controls CPU/platform power states; Modern Standby controls device-level D-states through the Windows power framework. The NIC firmware factor Not all NIC firmware versions are equally affected: Broadcom 22.92.x (IC 4.5.4000 baseline): Does not fully implement S0ix compliance. The NIC may partially honor or ignore the low-power request, keeping link up.Broadcom 23.3x+: Full S0ix compliance. The NIC will honor the Windows power-down request and drop link. This explains why the problem may appear after a NIC firmware update even with no other changes to the environment. What this is NOT Not a PowerFlex software defect — No SDS panics, no comb state issues, no rebuild/rebalance activity. SDS infrastructure is healthy; the SDC host is intermittently dropping off.Not a network/switch issue — Both hosts on different switch ports lose connectivity simultaneously. Switch logs show no link flap events (the NIC enters low-power without physically bringing the link down at the PHY layer).Not a single NIC hardware fault — Identical pattern on multiple independent hosts.Not a StorPort/MPIO issue — StorPort errors are secondary to the transport loss.
Apply both of the following on each affected Windows SDC host. A reboot is required after Step 1. Step 1 — Disable Connected Standby via Registry reg add "HKLM\SYSTEM\CurrentControlSet\Control\Power" /v PlatformAoAcOverride /t REG_DWORD /d 0 /f This tells Windows the platform does not support Always On/Always Connected (AoAc), which prevents Modern Standby idle transitions entirely. Step 2 — Disable Network Adapter Power Management on Storage NICs Get-NetAdapter | Where-Object {$_.InterfaceDescription -like "*Broadcom*25Gb*"} | ForEach-Object { Disable-NetAdapterPowerManagement -Name $_.Name -Confirm:$false } Adjust the filter to match your specific storage NIC model. This prevents the NIC driver from honoring any residual power management requests. Step 3 — Verify High Performance Power Plan powercfg /setactive 8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c Step 4 — Reboot A reboot is required for the registry change (Step 1) to take effect. Alternative / Additional: NIC Firmware Downgrade If the above OS-level fixes cannot be applied, or as an additional layer of protection, downgrade the NIC firmware to the IC-validated baseline version (e.g., Broadcom 22.92.06.10 for IC 4.5.4000). Older firmware does not fully implement S0ix compliance and will not honor the power-down request. This can be done through PFMP compliance remediation or via iDRAC firmware update using the IC catalog DUP. The matching Windows NIC driver should also be rolled back to the IC-validated version. Verification After applying the fix, monitor for 1-2 weeks: scli --query_all_sdc — confirm persistent CONNECTED state for the affected SDC IDsCheck Windows System event log for absence of Event 172 (Kernel-Power) with "NIC compliance" textCheck Windows System event log for absence of Event 153 (disk IO retry)Check Windows StorageStorport Operational log for absence of Event 549 with sense code (0x6,0x29,0x2)Check MDM events for absence of SDC_DISCONNECTED / SDC_CONNECTED cycles on the affected SDC IDs Quick PowerShell check for current standby state # Check if Connected Standby is enabled (should show "Standby (S0 Low Power Idle)" if active) powercfg /a # Check current PlatformAoAcOverride value (0 = disabled, absent or 1 = enabled) Get-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\Power" -Name PlatformAoAcOverride -ErrorAction SilentlyContinue # Check NIC power management status Get-NetAdapter | Where-Object {$_.InterfaceDescription -like "*Broadcom*25Gb*"} | Get-NetAdapterPowerManagement Proactive Recommendation For all PowerFlex deployments with physical Windows Server 2022+ SDC hosts: Include the PlatformAoAcOverride=0 registry key and Disable-NetAdapterPowerManagement on storage NICs as part of the standard Windows SDC host preparation procedure.Ensure NIC firmware on Windows SDC hosts matches the IC-validated version. Non-IC firmware (especially Broadcom 23.3x+) introduces S0ix compliance behavior that can trigger this issue.Add this check to pre-deployment / health-check tooling: flag Windows hosts where Connected Standby is enabled and storage NICs have power management active. Applies To PowerFlex 4.x (all versions) with Windows SDCWindows Server 2022 and newerAny NIC with Modern Standby (S0ix) compliant firmware (Broadcom 23.3x+ confirmed; other vendors may also be affected)Physical servers with UEFI S0 Low Power Idle support (most modern Dell PowerEdge) Impacted Versions Windows OS firmware versions that enable this feature by default
Click on a version to see all relevant bugs
Dell Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.