Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
Symptoms ESXi 5.1 or later hosts with a large number of 32-bit or 64-bit DMA physical NICs and NetQueue enabled, run out of memory when using jumbo frames (MTU is 9000 bytes).The lack of memory for network packets leads to lost virtual machine connectivity and may also lose connection with the vCenter Server.Network performance of network card is substantially degraded.NFS datastores mounted and accessed through this card become unmounted or flap between connected and disconnected state.vMotions time out.Restarting host management agents fail to complete when they attempt to re-initialize.In the vmkernel.log file, located at /var/log/, you see the message: Could not allocate xxxx bytes for dynamic heap NetPktHeapLow. Request returned Out of memory Notes: For information about configuring the packet memory allocation for DMA-constrained NICs on ESXi 5.1, see Virtual machines running on ESXi 5.1 lose connectivity and network traffic stops if a large number of physical NIC e1000e ports are configured with a jumbo MTU setting of 9000 Bytes (2033170).To ensure that you are running a supported configuration, see vSphere 5.1 Configuration Maximums guide.For information about the maximum number of physical NICs that a host with enabled jumbo frames can have, see Configuration maximums for NIC ports on ESXi/ESX 4.x and ESXi 5.x (1020808).For information on out of heap memory in ESXi 5.0, see Out of heap memory in ESXi 5.0 with NetQueue and Jumbo Frames enabled (2042587).For out of heap information related to ESX/ESXi 4.x, see Failed vMotions and degraded network performance with Emulex OneConnect 10GbE adapter and be2net driver (2013966).
To resolve the out-of-memory issue, increase the size of the network packet buffer pool. ESXi 5.1 uses a buffer pool for network packets. By default, the maximum memory allocated for packets is 656 MB. ESXi 5.5 uses a buffer pool that is calculated on per-GB basis, so the maximum memory allocated for packets is relative to the amount of physical memory in the system. To increase the maximum size of the network packet buffer pool and heap on the host, you can use the vSphere Web Client, vSphere Client, vCLI, or PowerCLI commands. Note: A Reboot is required on the ESXi host for changes to take in effect. Modifying the net packet pool size from the vSphere Web Client To modify the net packet pool size from the vSphere Web Client: In the vSphere Web Client, navigate to the host in the vSphere inventory.Click the Manage tab.Select Advanced Settings under System.Change the advanced parameter applicable to your ESXi version. ESXi versionProcedure5.1 Set the VMkernel.Boot.netPktPoolMaxSize parameter to the target maximum pool size in MB, for example, to 800MB.Set the VMkernel.Boot.netPktHeapMaxSize parameter from 64 to 256 size in MB. 5.5 and later Set the VMkernel.Boot.netPktHeapMaxMBPerGB parameter to the target maximum pool size in MB, for example, to 512MB.Set the VMkernel.Boot.netPktPoolMaxMBPerGB parameter from 75 to 200 size in MB. Click OK.Reboot the ESXi host. Modifying the net packet pool size from the vSphere Client To modify the net packet pool size from the vSphere Client: In the vSphere Client, navigate to the host in the vSphere inventory.Click the Configuration Tab.Select Advanced Settings under Software.Scroll Down and expand VMkernel.Select Boot.Change the advanced parameter applicable to your ESXi version. ESXi versionProcedure5.1 Set the VMkernel.Boot.netPktPoolMaxSize parameter to the target maximum pool size in MB, for example, to 800MB.Set the VMkernel.Boot.netPktHeapMaxSize parameter from 64 to 256 size in MB. 5.5 and later Set the VMkernel.Boot.netPktHeapMaxMBPerGB parameter to the target maximum pool size in MB, for example, to 512MB.Set the VMkernel.Boot.netPktPoolMaxMBPerGB parameter from 75 to 200 size in MB. Click OK.Reboot the ESXi host. Modifying the network packet heap and pool size on the ESXi host using vCLI commands To increase the size of the network packet buffer pool, set the netPktPoolMaxSize and netPktHeapMaxMBPerGB parameters by using the esxcli system settings kernel set command on the host: To set the advanced parameter applicable to your ESXi version: Run the commands applicable to your ESXi version: ESXi versionCommand5.1esxcli system settings kernel set -s netPktHeapMaxSize -v 256 esxcli system settings kernel set -s netPktPoolMaxSize -v 8005.5 and lateresxcli system settings kernel set -s netPktHeapMaxMBPerGB -v 512 esxcli system settings kernel set -s netPktPoolMaxMBPerGB -v 200 Run the reboot command to reboot the ESXi host or reboot from the UI.Run these commands and verify that the settings are persistent after reboot: ESXi versionCommand5.1esxcli system settings kernel list | grep netPktHeapMaxSize esxcli system settings kernel list | grep netPktPoolMaxSize5.5 and lateresxcli system settings kernel list | grep netPktHeapMaxMBPerGB esxcli system settings kernel list | grep netPktPoolMaxMBPerGB Modifying the network packet heap and pool size using vSphere PowerCLI To set the network packet heap and pool size: From a PowerCLI shell, run the commands applicable to your ESXi version: Connect-VIServer [IP_or_FQDN_of_Host] ESXi versionCommand5.1Set-VMHostAdvancedConfiguration -VMHost IP_or_FQDN_of_Host -Name VMkernel.Boot.netPktHeapMaxSize -Value 256 Set-VMHostAdvancedConfiguration -VMHost IP_or_FQDN_of_Host -Name VMkernel.Boot.netPktPoolMaxSize -Value 8005.5 and laterSet-VMHostAdvancedConfiguration -VMHost IP_or_FQDN_of_Host -Name VMkernel.Boot.netPktHeapMaxMBPerGB -Value 512 Set-VMHostAdvancedConfiguration -VMHost IP_or_FQDN_of_Host -Name VMkernel.Boot.netPktPoolMaxMBPerGB -Value 200 Run the reboot command to reboot the ESXi host or reboot from the UI.Run the commands applicable to your ESXi version to verify that the settings are persistent after reboot: ESXi versionCommand5.1Get-VMHostAdvancedConfiguration -VMHost IP_or_FQDN_of_Host -Name VMkernel.Boot.netPktHeapMaxSize Get-VMHostAdvancedConfiguration -VMHost IP_or_FQDN_of_Host -Name VMkernel.Boot.netPktPoolMaxSize5.5 and laterGet-VMHostAdvancedConfiguration -VMHost IP_or_FQDN_of_Host -Name VMkernel.Boot.netPktHeapMaxMBPerGB Get-VMHostAdvancedConfiguration -VMHost IP_or_FQDN_of_Host -Name VMkernel.Boot.netPktPoolMaxMBPerGB For more information, see the VMware PowerCLI Documentation.