Loading...
Loading...
Windows VM with NSX Network Introspection driver (vnetflt.sys) connected to USVM (Guest Introspection SVM) loses temporary TCP network connectivity for new connections. Running dmesg command on the USVM console to show the logs, you see entries similar to: Out of memory: Kill process <process_id> (java) score <score> or sacrifice child In the NSX Manager log, you see entries similar to: Code:'260007' Event Message: 'Lost communication with ESX module.'
NSX Network Introspection driver (vnetflt.sys) is used to send Network related events to USVM through Multiplexor (MUX). Network events obtained in USVM is used in Activity Monitoring and Identity Firewall.The driver collects TCP connectivity event and push it to USVM. Since there is memory leak issue in the underlying connection between MUX and USVM, USVM event manager process is restarted due to out of memory. While the event manager process is restarting, TCP connecting event processing stays incomplete for a while and may result connectivity issue in Windows VM.
This issue is resolved in: VMware NSX for vSphere 6.2.7, available at VMware Downloads.VMware NSX for vSphere 6.3.0, available at VMware Downloads. To work around this issue if you do not want to upgrade, disable the NSX Network Introspection driver. To disable the vnetflt.sys driver: Note: This procedure modifies the Windows registry. Before making any registry modifications, ensure that you have a current and valid backup of the registry and the virtual machine. For more information on backing up and restoring the registry, see the Microsoft Knowledge Base article 136393. Connect to the affected virtual machine with a console or RDP sessions.Click Start > run, type regedit and click OK.Navigate to: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\vnetflt\Right-click the Start key and select Modify.Change the value to 4 and click OK.Close the Registry Editor Window.Reboot the virtual machine. Notes: If you are using Agentless AV only, disabling the NSX Network Introspection Driver does not affect Agentless AV functionality.If you have IDFW, use the Active Directory Event Log Scraper instead of USVM. For more information on the Event Log Scraper, see the Identity Firewall Overview section of the NSX Administration Guide.
You experience these additional symptoms:ESXi syslog contains entries similar to:2015-08-07T07:30:03Z EPSecMux[23051084]: [WARNING] (EPSEC) [0x15fbb4c] SolutionHandler[0x1f001cd8] failed to connect to solution[100] at [169.254.1.24:48655]: Connection refused (111) 2015-08-07T07:30:03Z EPSecMux[23051084]: [WARNING] (EPSEC) [0x15fbb4c] SolutionHandler[0x1f001cd8] scheduling reconnect to solution[100] at 169.254.1.24:48655 in 100 ms Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.VM network timeouts / packet drops after upgrading to VMware Tools 11.x with Guest Introspection Driver on ESXi 6.5/6.7
Click on a version to see all relevant bugs
VMware Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.