OPERATIONAL DEFECT DATABASE
...
...
Between affected VMs the ping work.Connectivity issue with TCP traffic with Geneve overlay, such as when using NSX-T.When the VMs are running on two different hosts and the datapath involves the physical adapters, the issue arises.Packet captures from within the guest OS of the source VM show that the VM has received the Syn-Ack packet from the target VM. However, no ACK is generated on the source VM."sw encap csum error rx" can be observed in the NIC's private stats of ESXi at the receiving side, assuming these receiving esxi host also used the Broadcom NIC(bnxtnet): [rxq-drss61] sw encap csum error rx: 61964 < ------- [rxq-drss62] sw encap csum error rx: 63750 [rxq-drss63] sw encap csum error rx: 63454 Note: NIC private stats can be found in nicinfo.sh.txt in the ESXi support bundle, or by the the following command:localcli --plugin-dir /usr/lib/vmware/esxcli/int networkinternal nic privstats get -n <vmnicX>
The issue is caused by the NIC not computing the inner checksum for Geneve traffic when Geneve offload is enabled. The issue was introduced starting with firmware version 223.xx and is also present in versions 224.x & 225.x.
All TCP/IP traffic across the adapters is impacted.
The issue is fixed in firmware version 226.1.x and later. The customer should consult their OEM for the specific version number as it is different depending on the OEM and NIC model. And the matching bnxtnet driver version (according to the VGL or OEM) should be used.
Enable software simulation of Geneve Offload, which will takeover from hardware.The command is: esxcli network nic software set --vmnic=vmnicX --geneveoffload=1
Click on a version to see all relevant bugs
VMware Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.
