OPERATIONAL DEFECT DATABASE
...
...
Patching / Upgrading ESXi 7.0 may fail if you had NSX-V with VXLAN enabled on a vSphere Distributed Switch (VDS) of version 7.0 and migrated to NSX-T Data Center by using NSX V2T migration.vCenter Server UI will below error message while remediating the hosts using vCenter Server Lifecycle Manager (vLCM or VUM) An error occurred during host configuration.VMware vSphere Lifecycle Manager had an unknown error. Check the events and log files for details. vLCM / VUM logs will show below failure log entries : /var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log2021-12-17T17:30:42.851Z info vmware-vum-server[09882] [Originator@6876 sub=VciRemediateTask.RemediateTask{146}] [vciTaskBase 1372] SerializeToVimFault fault:--> (integrity.fault.VcIntegrityFault) {--> faultCause = (vmodl.MethodFault) null,--> faultMessage = <unset>--> msg = "Install error on host: test-esx01.vmware.com, error details: Platform Configuration Error: /usr/sbin/esxupdate returned with exit status: 32"--> }--> Converted fault:--> (vim.fault.ExtendedFault) {--> faultCause = (vmodl.MethodFault) null,--> faultMessage = <unset>,--> faultTypeId = "com.vmware.vcIntegrity.VcIntegrityFault",--> data = (vim.KeyValue) [--> (vim.KeyValue) {--> key = "faultCause",--> value = ""--> },--> (vim.KeyValue) {--> key = "faultMessage",--> value = ""--> }--> ]--> msg = "Install error on host: test-esx01.vmware.com, error details: Platform Configuration Error: /usr/sbin/esxupdate returned with exit status: 32"--> }2021-12-17T17:30:42.851Z info vmware-vum-server[09882] [Originator@6876 sub=VciRemediateTask.RemediateTask{146}] [vciTaskBase 1680] Description updated2021-12-17T17:30:42.855Z info vmware-vum-server[09882] [Originator@6876 sub=VciRemediateTask.RemediateTask{146}] [vciTaskBase 1390] VciTask { id: RemediateTask{146}, type: com.vmware.vcIntegrity.RemediateTask }: Setting VC task state to: error2021-12-17T17:30:42.866Z info vmware-vum-server[09882] [Originator@6876 sub=VciRemediateTask.RemediateTask{146}] [vciTaskBase 203] Delete VC task from database: task-152923
This issue is observed due to stale NSX-V entries in following scenario : Environment had NSX-V running with VXLAN feature enabled on VDS 7.0 and later migrated the system to NSX-T using NSX V2T migratorThis might leave some stale NSX-V properties in the VDS in some or all hosts and host upgrade fails.
This is a known issue, please follow the steps in workaround section to resolve this issue.
Follow below steps to workaround this issue : Download the CleanNSXV.py Python script attached to this KBUpload the script to path /tmp/ on vCenter Server Appliance Using WinSCP. Please refer KB Error when uploading files to vCenter Server Appliance using WinSCP if WinSCP fails to connect to VCSA.Login to VCSA with root credentials using any SSH Client (Eg. Putty)Run the CleanNSXV.py script as mentioned below : PYTHONPATH=$VMWARE_PYTHON_PATH python /tmp/CleanNSXV.py --user '<vc_admin_user>' --password '<passwd>'Note: The <vc_admin_user> parameter is a vCenter Server user with super administrative privileges and <passwd> parameter is the user password.For example:PYTHONPATH=$VMWARE_PYTHON_PATH python /tmp/CleanNSXV.py --user 'administrator@vsphere.local' --password 'Admin!23' Sample Results:Sample output of the script is below in which 2 hosts are listed:Namespace(password='Admin123', user='administrator@vsphere.local')Connecting to vCenter serviceConnected to vCenter serviceCleaning VDS 50 06 6f 76 ef 51 2e a9-36 3b f7 77 c8 e4 00 82 ...Begin to clean up NSXV properties on host[1x.xx.xx.xx]...NSXV properties get cleaned up on host[1x.xx.xx.xx].Begin to clean up NSXV properties on host[1x.xx.xx.xx]...NSXV properties get cleaned up on host[1x.xx.xx.xx].Cleaned VDS 50 06 6f 76 ef 51 2e a9-36 3b f7 77 c8 e4 00 82If no more VDS has the stale NSX-V properties, the output is as below:Namespace(password='Admin123', user='administrator@vsphere.local')No VDS needs clean up Verify if NSX-V properties "com.vmware.netoverlay.layer0" and "com.vmware.net.vxlan.udpport" are removed from the ESXi hosts: (Note: Select some random hosts to do the verification from the list of hosts showed in the script result from Step 4, no need to perform this test on all the hosts) Connect to an ESXi host by using an SSH client.Run the command 'net-dvs -l | grep "com.vmware.netoverlay.layer0\|com.vmware.net.vxlan.udpport"'.If you see no output, then the stale properties are removed.
Click on a version to see all relevant bugs
VMware Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.
