OPERATIONAL DEFECT DATABASE
...
...
vCenter Server patching to 7.0 U3f (Build 20051473) fails with error message "Exception occured in postInstallHook" in VAMI page. Patching VCSA through CLI fails with error "An error occured while starting sts"Upgrading vCenter Server from 6.x to 7.0 U3f will fail with "A problem occurred while - Starting VMware Security Token Service" VCSA is joined to Active Directory domain and is currently using, or has used in the past, Integrated Windows Authentication for identity sourceLog file /var/log/vmware/applmgmt/Patchrunner.log will show similar to below entries : 2022-07-15T15:50:31.439Z ERROR vmware_b2b.patching.phases.patcher Patch hook Patch got unhandled exception.Traceback (most recent call last): File "/storage/seat/software-updatew2oofv0c/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 203, in patch _patchComponents(ctx, userData, statusAggregator.reportingQueue) File "/storage/seat/software-updatew2oofv0c/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 84, in _patchComponents _startDependentServices(c) File "/storage/seat/software-updatew2oofv0c/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 53, in _startDependentServices serviceManager.start(depService) super(VMwareServiceController, self).start(serviceName) File "/storage/seat/software-updatew2oofv0c/stage/scripts/patches/libs/sdk/service_manager.py", line 665, in start raise IllegalServiceOperation(errorText)service_manager.IllegalServiceOperation: Service cannot be started. Error: Error executing start on service sts. Details { "detail": [ { "id": "install.ciscommon.service.failstart", "translatable": "An error occurred while starting service '%(0)s'", "args": [ "sts" ], "localized": "An error occurred while starting service 'sts'" } STS runtime log file log/vmware/sso/sts-runtime.log.stderr will show below error message : Starting service process with pid: 38715.Picked up JAVA_TOOL_OPTIONS: -Xms32M -Xmx128M -Dcom.sun.org.apache.xml.internal.security.ignoreLineBreaks=true -Dorg.apache.xml.security.ignoreLineBreaks=truefree(): corrupted unsorted chunksLog file will show any of the memory corruption errors mentioned below : free(): corrupted unsorted chunksdouble free or corruption (!prev)corrupted size vs. prev_sizefree(): invalid next size (normal) /var/core directory will show core dump files like below : core.Thread-2.38715core.Thread-2.54064
This issue is caused due to memory corruption in Secure Token Service (vmware-stsd) when VCSA is joined to Active Directory and is currently using, or has used in the past, Integrated Windows Authentication for identity source.
This issue is resolved in VMware vCenter Server 7.0 U3g build 20150588, available at VMware Downloads.Note: vCenter Server needs to be reverted to a healthy state to apply this patch, applying a patch in already failed state is not feasible.CVE-2021-22048 stands as un-resolved in 7.0 U3g as well, please refer to VMSA-2021-0025 for more information.
To workaround this issue, identity source configuration needs to be changed from IWA to "AD over LDAP" or "AD over LDAPs" and remove the VCSA from AD domain.You may follow below steps to change the identity source configuration : Revert/restore vCenter Server to a healthy state, which was taken before initiating the patchRemove the IWA Identity Source configurationRemove vCenter Server from the Active Directory domain. /opt/likewise/bin/domainjoin-cli leave Reboot the vCenter ServerAdd the identity source as "AD over LDAP" or "AD over LDAPS"Retry the Patching Note: This workaround cannot be applied in the Patching failed state, vCenter needs to be reverted to a healthy state, then apply the workaround.You may move back to IWA (Integrated Windows Authentication) Identity Source after patching to VMware vCenter Server 7.0 U3g build 20150588Detailed steps for each action is available in the documents mentioned in Related Information
Join or Leave an Active Directory DomainAdd or Edit a vCenter Single Sign-On Identity SourceActive Directory over LDAP and OpenLDAP Server Identity Source SettingsConfiguring a vCenter Single Sign-On Identity Source using LDAP with SSL (LDAPS)
Click on a version to see all relevant bugs
VMware Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.
