OPERATIONAL DEFECT DATABASE
...
...
vc-ws1a-broker is unable to startservice-control denotes that vc-ws1a-broker is in a stopped statePatch failure during the starting of servicesThe /var/log/vmware/applmgmtt/PatchRunner.log will denote the following stderr=Service-control failed. Error: Failed to start services in profile ALL. RC=1, stderr=Failed to start vc-ws1a-broker services. Error: Operation timed out The /var/log/vmware/vc-ws1a-broker/accesscontrol_application.properties will denote time="2023-11-06T16:26:09Z" level=error msg="runc create failed: unable to start container process: exec: \"runsvdir\": executable file not found in $PATH" time="2023-11-06T16:26:09Z" level=error msg="exec failed: container does not exist"
This issue is due to a database lock preventing vc-ws1a-broker from starting.
This is currently a known issue affecting vCenter 8.x
A database lock may be present in the following tables. You can review the corresponding logs to the isolate which lock may be present in the environment. Table Name Error logdatabasechangeloglocktoken-service.logacs_databasechangeloglockaccesscontrol-service.logfed_databasechangeloglockfederation-service.logugs_databasechangeloglockusergroup-service.logcryptodatabasechangeloglockcrypto-service.log Once the lock has been identified take offline snapshots before proceeding further.Use the below Postgres commands referenced for each lock type. Validate that the lock is present /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c 'SELECT * FROM vidm_schema."databasechangeloglock";' /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c 'SELECT * FROM vidm_schema."acs_databasechangeloglock";' /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c 'SELECT * FROM vidm_schema."fed_databasechangeloglock";' /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c 'SELECT * FROM vidm_schema."ugs_databasechangeloglock";' /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c 'SELECT * FROM vidm_schema."cryptodatabasechangeloglock";' Release the lock /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "UPDATE vidm_schema.\"databasechangeloglock\" SET \"locked\"=FALSE, \"lockgranted\"=NULL, \"lockedby\"=NULL WHERE "id"=1;" /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "UPDATE vidm_schema.\"acs_databasechangeloglock\" SET \"locked\"=FALSE, \"lockgranted\"=NULL, \"lockedby\"=NULL WHERE "id"=1;" /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "UPDATE vidm_schema.\"fed_databasechangeloglock\" SET \"locked\"=FALSE, \"lockgranted\"=NULL, \"lockedby\"=NULL WHERE "id"=1;" /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "UPDATE vidm_schema.\"ugs_databasechangeloglock\" SET \"locked\"=FALSE, \"lockgranted\"=NULL, \"lockedby\"=NULL WHERE "id"=1;" /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "UPDATE vidm_schema.\"cryptodatabasechangeloglock\" SET \"locked\"=FALSE, \"lockgranted\"=NULL, \"lockedby\"=NULL WHERE "id"=1;" Validate the lock has been released /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c 'SELECT * FROM vidm_schema."databasechangeloglock";' /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c 'SELECT * FROM vidm_schema."acs_databasechangeloglock";' /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c 'SELECT * FROM vidm_schema."fed_databasechangeloglock";' /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c 'SELECT * FROM vidm_schema."ugs_databasechangeloglock";' /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c 'SELECT * FROM vidm_schema."cryptodatabasechangeloglock";' Restart vc-ws1a-broker service-control --restart vc-ws1a-broker
curl -k --location --request GET 'https://{vc-hostname}/sddc/broker/health/ready' where {vc-hostname} must be replaced with the output of `hostname` commandWill return No healthy upstream.
Click on a version to see all relevant bugs
VMware Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.
