Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
After restoring vCenter Server or vCenter Server Appliance from backup or a snapshot, you experience these symptoms: In the /var/log/vmware/vpxd/vpxd.log file, you see entries similar to: 2017-02-10T16:54:07.586Z warning vpxd[7F38BD9C2700] [Originator@6876 sub=Default] Failed to connect socket; <io_obj p:0x00007f38a41cc910, h:32, <TCP '0.0.0.0:0'>, <TCP '127.0.0.1:10080'>>, e: system:111(Connection refused) 2017-02-10T16:54:07.586Z warning vpxd[7F38CC164800] [Originator@6876 sub=VpxdAuthClient] [ConnectAndLogin] Failed to loginBySamlToken: N7Vmacore15SystemExceptionE(Connection refused) --> [context]zKq7AVECAAAAAMJySwAPdnB4ZAAAkJ4rbGlidm1hY29yZS5zbwAAhE8bAOqXGADgXysAWVcjAApfJgD+YCYAhIMmAICKJgCfISQAb+0jAFbwIwB79isBVHQAbGlicHRocmVhZC5zby4wAAItjg5saWJjLnNvLjYA[/context] 2017-02-10T16:54:07.587Z info vpxd[7F38CC164800] [Originator@6876 sub=VpxdAuthClient] fallback to loginByCertificate 2017-02-10T16:54:07.587Z warning vpxd[7F38B7C78700] [Originator@6876 sub=Default] Failed to connect socket; <io_obj p:0x00007f38a41c4310, h:32, <TCP '0.0.0.0:0'>, <TCP '127.0.0.1:10080'>>, e: system:111(Connection refused) 2017-02-10T16:54:07.587Z error vpxd[7F38CC164800] [Originator@6876 sub=httpUtil] [HttpUtil::ExecuteRequest] Error in sending request - Connection refused 2017-02-10T16:54:07.587Z error vpxd[7F38CC164800] [Originator@6876 sub=ServerAccess] Remote login failed: N3Vim5Fault9HttpFault9ExceptionE(vim.fault.HttpFault) --> [context]zKq7AVECAAAAAMJySwAUdnB4ZAAAkJ4rbGlidm1hY29yZS5zbwAAhE8bAOqXGAEOylR2cHhkAAH65ogBK+6IAbvs9QEchqIBJ3GiArgWAWxpYmF1dGh6Y2xpZW50LnNvAALYSAEC8E4BAsraAQJI0AEBhuWgAQrpVAE48lQBapVTA+AFAmxpYmMuc28uNgABdY1T[/context] 2017-02-10T16:54:07.589Z error vpxd[7F38CC164800] [Originator@6876 sub=AuthzStorageProvider] [AuthzStorageProvider::CreateAuthzMgr] Failed to connect to IS: <N5Vmomi5Fault17HostCommunication9ExceptionE(vmodl.fault.HostCommunication)</font> --> [context]zKq7AVECAAAAAMJySwATdnB4ZAAAkJ4rbGlidm1hY29yZS5zbwAAhE8bAOqXGAEOylR2cHhkAAF2E2IBsu/1ARyGogEncaICuBYBbGliYXV0aHpjbGllbnQuc28AAthIAQLwTgECytoBAkjQAQGG5aABCulUATjyVAFqlVMD4AUCbGliYy5zby42AAF1jVM=[/context]> 2017-02-10T16:54:07.590Z info vpxd[7F38CC164800] [Originator@6876 sub=AuthzStorageProvider] [AuthzStorageProvider::CreateAuthzMgr] Retry for this error: attempt count 1 2017-02-10T16:54:10.632Z warning vpxd[7F38B6346700] [Originator@6876 sub=Default] Failed to connect socket; <io_obj p:0x00007f385c24cb90, h:32, <TCP '0.0.0.0:0'>, <TCP '127.0.0.1:10080'>>, e: system:111(Connection refused) 2017-02-10T16:54:10.633Z warning vpxd[7F38CC164800] [Originator@6876 sub=VpxdAuthClient] [ConnectAndLogin] Failed to loginBySamlToken: N7Vmacore15SystemExceptionE(Connection refused) --> [context]zKq7AVECAAAAAMJySwAPdnB4ZAAAkJ4rbGlidm1hY29yZS5zbwAAhE8bAOqXGADgXysAWVcjAApfJgD+YCYAhIMmAICKJgCfISQAb+0jAFbwIwB79isBVHQAbGlicHRocmVhZC5zby4wAAItjg5saWJjLnNvLjYA[/context] 2017-02-10T16:54:10.633Z info vpxd[7F38CC164800] [Originator@6876 sub=VpxdAuthClient] fallback to loginByCertificate 2017-02-10T16:54:10.633Z warning vpxd[7F38B75EB700] [Originator@6876 sub=Default] Failed to connect socket; <io_obj p:0x00007f385c24cb90, h:32, <TCP '0.0.0.0:0'>, <TCP '127.0.0.1:10080'>>, e: system:111(Connection refused) 2017-02-10T16:54:10.634Z error vpxd[7F38CC164800] [Originator@6876 sub=httpUtil] [HttpUtil::ExecuteRequest] Error in sending request - Connection refused 2017-02-10T16:54:10.634Z error vpxd[7F38CC164800] [Originator@6876 sub=ServerAccess] Remote login failed: N3Vim5Fault9HttpFault9ExceptionE(vim.fault.HttpFault) --> [context]zKq7AVECAAAAAMJySwAUdnB4ZAAAkJ4rbGlidm1hY29yZS5zbwAAhE8bAOqXGAEOylR2cHhkAAH65ogBK+6IAbvs9QEchqIBJ3GiArgWAWxpYmF1dGh6Y2xpZW50LnNvAALYSAEC8E4BAsraAQJI0AEBhuWgAQrpVAE48lQBapVTA+AFAmxpYmMuc28uNgABdY1T[/context] 2017-02-10T16:54:10.635Z error vpxd[7F38CC164800] [Originator@6876 sub=AuthzStorageProvider] [AuthzStorageProvider::CreateAuthzMgr] Failed to connect to IS: <N5Vmomi5Fault17HostCommunication9ExceptionE(vmodl.fault.HostCommunication)</font> --> [context]zKq7AVECAAAAAMJySwATdnB4ZAAAkJ4rbGlidm1hY29yZS5zbwAAhE8bAOqXGAEOylR2cHhkAAF2E2IBsu/1ARyGogEncaICuBYBbGliYXV0aHpjbGllbnQuc28AAthIAQLwTgECytoBAkjQAQGG5aABCulUATjyVAFqlVMD4AUCbGliYy5zby42AAF1jVM=[/context]> ... 2017-02-10T16:57:10.714Z error vpxd[7F38CC164800] [Originator@6876 sub=Authorize] Failed to initialize authorizeManager 2017-02-10T16:57:10.714Z warning vpxd[7F38CC164800] [Originator@6876 sub=VpxProfiler] Start [VpxdAuthorize::Start()] took 183263 ms 2017-02-10T16:57:10.714Z warning vpxd[7F38CC164800] [Originator@6876 sub=VpxProfiler] ServerApp::Start [TotalTime] took 183356 ms 2017-02-10T16:57:10.714Z error vpxd[7F38CC164800] [Originator@6876 sub=Default] Failed to start VMware VirtualCenter. Shutting down 2017-02-10T16:57:10.714Z info vpxd[7F38CC164800] [Originator@6876 sub=SupportMgr] Wrote uptime information 2017-02-10T16:59:05.419Z info vpxd[7F38BE15F700] [Originator@6876 sub=HostGateway] CmConnectionFSM::RunFSM(ST_CM_CALL_FAILED) 2017-02-10T16:59:05.420Z warning vpxd[7F38BE15F700] [Originator@6876 sub=HostGateway] State(ST_CM_LOGIN) failed with: Invalid argument : cmStub 2017-02-10T16:59:05.420Z warning vpxd[7F38BE15F700] [Originator@6876 sub=HostGateway] State(ST_ACQUIRE_TOKEN) failed with: Invalid argument : stsClient 2017-02-10T16:59:05.420Z info vpxd[7F38BE15F700] [Originator@6876 sub=HostGateway] stsUrlFromConfig: https://comp01psc01.sfo01.rainpole.local/sts/STSService/vsphere.local ssoAdminUrlFromConfig: https://comp01psc01.sfo01.rainpole.local/sso-adminserver/sdk/vsphere.local 2017-02-10T16:59:05.448Z info vpxd[7F38BE15F700] [Originator@6876 sub=vpxCrypt] Failed to read X509 cert; err: 151441516 2017-02-10T16:59:05.478Z info vpxd[7F38BE15F700] [Originator@6876 sub=vpxCrypt] Failed to read X509 cert; err: 151441516 In the /var/log/vmware/vmafdd/vmafdd-syslog.log files, you see entries similar to: 17-02-10T16:56:21.213900+00:00 err vmafdd t@139929541560064: VmAfSrvCfgGetMachineID failed. Error(9234) 17-02-10T16:56:21.214035+00:00 err vmafdd t@139929541560064: [Error - 9234, ../../../server/vmafd/config.c:440] 17-02-10T16:56:21.214166+00:00 err vmafdd t@139929541560064: VmAfSrvGetCMLocation failed. Error(9234) 17-02-10T16:56:21.214295+00:00 err vmafdd t@139929541560064: ERROR! [VmAfdIpcGetCMLocation] is returning [9234] 17-02-10T16:56:22.020892+00:00 err vmafdd t@139929716352768: [Error - 9234, ../../../server/vmafd/ldap.c:170] 17-02-10T16:56:22.021971+00:00 err vmafdd t@139929716352768: [Error - 9234, ../../../server/vmafd/rootfetch.c:256] 17-02-10T16:56:22.022297+00:00 notice vmafdd t@139929716352768: Failed to update trusted roots. Error [9234] 17-02-10T16:56:22.026505+00:00 err vmafdd t@139929564739328: [Error - 9234, ../../../server/vmafd/ldap.c:170] 17-02-10T16:56:33.704271+00:00 err vmafdd t@139929541560064: [Error - 9234, ../../../server/vmafd/vmdir.c:1468] 17-02-10T16:56:33.704528+00:00 notice vmafdd t@139929541560064: VmAfSrvDirOpenConnection failed. Error(9234) 17-02-10T16:56:33.704686+00:00 err vmafdd t@139929541560064: [Error - 9234, ../../../server/vmafd/vmdir.c:1023] 17-02-10T16:56:33.704825+00:00 notice vmafdd t@139929541560064: VmAfSrvDirGetMachineId failed. Error(9234) 17-02-10T16:56:33.704983+00:00 err vmafdd t@139929541560064: [Error - 9234, ../../../server/vmafd/config.c:1307] When attempting to execute one of the VMAFD-CLI get-* or set-* commands, you see output similar to: root@mgmt01vc01 [ /var/log/vmware ]# /usr/lib/vmware-vmafd/bin/vmafd-cli get-machine-id --server-name localhost Error 9234: Authentication to VMware Directory Service failed. Verify the username and password. root@mgmt01vc01 [ /var/log/vmware/vpxd ]# /usr/lib/vmware-vmafd/bin/vmafd-cli set-dc-name --server-name localhost --dc-name mgmt01psc01.sfo01.rainpole.local Error 9234: Authentication to VMware Directory Service failed. Verify the username and password. Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
Periodically, the password used by the Machine Accounts' is automatically refreshed. This password is stored on the Machine Account on vCenter Server and within the VMware Directory Service residing on the Platform Services Controllers. If the backup used to perform a restore which pre-dates the most recent password refresh, upon restore the Machine Account will be unable to log into the VMware Directory Service.Additionally, if the vCenter Server is rolled back to a snapshot state which pre-dates the most recent password refresh, upon restore the Machine Account will be unable to log into the VMware Directory Service.
To update the password used by the vCenter Server's Machine Account after a restore, use the vcenter-restore CLI to issue a reset request. For more information, see the Restore the Failed vCenter Server Virtual Machine With the Direct-to-Host Emergency Restore Operation section in the vSphere Installation and Setup Guide. For vCenter Server Appliance Login to vCenter Server via SSH as the root user Run this command to switch to BASH shell Run this command to Stop all of the vCenter Server services: service-control --stop --all Run this command (Below does not work on embedded node) vcenter-restore -u administrator -p <administrator@vsphere.local password> For example: vcenter-restore -u administrator -p VMware1! Note: This CLI can be executed without either option, and the user will be prompted to provide the administrator and administrator@vsphere.local password.Run this command to confirm that all services are now started: service-control --status --all For vCenter Server on Windows Open an elevated command prompt. Run this command to change the directories: cd C:\Program Files\VMware\vCenter Server\bin Run this command to Stop all of the vCenter Server services: service-control --stop --all Run this command (Below does not work on embedded node) vcenter-restore -u administrator -p <administrator@vsphere.local password> For example: vcenter-restore -u administrator -p VMware1! Note: This CLI can be executed without either option, and the user will be prompted to provide the administrator and administrator@vsphere.local password. Run this command to confirm that all services are now started: service-control --status --all
To manually invoke an update of vCenter Server's Machine Account password, use the dir-cli computer password-reset CLI to issue a reset request. For vCenter Server Appliance Login to the vCenter Server via SSH as the root user Run this command to switch to BASH shell Run this command: /usr/lib/vmware-vmafd/bin/dir-cli computer password-reset --login administrator --live-dc-hostname <Platform Services Controller FQDN> --password <administrator@vsphere.local password> For example: /usr/lib/vmware-vmafd/bin/dir-cli computer password-reset --login administrator --live-dc-hostname mgmt01psc01.sfo01.rainpole.local --password 'VMware1!' A successful password reset will output: Password for machine account reset. Note: If you are using a PSC HA configuration using a Load Balancer, execute this command against one of the backing PSC members and wait 30 seconds for the VMware Directory Service replication cycle to push this update to all nodes. Run these commands to Start the vCenter Server services: service-control --stop --all service-control --start --all 从备份或快照还原 vCenter Server 之后，vCenter Server 无法启动并显示“Remote login failed:N3Vim5Fault9HttpFault9ExceptionE(vim.fault.HttpFault)”VCenter Server が起動に失敗し、次のエラーが表示される：「リモート ログインに失敗しました：N3Vim5Fault9HttpFault9ExceptionE(vim.fault.HttpFault)」、vCenter Server がバックアップまたはスナップショットからリストアされた後("Remote login failed:N3Vim5Fault9HttpFault9ExceptionE(vim.fault.HttpFault)", After vCenter Server is re