Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
After restarting the services, the sps service gets stuck while starting as the vpxd service crashes. vpxd.log 2020-05-15T16:27:45.328Z error vpxd[7F2F43D7A700] [Originator@6876 sub=vmomi.soapStub[9]] initial service state request failed, disabling pings. error=HTTP Status:400 'Bad Request'2020-05-15T16:27:45.328Z warning vpxd[7F2F43D7A700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 32020-05-15T16:27:45.339Z error vpxd[7F2F7639E800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\Administrator"2020-05-15T16:27:45.339Z error vpxd[7F2F7639E800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\zabbix"2020-05-15T16:27:45.342Z info vpxd[7F2F7639E800] [Originator@6876 sub=[SSO][AdminAdapter]] [LazyInitAdmin] Initializing2020-05-15T16:27:45.342Z info vpxd[7F2F7639E800] [Originator@6876 sub=[SSO][AdminAdapter]] [InitSsoAdminServices]2020-05-15T16:27:45.342Z info vpxd[7F2F7639E800] [Originator@6876 sub=[SSO][AdminAdapter]] Try to connect to SSO VMOMI endpoint2020-05-15T16:27:45.363Z info vpxd[7F2F7639E800] [Originator@6876 sub=[SSO][AdminAdapter]] [InitSsoAdminServices] successful.2020-05-15T16:27:45.363Z info vpxd[7F2F7639E800] [Originator@6876 sub=[SSO][SsoAdminFacadeImpl]] [CheckTokenValidity]2020-05-15T16:27:45.363Z info vpxd[7F2F7639E800] [Originator@6876 sub=[SSO][SsoAdminFacadeImpl]] [CheckTokenValidity] Refreshing SSO token ...2020-05-15T16:27:45.363Z info vpxd[7F2F7639E800] [Originator@6876 sub=[SSO][SsoAdminFacadeImpl]] [RefreshSsoToken]2020-05-15T16:27:45.402Z info vpxd[7F2F7639E800] [Originator@6876 sub=[SSO][SsoAdminFacadeImpl]] [RefreshSsoToken] The VC HOK token has been successfully refreshed.2020-05-15T16:27:45.444Z info vpxd[7F2F7639E800] [Originator@6876 sub=[SSO][AdminAdapter]] Login successful.2020-05-15T16:27:45.469Z warning vpxd[7F2F7639E800] [Originator@6876 sub=AuthzStorageProvider] [AuthzStorageProvider::SyncClientCache] ignore the exception for adding global permission for user N7Vmacore9Authorize26AuthAlreadyExistsExceptionE(ACE already exists for user 'VSPHERE.LOCAL\Administrator')--> [context]zKq7AVECAAAAAJ7V6AAQdnB4ZAAA4LArbGlidm1hY29yZS5zbwAAwhcbAB5aGAAtLCAA42sgAAXTIAHrzgFsaWJhdXRoemNsaWVudC5zbwABRtkBAaHaAQGQ3QECDgCidnB4ZAACDDJVAnQ7VQI63lMDkAUCbGliYy5zby42AAJF1lM=[/context]: VSPHERE.LOCAL\Administrator2020-05-15T16:27:45.470Z error vpxd[7F2F7639E800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for group "VSPHERE.LOCAL\Administrators"2020-05-15T16:27:45.470Z error vpxd[7F2F7639E800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\vsphere-webclient-42fe501d-da41-4c9b-b444-4e036cf7245c"2020-05-15T16:27:45.470Z error vpxd[7F2F7639E800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\vpxd-42fe501d-da41-4c9b-b444-4e036cf7245c"2020-05-15T16:27:45.470Z error vpxd[7F2F7639E800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\vpxd-extension-42fe501d-da41-4c9b-b444-4e036cf7245c"2020-05-15T16:27:45.475Z error vpxd[7F2F424C9700] [Originator@6876 sub=vmomi.soapStub[10]] initial service state request failed, disabling pings. error=HTTP Status:400 'Bad Request'2020-05-15T16:27:45.475Z warning vpxd[7F2F424C9700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 32020-05-15T16:27:45.505Z info vpxd[7F2F427CF700] [Originator@6876 sub=[SSO]] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\Administrator, false)2020-05-15T16:27:45.517Z error vpxd[7F2F7639E800] [Originator@6876 sub=Main] vpxd user does not exist2020-05-15T16:27:45.518Z error vpxd[7F2F7639E800] [Originator@6876 sub=Daemon] Unhandled exception: N5Vmomi5Fault11SystemError9ExceptionE(vmodl.fault.SystemError)--> [context]zKq7AVECAAAAAJ7V6AAKdnB4ZAAA4LArbGlidm1hY29yZS5zbwAAwhcbAB5aGAHoElV2cHhkAAH9FFUBrzNVAXQ7VQE63lMCkAUCbGliYy5zby42AAFF1lM=[/context]2020-05-15T16:27:45.518Z info vpxd[7F2F7639E800] [Originator@6876 sub=SupportMgr] Wrote uptime information
The issue is seen when the vpxd solution user is either removed or not found.In the etc/passwd file the vpxd user will be missing or it's name might have been changed.
vpxd service keeps crashing and user is unable to login to the vCenter.
To resolve the issue make the suggested changes below: Take a snapshot of the vcenter without memoryLog in to the VCSA using SSH.Open the /etc/passwd file with editor : vi /etc/passwdPress Shift+I to insert.Edit the name of the vpxd user or add the missing vpxd user.Validate that the file looks like below : root:x:0:0:root:/root:/bin/applianceshbin:x:1:1:bin:/dev/null:/bin/falsedaemon:x:6:6:Daemon User:/dev/null:/bin/falsemessagebus:x:18:18:D-Bus Message Daemon User:/var/run/dbus:/bin/falsesystemd-bus-proxy:x:72:72:systemd Bus Proxy:/:/bin/falsesystemd-journal-gateway:x:73:73:systemd Journal Gateway:/:/bin/falsesystemd-journal-remote:x:74:74:systemd Journal Remote:/:/bin/falsesystemd-journal-upload:x:75:75:systemd Journal Upload:/:/bin/falsesystemd-network:x:76:76:systemd Network Management:/:/bin/falsesystemd-resolve:x:77:77:systemd Resolver:/:/bin/falsesystemd-timesync:x:78:78:systemd Time Synchronization:/:/bin/falsenobody:x:65534:65533:Unprivileged User:/dev/null:/bin/falsesshd:x:50:50:sshd PrivSep:/var/lib/sshd:/bin/falserpc:x:31:31::/var/lib/rpcbind:/bin/falsetftp:x:999:999:tftp:/home/tftp:/sbin/nologinapache:x:25:25:Apache Server:/srv/www:/bin/falsentp:x:87:87:Network Time Protocol:/var/lib/ntp:/bin/falsesmmsp:x:26:26:Sendmail Daemon:/dev/null:/bin/falsevapiEndpoint:x:1002:100::/home/vapiEndpoint:/sbin/nologinsso-user:x:65536:65535::/var/lib/sso-user:/bin/applianceshdnsmasq:x:998:998::/home/dnsmasq:/bin/applianceshpod:x:1032:59001::/home/pod:/sbin/nologincm:x:1001:100::/home/cm:/sbin/nologinnetdumper:x:1015:997:VMware Netdumper User:/var/core/netdumps:/sbin/nologinvpostgres:x:1012:100::/opt/vmware/vpostgres/current:/sbin/nologincontent-library:x:1006:100::/home/content-library:/sbin/nologineam:x:1005:100::/home/eam:/sbin/nologinimagebuilder:x:1007:100:imagebuilder:/usr/lib/vmware-imagebuilder:/sbin/nologinmbcs:x:1004:100::/home/mbcs:/sbin/nologindeploy:x:1019:996:Deploy User:/var/lib/rbd:/sbin/nologinupdatemgr:x:1017:995:Update Manager User:/usr/lib/vmware-updatemgr:/sbin/nologinvsan-dps:x:1020:100::/home/vsan-dps:/sbin/nologinvsan-health:x:1008:100::/home/vsan-health:/bin/nologinvsm:x:1009:100::/home/vsm:/sbin/nologinvsphere-client:x:1010:100::/home/vsphere-client:/bin/bashperfcharts:x:1011:100::/home/perfcharts:/sbin/nologinvsphere-ui:x:1016:100::/home/vsphere-ui:/bin/bashvpxd:x:1014:59001::/home/vpxd:/bin/false Once the file is validated, restart the vCenter services.