OPERATIONAL DEFECT DATABASE
...
...
This article provides information on how to manually reviewing the Certificate Authority (CA) signed SSL certificates in a vSphere 6 or 7 environment. In vSphere 6 and 7, certificates generated by the VMware Certificate Authority (VMCA) can be monitored through the vSphere Web Client. For more information, see the View vCenter Certificates with the vSphere Web Client section in the Platform Services Controller Administration Guide.Note: You will need to manage your own certificate validity if you are using your own Private Key Infrastructure (PKI) in your environment.This article uses the vecs-cli command to list certificates stored in the VMware Endpoint Certificate Store (VECS) as well as references the individual keystores used by vSphere. Before proceeding, familiarize yourself by reviewing the Where vSphere Uses Certificates and vecs-cli Command Reference section of the vSphere Authentication Guide.
Reviewing stored certificates for WindowsReviewing stored certificates for vCenter Server ApplianceExporting stored certificates for WindowsExporting stored certificates for vCenter Server Appliance Reviewing stored certificates for Windows: Note: These steps are written using the default installation path of C:\ drive on Windows. If you are using a non-default installation path, these steps need to be modified. Open an elevated command prompt. If an elevated command prompt is not used, the entry list commands fail with Win Error: access is denied.Run this command to list all of the Keystores stored within VECS. This command can be run from a vCenter Server or a Platform Services Controller. "C:\Program Files\VMware\vCenter Server\vmafdd\"vecs-cli store list This will output one of these lists depending on what node this command is performed on. For External Platform Services Controller MACHINE_SSL_CERT TRUSTED_ROOTS TRUSTED_ROOT_CRLS machine For vCenter Server with external Platform Services Controller or embedded Platform Services Controller MACHINE_SSL_CERT TRUSTED_ROOTS TRUSTED_ROOT_CRLS machine vpxd vpxd-extension vsphere-webclient SMS To review the individual Keystores, use these examples to list the certificate stored: "C:\Program Files\VMware\vCenter Server\vmafdd\"vecs-cli entry list --store TRUSTED_ROOTS --text | more "C:\Program Files\VMware\vCenter Server\vmafdd\"vecs-cli entry list --store MACHINE_SSL_CERT --text | more "C:\Program Files\VMware\vCenter Server\vmafdd\"vecs-cli entry list --store machine --text | more "C:\Program Files\VMware\vCenter Server\vmafdd\"vecs-cli entry list --store vpxd --text | more "C:\Program Files\VMware\vCenter Server\vmafdd\"vecs-cli entry list --store vpxd-extension --text | more "C:\Program Files\VMware\vCenter Server\vmafdd\"vecs-cli entry list --store vsphere-webclient --text | more "C:\Program Files\VMware\vCenter Server\vmafdd\"vecs-cli entry list --store SMS --text | more "C:\Program Files\VMware\vCenter Server\vmafdd\"vecs-cli entry list --store TRUSTED_ROOT_CRLS --text | more Note: Press q to leave the session. Review the certificates displayed. Notes: When reviewing the MACHINES_SSL_CERT or any of the Solution User stores, take note of the X509v3 extensions, particularly Key Usages, Validity, and Subject Alternate Name. For customers who upgraded to vSphere 6, the MACHINE_SSL_CERT will now be the certificate previously used for the vCenter Server.When reviewing the TRUSTED_ROOTS store, take note of the X509v3 extensions, particularly the Key Usage Certificate Sign and Validity. If the Certificate Sign Key Usage is missing, the VMCA is unable to sign and provision certificates thus causing installation and certificate regeneration failuresYou might see below Stores as well in some situations depending on vCenter build, use the vecs-cli commands mentioned above to list the certificates stored in these stores: BACKUP_STOREdata-enciphermentKMS_ENCRYPTION Reviewing stored certificates for vCenter Server Appliance: Open an SSH session to the vCenter Server Appliance. Log in as root. Switch to using a BASH shell session by using this command: shell.set --enabled true shell Run this command to list all of the Keystores stored within VECS. This command can be run from a vCenter Server or a Platform Services Controller. /usr/lib/vmware-vmafd/bin/vecs-cli store list This will output one of the following lists depending on what node this command is performed on. External Platform Services Controller MACHINE_SSL_CERT TRUSTED_ROOTS TRUSTED_ROOT_CRLS machine For vCenter Server Appliance with external Platform Services Controller or embedded Platform Services Controller MACHINE_SSL_CERT TRUSTED_ROOTS TRUSTED_ROOT_CRLS machine vpxd vpxd-extension vsphere-webclient SMS In order to review the individual Keystores, use these examples to list the certificate stored: /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store TRUSTED_ROOTS --text | less /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store MACHINE_SSL_CERT --text | less /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store machine --text | less /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store vpxd --text | less /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store vpxd-extension --text | less /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store vsphere-webclient --text | less /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store SMS --text | less /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store TRUSTED_ROOT_CRLS --text | less Note: Press q to leave the session. Review the certificates displayed. Notes: When reviewing the MACHINES_SSL_CERT or any of the Solution User stores, take note of the X509v3 extensions, particularly Key Usages, Validity, and Subject Alternate Name For customers who upgraded to vSphere 6, the MACHINE_SSL_CERT will now be the certificate previously used for the vCenter Server.When reviewing the TRUSTED_ROOTS store, take note of the X509v3 extensions, particularly the Key Usage Certificate Sign and Validity. If the Certificate Sign Key Usage is missing, the VMCA is unable to sign and provision certificates thus causing installation and certificate regeneration failures.You might see below Stores as well in some situations depending on vCenter build, use the vecs-cli commands mentioned above to list the certificates stored in these stores: BACKUP_STOREdata-enciphermentKMS_ENCRYPTION Exporting stored certificates for Windows: Note: These steps are written using the default installation path of C:\ drive on Windows. If you are using a non-default installation path, these steps need to be modified. Open an elevated command prompt. If an elevated command prompt is not used, the entry list commands fail with Win Error: access is denied.Run these commands to list all of the Keystores stored within VECS. These commands can be run from a vCenter Server or a Platform Services Controller. Create directory "C:\Certificates" before proceeding with below steps. "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getcert --store <stored name> --alias <alias name> --output c:\certificates\<certificate usage name>.crt "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getkey --store <stored name> --alias <stored name> --output c:\certificates\<certificate usage name>.key Users have the option to output one of the following store's pair depending on what node this command is performed on. For External Platform Services Controller MACHINE_SSL_CERT TRUSTED_ROOTS TRUSTED_ROOT_CRLS machine For vCenter Server with external Platform Services Controller or embedded Platform Services Controller MACHINE_SSL_CERT TRUSTED_ROOTS TRUSTED_ROOT_CRLS machine vpxd vpxd-extension vsphere-webclient SMS To review the individual Keystores, use these examples to list the certificate stored: "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getcert --store MACHINE_SSL_CERT --alias __MACHINE_CERT --output c:\certificates\machine_ssl.crt "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getkey --store MACHINE_SSL_CERT --alias __MACHINE_CERT --output c:\certificates\machine_ssl.key "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getcert --store vpxd-extension --alias vpxd-extension --output c:\certificates\vpxd-extension.crt "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getkey --store vpxd-extension --alias vpxd-extension --output c:\certificates\vpxd-extension.key "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getcert --store vpxd --alias vpxd --output c:\certificates\vpxd.crt "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getkey --store vpxd --alias vpxd --output c:\certificates\vpxd.key "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getcert --store machine --alias machine --output c:\certificates\machine.crt "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getkey --store machine --alias machine --output c:\certificates\machine.key "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getcert --store vsphere-webclient --alias vsphere-webclient --output c:\certificates\vsphere-webclient.crt "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getkey --store vsphere-webclient --alias vsphere-webclient --output c:\certificates\vsphere-webclient.key Exporting stored certificates for vCenter Server Appliance: Open an SSH session to the vCenter Server Appliance. Log in root. Switch to using a BASH shell session by using the command: shell.set --enabled true shell Create the export location directory by running this command "mkdir /certificate".Run these commands to export the Key and Certificate pairs stored within VECS one by one. These commands can be run from a vCenter Server or a Platform Services Controller. /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store <stored name> --alias <alias name> --output /certificate/<certificate usage name>.crt /usr/lib/vmware-vmafd/bin/vecs-cli entry getkey --store <stored name> --alias <stored name> --output /certificate/<certificate usage name>.key Users have the option to output one of the following store's pair depending on what node this command is performed on. External Platform Services Controller MACHINE_SSL_CERT TRUSTED_ROOTS TRUSTED_ROOT_CRLS machine For vCenter Server Appliance with external Platform Services Controller or embedded Platform Services Controller MACHINE_SSL_CERT TRUSTED_ROOTS TRUSTED_ROOT_CRLS machine vpxd vpxd-extension vsphere-webclient SMS Use these commands as examples for exporting the stored pairs. /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store MACHINE_SSL_CERT --alias __MACHINE_CERT --output /certificate/Machine_SSL.crt /usr/lib/vmware-vmafd/bin/vecs-cli entry getkey --store MACHINE_SSL_CERT --alias __MACHINE_CERT --output /certificate/Machine_SSL.key /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store vpxd-extension --alias vpxd-extension --output /certificate/vpxd-extension.crt /usr/lib/vmware-vmafd/bin/vecs-cli entry getkey --store vpxd-extension --alias vpxd-extension --output /certificate/vpxd-extension.key /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store vpxd --alias vpxd --output /certificate/vpxd.crt /usr/lib/vmware-vmafd/bin/vecs-cli entry getkey --store vpxd --alias vpxd --output /certificate/vpxd.key /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store machine --alias machine --output /certificate/machine.crt /usr/lib/vmware-vmafd/bin/vecs-cli entry getkey --store machine --alias machine --output /certificate/machine.key /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store vsphere-webclient --alias vsphere-webclient --output /certificate/vsphere-webclient.crt /usr/lib/vmware-vmafd/bin/vecs-cli entry getkey --store vsphere-webclient --alias vsphere-webclient --output /certificate/vsphere-webclient.key
VMware Skyline Health Diagnostics for vSphere - FAQ (81931)"Error in creating a new entry for __MACHINE_CERT in VECS Store MACHINE_SSL_CERT", Certificate Replacement with Custom Certificate fails on 6.x/7.x (70777)"Operation failed with error ERROR_INVALID_PARAMETER (87)" & "Failed to get vecs users and permissions" while upgrading from vCSA 6.5U2d to 6.7U1 (59508)"Unable to enumerate and validate the root certificates from the TRUSTED_ROOTS VECS store" pre-upgrade check error while upgrading to VC 6.7 (70902)"VasaServiceException: org.apache.axis2.AxisFault: certificate has expired", SMS Certificate Expiry Alarm after upgrading vCenter Server from 5.x to 6.x (2120105)How to replace the vSphere 6.0 Solution User certs with VMCA issued certs (2112281) For information on renewing VMware certificates, see: Replacing the vSphere 6.0 Machine SSL certificate with a VMware Certificate Authority issued certificate (2112279)
Click on a version to see all relevant bugs
VMware Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.
