Symptoms

The VxRail plug-in may fail to load due to issues with the VxRail Manager no longer trusting vCenter. When performing a VxRail code upgrade, the VxRail upgrade composite bundles may fail to upload. lcm-web.log 2023-05-19 19:31:15,594 ERROR [LCM] [tomcat-http--42] c.e.m.m.w.w.RestErrorHandler [RestErrorHandler.java:189] Get REST error: error code=1, http status code=500, message=The server has encountered an unexpected internal error. Please try again later. The old certificate files have the fingerprint of the certificate as the name and can be found in the following directories: /var/lib/vmware-marvin/trust /var/lib/vmware-marvin/trust/crl

Cause

The cause is currently unknown.

Resolution

NOTE: It is always recommended to have a snapshot of the VxRail Manager VM prior to modifying certificates. If the wrong files are deleted, the VxRail plug-in may no longer load within vCenter. Take a snapshot of the VxRail Manager VM. SSH to the VxRM as the mystic user, then switch users to root using the su command. Change directories to the /var/lib/vmware-marvin/trust directory. cd /var/lib/vmware-marvin/trust Identify the files using the ls command within the trust directory and determine if any of the old certificates exist. The old certificate files contain the fingerprint of the certificate as the file name (example: 33:3B:CC:91:5C:C1:C7:43:DF:11:BD:FC:DB:D1:CF:76:A6:38:B5:ED) while the valid certificates are named similar to "6bb07dc7.0" Delete the old certificate files stored in the trust directory using the rm -r command. NOTE: When removing the files it is recommended to type the first two number or letters of the file name, then press the Tab key to autocomplete the file name since when using the command it needs a backslash before each colon symbol. Change directories to the /var/lib/vmware-marvin/trust/crl directory, then repeat the steps above to remove any old CRL files identified by the thumbprint naming convention. After the old certificate and CRL files have been removed, restart the vmware-marvin and runjars services on the VxRail Manager. systemctl restart vmware-marvin; systemctl restart runjars

Support Cases