Symptoms
The script responsible for initial vRA deployment /opt/scripts/deploy.sh fails with a 403 error=========================[2020-07-18 09:48:46.575+0000] Populating initial identity-service data=========================vaconfig.prelude.vmware.com/prelude-vaconfig patched (no change % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0^M100 29 0 0 100 29 0 24 0:00:01 0:00:01 --:--:-- 24^M100 29 0 0 100 29 0 15 0:00:01 0:00:01 --:--:-- 15curl: (22) The requested URL returned error: 403 ForbiddenDeployment failed. Collecting log bundle ...vIDM accesscontrol-service.log displays:2020-07-18 09:48:46,474 GMT WARN vidm.lab:accesscontrol (ForkJoinPool-4-worker-0) [TENANT1;<UUID>;<IP>;<UUID>] com.vmware.vidm.common.interservice.ServiceTokenProviderGateway - Invalid credentials for service token. Http Response code: 4032020-07-18 09:48:46,474 GMT WARN vidm.lab:accesscontrol (ForkJoinPool-4-worker-0) [TENANT1;<UUID>;<IP>;<UUID>] com.vmware.vidm.accesscontrol.gateway.UserGroupServiceGateway - UserGroup service call failed com.vmware.vidm.common.interservice.exception.InvalidClientCredentialsException: Invalid credentials used to get service token ...Caused by: ScForbiddenException[Operation: GET -> https://master.vidm.lab:443/SAAS/jersey/manager/api/scim/Users/<UUID>?attributes=groups][Status:403] ...
Resolution
Take a Snapshot of the vIDM cluster using vRealize Suite Lifecycle Manager1. Connect the the vIDM primary database via SSH connection. To validate which is the primary database use this command:
su postgres -c "echo -e 'password'|/opt/vmware/vpostgres/current/bin/psql -h localhost -p 9999 -U pgpool postgres -c \"show pool_nodes\""
2. Extract the database password
cat /usr/local/horizon/conf/db.pwd
3. connect to vPostgres database using the earlier extracted password
/opt/vmware/vpostgres/current/bin/psql -U postgres saas
4. Validate the field "inheritanceAllowed" is set to false
SELECT "inheritanceAllowed" FROM "OAuth2Client" WHERE "clientId"='acs';
5. If the field "inheritanceAllowed" is set to false the update it to true
UPDATE "OAuth2Client" SET "inheritanceAllowed"=true WHERE "clientId"='acs' AND "idOrganization"=2;
6. Exit vPostgres and the shell session on vIDM
\q
7. Open an SSH session to one of the vRA 8.1 nodes and run the deploy script
/opt/scripts/deploy.sh
