Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
Analytics service fails to start during the course of upgrade, at stage 10/14.Similar error stack observed in analytics-*.log: ERROR [Analytics Main Thread] com.integrien.analytics.AnalyticsMain.run - AnalyticsMain.run failed with error: AuthException: Auth Exception occurred : Auth Exception occurred : Auth Exception occurred : Parallel deserialization failurecom.vmware.vcops.auth.exception.AuthException: Auth Exception occurred : Auth Exception occurred : Auth Exception occurred : Parallel deserialization failure at com.vmware.vcops.auth.server.AuthorizationServer.initializeAuthObjects(AuthorizationServer.java:283) ~[vcops-auth-server-1.0-SNAPSHOT.jar:?]...Caused by: com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$UnknownFieldException: No such field com.vmware.vcops.auth.xml.bean.UserWrapper.allowAllResources---- Debugging information ----message : No such field com.vmware.vcops.auth.xml.bean.UserWrapper.allowAllResourcesfield : allowAllResourcesclass : com.vmware.vcops.auth.xml.bean.UserWrapperrequired-type : com.vmware.vcops.auth.xml.bean.UserWrapperconverter-type : com.thoughtworks.xstream.converters.reflection.ReflectionConverterline number : -1version : 1.4.19------------------------------- at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.handleUnknownField(AbstractReflectionConverter.java:520) ~[xstream-1.4.19.jar:1.4.19]
Aria Operations upgrades (specifically from 8.6.x to 8.10.x) failing due to JSON deserialization error occurring on auth UserWrapper rows having immediate allowAllResources child element.
Aria Operations upgrade gets stuck with analytics service failing to start.
This issue is known to us for Aria Operations upgrades from version 8.6.x to 8.10.x. This might be fixed in the upcoming releases.
Follow the below steps: Please check for error like "No such field com.vmware.vcops.auth.xml.bean.UserWrapper.allowAllResources" in the analytics logs and observe that the analytics service has stopped/not running. Run the following commands (in Primary node) to check and remove the invalid "UserWrapper -> allowAllResources" for affected rows in postgres repl db kv_auth table: Show before/after for rows containing invalid UserWrapper -> allowAllResources json element where objecttype = 'User-' su - postgres -c "/opt/vmware/vpostgres/current/bin/psql -d vcopsdb -p 5433 -c \"SELECT objecttype,jsonb_pretty(col__kv_strvalue::jsonb) AS jsonbefore, jsonb_pretty((col__kv_strvalue::jsonb) #- '{UserWrapper,allowAllResources}') AS jsonafter FROM kv_auth WHERE objecttype ='User-' AND (col__kv_strvalue::jsonb)->'UserWrapper' ? 'allowAllResources';\"" Delete invalid UserWrapper -> allowAllResources json element from all rows where objecttype = 'User-' su - postgres -c "/opt/vmware/vpostgres/current/bin/psql -d vcopsdb -p 5433 -c \"UPDATE kv_auth SET col__kv_strvalue = (col__kv_strvalue::jsonb) #- '{UserWrapper,allowAllResources}' WHERE objecttype ='User-' AND (col__kv_strvalue::jsonb)->'UserWrapper' ? 'allowAllResources';\"" After fixing the affected kv_auth UserWrapper rows, manually start analytics service on primary node (systemctl start analytics) and the upgrade should continue and complete successfully thereafter.
Required to execute a psql command during the upgrade to fix the UserWrapper rows having an invalid/obsolete allowAllResources immediate child element.