Symptom

OOB (Out Of Band) workflow is a set of APIs CDO runs every 10mins on cdFMC to know the system status. These APIs are run by CDO's system state machine and translate to ANUBIS:system-user on cdfmc. Due to a race condition between Tomcat process startup and OOB workflow running an API, ANUBIS user's session is not created cleanly and ends up missing a vms_session_id / sf_action_id. This causes the APIs to return 500 error due to a null pointer exception when the API layer expects the session id to be present in the session.

Conditions

OOB workflow in CDO fails due to 500 error in response for /serverversion API Check tomcat stdout.logs to see if /serverversion API fails with 500 error

Workaround

OmniQuery.pl mdb select * from sessions; Check the ANUBIS user's session and notice if its missing vms_session_id or sf_action_id delete from sessions where id=''; exit;

Further Problem Description