Symptoms

After leaving the NMC > Monitoring > Policies > Show Details for a particular Policy's Workflow open longer than the Service Token Timeout period, the access to the Show Details for this policy results in 38% progress (38 percent) which goes no further. Even after 10 minutes the 38% progress remains.The NMC UI must End Task in Windows 2012.Reviewing Windows Task Manager on the NetWorker for Windows server, which is also the NMC server and the authentication server, shows: Java (TM) Web Launcher (2) (jp2launcher.exe) is using 10% to 45% CPU. gstd.exe and java.exe are using 10% to 30% CPU, and fluctuating. nsrexecd.exe is using between 2% and 15% CPU and fluctuating. The gstd.raw file is reporting (notice the frequency in the timestamps): C:\<directory>\Management\GST\logs>nsr_render_log -B -10 gstd.raw 144502 2017-08-13 11:51:35 AM 5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>: Security token has expired. 144502 2017-08-13 11:51:35 AM 5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>: Security token has expired. 144502 2017-08-13 11:51:35 AM 5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>: Security token has expired. 144502 2017-08-13 11:51:35 AM 5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>: Security token has expired. 144502 2017-08-13 11:51:35 AM 5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>: Security token has expired. 144502 2017-08-13 11:51:35 AM 5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>: Security token has expired. 144502 2017-08-13 11:51:35 AM 5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>: Security token has expired. 144502 2017-08-13 11:51:35 AM 5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>: Security token has expired. 144502 2017-08-13 11:51:35 AM 5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>: Security token has expired. 144502 2017-08-13 11:51:36 AM 5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>: Security token has expired. 144502 2017-08-13 11:51:36 AM 5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>: Security token has expired. Stopped the gstd.exe service C:\<directory>\Management\GST\logs>net stop gstd ................. The EMC GST Service service was stopped successfully. Checked the Windows tasklist to ensure that all executables had stopped: C:\<directory>\Management\GST\logs>tasklist | findstr gst C:\<directory>\Management\GST\logs>tasklist | findstr http httpd.exe 1836 Services 0 6,296 K httpd.exe 596 Services 0 6,568 K C:\<directory>\Management\GST\logs>tasklist | findstr post postgres.exe 3296 Services 0 12,176 K postgres.exe 988 Services 0 4,756 K postgres.exe 2156 Services 0 6,464 K postgres.exe 2100 Services 0 5,536 K postgres.exe 1532 Services 0 5,328 K postgres.exe 3648 Services 0 6,684 K postgres.exe 3132 Services 0 4,940 K Used taskkill to end the errant httpd.exe and postgres.exe: C:\<directory>\Management\GST\logs>TASKKILL /F /IM httpd.exe SUCCESS: The process "httpd.exe" with PID 1836 has been terminated. SUCCESS: The process "httpd.exe" with PID 596 has been terminated. C:\<directory>\Management\GST\logs>TASKKILL /F /IM postgres* SUCCESS: The process "postgres.exe" with PID 3296 has been terminated. SUCCESS: The process "postgres.exe" with PID 988 has been terminated. SUCCESS: The process "postgres.exe" with PID 2156 has been terminated. SUCCESS: The process "postgres.exe" with PID 2100 has been terminated. ERROR: The process "postgres.exe" with PID 1532 could not be terminated. Reason: There is no running instance of the task. ERROR: The process "postgres.exe" with PID 3648 could not be terminated. Reason: The operation attempted is not supported. ERROR: The process "postgres.exe" with PID 3132 could not be terminated. Reason: The operation attempted is not supported. C:\<directory>\Management\GST\logs> Verified that the gstd processes are no longer listed in the tasklist: C:\<directory>\Management\GST\logs>tasklist | findstr gst C:\<directory>\Management\GST\logs>tasklist | findstr http C:\<directory>\Management\GST\logs>tasklist | findstr post Started the NMC services: C:\<directory>\Management\GST\logs>net start gstd The EMC GST Service service is starting. The EMC GST Service service was started successfully. THE ISSUE PERSISTS! The errant httpd.exe and postgres.exe are not the issue.Working through instructions in the Security and Configuration Guide, stopping the gstd.exe, and ensuring all are stopped: tasklist | findstr gst tasklist | findstr http tasklist | findstr post With the GST service down, run: gstauthcfg -c -t -h New_authentication_service_hostname -p port_number Notice that the default port number is 9090.Start the gstd service.This message is now returned to the gstd.raw: C:\<directory>\Management\GST\logs>nsr_render_log -B -10 gstd.raw 0 2017-08-13 1:02:34 PM 2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect] 0 2017-08-13 1:03:35 PM 2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect] 0 2017-08-13 1:04:36 PM 2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect] 0 2017-08-13 1:05:38 PM 2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect] 0 2017-08-13 1:06:39 PM 2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect] 0 2017-08-13 1:07:40 PM 2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect] 0 2017-08-13 1:08:41 PM 2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect] 0 2017-08-13 1:09:43 PM 2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect] 0 2017-08-13 1:10:44 PM 2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect] 0 2017-08-13 1:11:45 PM 2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect] C:\<directory>\Management\GST\logs>

Cause

The authentication from the GST services to the Authentication services for NetWorker expired.

Resolution

Log in to the NetWorker Management Console as administratorClick the Setup Window iconOn the toolbar, click SetupFrom the Setup menu, select Configure Service Account This launches the Configure Service Account Click Ok, which prompts back to the Setup Window. Nothing is returned when the NMC server updates the account with new credentials.Check the gstd.raw log file, and the messages are stopped.The Show Messages for the policy is now available.

Support Cases