Symptoms
The NetWorker server intermittently becomes unresponsive through the NetWorker Management Console (NMC), nsrwatch and nsradmin command-line utilities.The NetWorker datazone is configured with many devices. The NetWorker server may have additional storage nodes managing devices; however, there are many devices configured overall. The number of devices is near or exceeding 100.The NetWorker server's ..\nsr\logs\daemon.raw contains many RPC errors regarding the NetWorker port range:
MM/DD/YYYY HH:MM:SS AM 3 2 107 12236 17644 0 NSR_SERVER_NAME nsrmmdbd RPC error Unable to create the connection with 'nsrd' to host 'localhost' with address '127.0.0.1' at port number 9537.
MM/DD/YYYY HH:MM:SS AM 5 2 107 12236 17644 0 NSR_SERVER_NAME nsrmmdbd RPC critical Check whether the client services are running on the host '127.0.0.1'.
NetWorker: How to use nsr_render_log
There are hundreds of nsrmmd processes running on the server. Restarting NetWorker server services temporarily resolves the issue.NetWorker server is installed on Microsoft Windows.
Cause
Due to the number of devices configured through the NetWorker server and server activity port exhaustion is occurring, resulting in RPC connection issues for the NetWorker server services.
Resolution
Adjust TCP keepalive registries on the NetWorker server.
a. On the NetWorker server open regedit and go to: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parametersb. Set KeepAliveTime to 300000 millisecond (5 minutes)c. Set TcpTimedWaitDelay to 30 (decimal)
NOTE: If either of the above changes do not exist on the server, they can be created. Changes to the system registry requires a system reboot.
From the NetWorker Management Console, check the Devices->Storage Node properties of each of the storage nodes that are configured in the NetWorker server. If checked, clear the dynamic nsrmmds checkbox:
NOTE: View->Diagnostic view must be enabled to see the Dynamic nsrmmds checkbox. This is to avoid additional nsrmmd processes getting started and consuming ports when many devices exists in the environment.
