Symptoms

Issue The Master Control Program (MCP) is not running on one or more nodes in the cluster. Symptoms: Cluster jobs like FlexProtect, QuotaScan, and SnapshotDelete do not run.The /var/log/isi_mcp log file contains entries similar to the following: 2013-10-28T13:21:04+05:30 <3.3> example-1(id1) isi_mcp[4862]: Error while loading /etc/mcp/mlist.xml: [/etc/mcp/mlist.xml] unknown error

Cause

An invalid mlist.xml file for a node can prevent the proper startup of the isi_mcp service.

Resolution

Solution Open an SSH connection on any node in the cluster and log in using the "root" account.Run the following command to verify that isi_mcp is running on all nodes: isi_for_array -s ps auwwwx | grep mcp | grep -v grep Output similar to the following appears: example-2: root 91294 0.0 0.1 53696 7036 ?? Ss 12:32AM 0:00.00 isi_mcp: master (isi_mcp) example-2: root 91295 0.0 0.1 54720 8364 ?? S 12:32AM 0:00.07 isi_mcp: child (isi_mcp) example-3: root 86820 0.0 0.1 53696 7056 ?? Ss 12:32AM 0:00.00 isi_mcp: master (isi_mcp) example-3: root 86821 0.0 0.1 54720 8380 ?? S 12:32AM 0:00.07 isi_mcp: child (isi_mcp) Each node should display a master (isi_mcp) and child (isi_mcp) entry. If it does not, isi_mcp is not running on that node. Attempt to start isi_mcp on the node by running the following command, where <lnn> is the node number where isi_mcp is not running: isi_for_array -n<lnn> isi_mcp Repeat step 2 to confirm that isi_mcp is now running. If it is not, go to the next step.Open an SSH connection to the node where isi_mcp is not running and log in using the "root" account.Run the following command to rename the mlist.xml file: mv /etc/mcp/mlist.xml /etc/mcp/mlist.xml.bak Attempt to start isi_mcp on the node by running the following command: isi_mcp Run the following command to verify that isi_mcp is running on all nodes: isi_for_array -s ps auwwwx | grep mcp | grep -v grep Output similar to the following appears: example-1: root 2686 0.0 0.1 53696 7208 ?? Is 1:09AM 0:00.00 isi_mcp: master (isi_mcp) example-1: root 2687 0.0 0.1 54720 8504 ?? S 1:09AM 0:01.31 isi_mcp: child (isi_mcp) example-2: root 91294 0.0 0.1 53696 7036 ?? Ss 12:32AM 0:00.00 isi_mcp: master (isi_mcp) example-2: root 91295 0.0 0.1 54720 8364 ?? S 12:32AM 0:00.07 isi_mcp: child (isi_mcp) example-3: root 86820 0.0 0.1 53696 7056 ?? Ss 12:32AM 0:00.00 isi_mcp: master (isi_mcp) example-3: root 86821 0.0 0.1 54720 8380 ?? S 12:32AM 0:00.07 isi_mcp: child (isi_mcp)

Support Cases