Symptoms

VxVerify on VxRail Manager can attempt to correct iDRAC and iSM fault by restarting iDRAC and related VxRail node services.Before running tests directly on each node, using VxVerify minion, VxVerify on VxRail Manager first queries the Dell iSM (dcism or dellism).Alternatively, if iDRAC issues were found when running health-checks, this Auto-fix is attempted before retrying the health-checks.If the Auto-fix option is enabled (either by the test profile or with argument --fix), the attempt to correct this takes around 10 minutes.The result of this autocorrection is listed as one of the following: Test Result Result code Result Interpretation Pass 0 Correcting iSM status was either unnecessary or not enabled under the test profile. Warning 1 Dell iSM status was running correctly after the restart.Failure2 Dell iSM and iDRAC were restarted, but iSM was still not running correctly afterwards. Critical 3 This test has no critical result. This fix can also be triggered after the VxVerify checks run, if iDRAC queries fail. In that case, the VxVerify minion will be run for a second time after the fix. The repeated tests check whether iSM and iDRAC issues are fixed.Each test that passes is not listed in the summary report, for ease of reading.An example of the health-check output is shown below: #========================#======#=========#====================================================================#==============# | Hostname / Category |Status Dell_KB | Warnings or Failures, unless tests Passed ; Product S.N. | #========================#======#=========#====================================================================#==============# | _cluster | Warning 205179 | ism_fix: iSM and iDRAC fixed for node1.lab.local, node4.lab.local .| | `` | Warning 205179 | rac_fix: iSM and iDRAC fixed for node2.lab.local | The 'ism_fix' operation runs prior to the minions and the fix commands are run remotely from VxRM using SSH. For example: Running VxVerify 3.21.108, pre-upgrade healthcheck on VxRail 7.0.372. In case of program errors consult article https://www.dell.com/support/kbdoc/000066460. Step 1: Fixing iSM issue, prior to running health-checks, on node: lab-08-esxi-01.lab.local Step 1: Fixing iSM issue, prior to running health-checks, on node: lab-08-esxi-02.lab.local Step 1: Stopping ISM and platform service on lab-08-esxi-01.lab.local Step 1: Stopping ISM and platform service on lab-08-esxi-02.lab.local Step 1: Pausing for 266 seconds more after iDRAC restarted on ['lab-08-esxi-01.lab.local', 'lab-08-esxi-02.lab.local'] ... Step 1: Starting iSM on lab-08-esxi-01.lab.local Step 1: Starting iSM on lab-08-esxi-02.lab.local Step 1: Pausing for 84 seconds more after Dell iSM started on ['lab-08-esxi-01.lab.local', 'lab-08-esxi-02.lab.local'] ... Step 1: Starting Platform service on lab-08-esxi-01.lab.local Step 1: Starting Platform service on lab-08-esxi-02.lab.local The Autofix can also be seen in the vxv.log prior to the minion_run events: 2022-11-11 09:51:26-INFO [ism_fix] Fixing phase 1 Dell ISM on node on lab-08-esxi-01.lab.local 2022-11-11 09:51:31-INFO [ism_fix] lab-08-esxi-01.lab.local Auto-fix continuing with vSAN objecthealth: green 2022-11-11 09:51:32-INFO [ism_fix] iDRAC restarting on lab-08-esxi-01.lab.local: _ ... 2022-11-11 09:58:58-INFO [ism_fix] Checking hosts for auto-fix success: ['lab-08-esxi-01.lab.local', 'lab-08-esxi-02.lab.local']

Cause

To correct dcism not running, the following steps are by VxVerify, if this autoremediation feature is enabled in the test profile: Stop services: sfcbd, dcism, PTAgent (if present) and Platform-serviceRestart iDRAC, then wait 5 minutes for iDRAC to come back onlineStart services (listed above)

Resolution

The Autofix for iSM reports its success or failure depending on the 'dcism' or 'dellism' status, when polled remotely by VxVerify The minions are then be started as normal.The iSM status is retested using the dcism health-check directly on that node. This can report a different result, because this is polled a few minutes after the Autofix. If the result does differ, the dcism test should be viewed as the more accurate result for the status of iSM.The results of the commands to start the services can be found in the vxv.log. See article VxRail: Troubleshooting issues with VxVerify prior to a VxRail upgrade (Login with support account required) 2022-11-25 09:16:26-DEBUG [ism_fix] node-04.lab.local iSM start: _ 2022-11-25 09:18:26-DEBUG [ism_fix] node-04.lab.local Platform service start: Starting Platform Service Daemon. Check hostd status. hostd is ready. Platform Service started. 2022-11-25 09:18:26-INFO [ism_fix] Checking hosts for auto-fix success: ['node-04.lab.local'] 2022-11-25 09:18:26-INFO [ism_check] Querying DC or Dell ISM status on host 2022-11-25 09:18:26-INFO [ism_check] iSM status on node-04.lab.local : iSM is active (running) If iSM cannot be fixed by the steps above, which the health-check can run automatically, then see article: VxRail: Node Health-check Fails for Test dcism

Support Cases