Symptoms

The following event can be logged as a Dial Home, a UI event or seen on the command line on the Avamar Utility Node. Management Console Server (MCS) UI Event: <4004> hfscheck of cp.xxxxxxxxxxxxxx failed on error: MSG_ERR_NO_CHECKPOINT Output from the status.dpn command: Last hfscheck failed with result MSG_ERR_NO_CHECKPOINT

Cause

No checkpoint ID was specified or there are no unvalidated checkpoints found.Common causes:1. Avamar Data Nodes were recently added to the Avamar grid, and a new checkpoint was not taken after the node addition before running a checkpoint validation (hfscheck).2. A checkpoint was not able to start during the normal maintenance window and hfscheck tried to start. As there was no new checkpoint, hfscheck failed.3. An hfscheck was started manually (ad hoc) and there was no new checkpoint available to validate.4. The Avamar server was recently upgraded, and a new checkpoint was not taken following the upgrade. A checkpoint validation was trying to run against an already validated checkpoint.5. The ddboost password has expired, the ddboost account has been locked for an extended period of time, or data domain snapshots are not being removed. This can cause too many checkpoints to exist in /data directories, which in turn, causes avmaint to fail to identify valid checkpoints, even when they exist.

Resolution

1. Verify the issue: a. Log in to Avamar Utility Node or single node grid as admin. b. If working on a multi-node grid, load the admin keys per Avamar: How to Log in to an Avamar Server and Load Various Keys. c. Get a list of the current checkpoints: cplist --full d. If the output is similar to the following (with invalid checkpoints), Create a Service Request for the Dell Technologies support team to assist with manual deletion (referencing this article). Invalid checkpoint example: cp.20220727130327 Wed Jul 27 08:03:27 2022 invalid --- del nodes 1/1 stripes 721 cp.20220728130240 Thu Jul 28 08:02:40 2022 invalid --- del nodes 1/1 stripes 721 cp.20220728130332 Thu Jul 28 08:03:32 2022 invalid --- del nodes 1/1 stripes 721 cp.20220729130237 Fri Jul 29 08:02:37 2022 invalid --- del nodes 1/1 stripes 721 cp.20220729130330 Fri Jul 29 08:03:30 2022 invalid --- del nodes 1/1 stripes 721 cp.20220729180313 Fri Jul 29 13:03:13 2022 valid --- --- nodes 1/1 stripes 721 cp.20220729180814 Fri Jul 29 13:08:14 2022 valid --- --- nodes 1/1 stripes 721 (typically, there would be far more invalid entries): e. Determine when the issue started and what occurred on the grid around that time. (Such as a node addition, upgrade, and so on.) f. Resolve any outstanding issues that are contributing to the cause of the issue. 2. Stop the maintenance scheduler and take a new checkpoint: dpnctl stop maint avmaint checkpoint --ava 3. Once the checkpoint has finished, start a checkpoint validation (hfscheck): avmaint hfscheck --rolling --ava (it takes a few minutes for the prompt to return after running the following command.) 4. Confirm that hfscheck has started: status.dpn |grep hfscheck avmaint hfscheckstatus 5. Allow hfscheck to complete. 6. Restart the maintenance scheduler: dpnctl start maint 7. If the problem persists, Create a Service Request with the Dell Technologies support team for assistance.

Support Cases