Loading...
Loading...
Avamar Maintenance activities ( checkpoint , checkpoint validation, (aka hfscheck ) and garbage collection, (aka GC) report failures with MSG_ERR_DDR_ERROR: This output can be seen in the output of the status.dpn command: Checkpoint failed with result MSG_ERR_DDR_ERROR : cp.20140626152425 started Thu Jun 26 11:24:26 2014 ended Thu Jun 26 11:26:18 2014, completed 40887 of 40888 stripes Last GC: failed with result MSG_ERR_DDR_ERROR Thu Jun 26 08:01:44 2014 after 46m 42s >> recovered 49.44 GB Last hfscheck failed with result MSG_ERR_DDR_ERROR : started Thu Jun 26 10:12:15 2014 ended Thu Jun 26 11:24:23 2014, nodes queried/replied/total(3/3/3) Note: Depending on when the issue occurred, not all maintenance activities show as failed. The following events are seen in the ddrmaint.log ( /usr/local/avamar/var/ddrmaint.log *): grep -i error /usr/local/avamar/var/ddrmaintlogs/ddrmaint.log May 7 08:00:48 avamar ddrmaint.bin[6890]: Error: gc-finish::process_all_backups: DDR_ReadDir: Failed for client dir 'cur/8b442cbcc86665ff42a2642ff84aa9c5c6dcee3e', ddr result code: 5005, desc: no room left May 7 08:00:48 avamar ddrmaint.bin[6890]: Error: <4710>Datadomain garbage collect operation failed. May 7 08:01:15 avamar ddrmaint.bin[7651]: Warning: Calling DDR_CREATE_SNAPSHOT returned result code:5009 message:I/O error May 7 08:01:15 avamar ddrmaint.bin[7651]: Error: cp-create::execute_create_checkpoint - Failed to create checkpoint for avamar-1480467655 to snapshot cp.20170507150049 on test-dd.emc.edu(1), DDR result code: 5009, desc: I/O error May 7 08:01:15 avamar ddrmaint.bin[7651]: Error: <4760>Datadomain checkpoint create operation failed. May 7 08:06:12 avamar ddrmaint.bin[13088]: Error: hfscheck-finish::createFileOnDDR - failed opening ddr stream to write file VALIDATED/cp.20170506150629/checkpointvalid, ddr: test-dd.emc.edu, lsu: avamar-1480467655, DDR result code: 5004, desc: nothing matched May 7 08:06:12 avamar ddrmaint.bin[13088]: Error: hfscheck-finish::set_checkpoint_validation failed to create file VALIDATED/cp.20170506150629/checkpointvalid on test-dd.emc.edu(1), lsu: avamar-1480467655 May 7 08:06:12 avamar ddrmaint.bin[13088]: Error: <4700>Datadomain hfscheck operation failed. May 7 08:06:52 avamar ddrmaint.bin[15379]: Warning: Calling DDR_CREATE_SNAPSHOT returned result code:5009 message:I/O error May 7 08:06:52 avamar ddrmaint.bin[15379]: Error: cp-create::execute_create_checkpoint - Failed to create checkpoint for avamar-1480467655 to snapshot cp.20170507150630 on test-dd.emc.edu(1), DDR result code: 5009, desc: I/O error May 7 08:06:52 avamar ddrmaint.bin[15379]: Error: <4760>Datadomain checkpoint create operation failed.
The Data Domain (DD) has reached capacity.
Create a Service Request with the Data Domain Technical Support Team to assist. See Data Domain: Scheduling Cleaning on a DDR for more information about scheduling cleaning. Note: While Data Domain cleaning may correct the issue, it may be necessary to access the reserved blocks to free up additional space before cleaning is run.
Click on a version to see all relevant bugs