Loading...
Loading...
NDMP backups fail when the dataset contains a large number of volumes. For example, the failure can occue when the dataset contains approximately 300 volumes. The following errors appear in the avtar logs: 2016/04/11-08:44:07.52813 [avtar] sysfile::open open64 failed: code 2: No such file or directory 2016/04/11-08:44:07.52814 [avtar] filedisk::doopen, fdesc == -1 after open attempt 2016/04/11-08:44:07.52814 [avtar] Problem reading file (attempt#0), sleeping 30 sec and trying again 2016/04/11-08:44:25.01582 [ade-async] servercontext::sendPingToAll(): dpninx=0, connections=1 .... 2016/04/11-08:58:40.28278 [avtar] sysfile::open open64 failed: code 2: No such file or directory 2016/04/11-08:58:40.28279 [avtar] filedisk::doopen, fdesc == -1 after open attempt 2016/04/11-08:58:40.28280 [avtar] Problem reading file (attempt#29), sleeping 30 sec and trying again .....2016/04/11-08:59:10.38269 [avtar] Could not open file '<vadir>/<clientname>/<volume_name>.comment' (waited 30 secs) 2016-04-11 04:59:10 avtar Error <40091>: Comment entry "comment" file "<vadir>/<clientname>/<volume_name>.comment" not found.
This issue occurs when the dataset contains a large number of volumes. During the backup, Avamar performs the following processing: Avamar builds a composite tree of directory elements for the top-level backup structure. Avamar searches the composite tree to locate a valid base backup for each volume. For each volume, Avamar searches up to 30 recent backups for a valid base backup. As the backup progresses, the following conditions occur: The time that Avamar requires to find a valid base increases. For volumes that Avamar processes later in the dataset, the search for a valid base backup can take 3 to 4 minutes per volume. While Avamar performs the extended search, the following conditions occur: Other "avndmp" threads cannot run. The threads that write the ".comment" files for completed volumes become blocked. As a result, the blocked threads do not create the ".comment" file before Avamar requires the file. The missing ".comment" file causes the backup to fail.
To limit the incremental search depth, apply the following workaround: Open the "avndmp.cmd" configuration file. Add the following parameter: --limit-incremental-search=10 Save the file. Restart the Avamar NDMP backup process or rerun the backup job. This parameter reduces the number of previous backups that Avamar searches for a valid base backup. With the reduced search depth, backup performance improves. The threads that writes the ".comment" files are no longer blocked.
Click on a version to see all relevant bugs