Loading...
Loading...
The rendered daemon.raw of the NetWorker Server will look like this when the condition occurs. 10/11/2021 02:18:00 PM nsrd NSR info unity.delltechnologies.com:/saveset/name saving to pool 'NDMPBCKPSPOOL' (560056) 10/11/2021 02:18:00 PM nsrmmd#138 NSR warning ndmp header: I/O error 10/11/2021 02:18:00 PM nsrd NSR info media warning: rd=unity.delltechnologies.com:c64t0l3 (NDMP) writing: Input/output error, at file 1023 record 1 10/11/2021 02:18:00 PM nsrd NSR info Media Notice: LTO Ultrium-5 tape 560056 on rd=unity.delltechnologies.com:c64t0l3 (NDMP) is full 10/11/2021 02:18:00 PM nsrd NSR info Media Notice: LTO Ultrium-5 tape 560056 used 1004 GB of 1500 GB capacity 10/11/2021 02:18:00 PM nsrd NSR info Media Notice: NetWorker media: (Warning) 560056 marked full prematurely. Verify possible error on the device rd=unity.delltechnologies.com:c64t0l3 (NDMP), advertised capacity is 1500 GB marked full at 1004 GB 10/11/2021 02:18:00 PM nsrmmd#138 NSR notice 10/11/21 14:18:00.270807 NDMP Service Debug: The process id for NDMP service is 0x1ccca778 10/11/2021 02:18:00 PM nsrmmd#138 SYSTEM critical An I/O error has been detected. This could be due to device block size mismatch in the NetWorker configuration and the device. 10/11/2021 02:18:00 PM nsrd NSR info Write completion Notice: Writing to volume 560056 completed 10/11/2021 02:18:00 PM nsrd NSR info unity.delltechnologies.com:/saveset/name done saving 10/11/2021 02:18:00 PM nsrd NSR info : done saving The NDMP backup log will look like this when the condition occurs. 42909:nsrndmp_save: Performing DAR Backup.. 83320:nsrndmp_save: Performing incremental backup, BASE_DATE = 44583379817 42785:nsrndmp_save: Performing backup to NDMP type of device 174908:nsrndmp_save:Saving the backup data in the pool 'NDMPBCKPSPOOL'. 175019:nsrndmp_save:Received the media management binding information on the host 'nwserver.delltechnologies.com'. 174910:nsrndmp_save:Connected to the nsrmmd process on the host 'nwserver.delltechnologies.com'. 175297:nsrndmp_save: Unable to set up the direct save with server 'nwserver.delltechnologies.com': XDR flush error from ndmp header: -999. 42786:nsrndmp_save: NSR server nwserver.delltechnologies.com: XDR flush error from ndmp header: -999 5766:nsrndmp_save: waiting 30 seconds then retrying 174908:nsrndmp_save:Saving the backup data in the pool 'NDMPBCKPSPOOL'. 175019:nsrndmp_save:Received the media management binding information on the host 'nwserver.delltechnologies.com'. 174910:nsrndmp_save:Connected to the nsrmmd process on the host 'nwserver.delltechnologies.com'. 175297:nsrndmp_save: Unable to set up the direct save with server 'nwserver.delltechnologies.com': retry needed. 42786:nsrndmp_save: NSR server nwserver.delltechnologies.com: retry needed 5766:nsrndmp_save: waiting 30 seconds then retrying 174908:nsrndmp_save:Saving the backup data in the pool 'NDMPBCKPSPOOL'. 175019:nsrndmp_save:Received the media management binding information on the host 'nwserver.delltechnologies.com'. 174910:nsrndmp_save:Connected to the nsrmmd process on the host 'nwserver.delltechnologies.com'. 175297:nsrndmp_save: Unable to set up the direct save with server 'nwserver.delltechnologies.com': retry needed. 42786:nsrndmp_save: NSR server nwserver.delltechnologies.com: retry needed 5766:nsrndmp_save: waiting 30 seconds then retrying 7136:nsrndmp_save: (interrupted), exiting
Inconsistent block sizes are utilized between NetWorker and Unity. By default, the NetWorker NDMP tape devices are set to use "handler default" and the Unity uses readWriteBlockSizeInKB=64 and bufsz=256 which means the backup will write NDMP data blocks to the tape in 64kB blocks. It is recommended to always use these default settings where possible. In the particular case that prompted creation of this knowledgebase article, NetWorker NDMP tape devices were set to use "handler default" and the Unity used readWriteBlockSizeInKB=128 and bufsz=128 , this resulted in the undesired behaviour because NetWorker was still configured to utilize 64kB blocks whereas in comparison the Unity was configured to utilize 128kB blocks.
After setting the readWriteBlockSizeInKB and bufsz on the Unity back to their original default values readWriteBlockSizeInKB=64 and bufsz=256 and rebooting the Unity, normal functionality resumed. It is possible to use a larger block size than the default 64kB and the correct way to do this is mentioned below. 64kB block size (default behaviour): NetWorker NDMP tape devices are set to use "handler default" + Unity client instance in NetWorker does not have NDMP_AUTO_BLOCK_SIZE=Y appended to it's Application Information field + the Unity itself uses readWriteBlockSizeInKB=64 and bufsz=256 (and full reboot of Unity after changing).128kB block size: NetWorker NDMP tape devices are set to use "128kB" + Unity client instance in NetWorker has NDMP_AUTO_BLOCK_SIZE=Y appended to it's Application Information field + the Unity itself uses readWriteBlockSizeInKB=128 and bufsz=256 (and full reboot of Unity after changing).256kB block size: NetWorker NDMP tape devices are set to use "256kB" + Unity client instance in NetWorker has NDMP_AUTO_BLOCK_SIZE=Y appended to it's Application Information field + the Unity itself uses readWriteBlockSizeInKB=256 and bufsz=512 (and full reboot of Unity after changing). To check what are the readWriteBlockSizeInKB and bufsz settings on the Unity are set to, the following commands may be used. svc_nas ALL -param -facility NDMP -list svc_nas ALL -param -facility PAX -info readWriteBlockSizeInKB
Click on a version to see all relevant bugs
Dell Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.