Symptoms

Customer backup server reported a tape close error with no tape opened.Example From VNX server log:Messages will with streaming if the same issue is encountered. 2018-01-02 05:51:28: NDMP: 3: < LOG type: 2, msg_id: 0, entry: Tape close error with no tape opened, hasAssociatedMsg: 0, associatedMsgSeq: 0 > 2018-01-02 05:59:37: NDMP: 3: last message repeated 12 times Backup server stops responding and fails to start backup as tape cannot be loaded.Depending on the backup server, it should also report NDMP_DEVICE_BUSY_ERR when trying to open SCSI devices zoned to VNX. Check the DM Bind table to confirm which DMs have AUX 1&2 zoned to tape library storage. # .server_config ALL -v "fcp bind show" The example below shows Server_3 is activated/failover to slot_4. Both server_3 and server_4 have been zoned to TLUs. server_3.faulted.server_4 : commands processed: 1 command(s) succeeded output is complete *** Persistent Binding Table *** Chain 0000: WWN xxxxxxxxxxxxxxxx HBA 0 SP-a09 Bound Chain 0016: WWN xxxxxxxxxxxxxxxx HBA 1 SP-b09 Bound Chain 0032: WWN xxxxxxxxxxxxxxxx HBA 2 N_PORT Bound Chain 0048: WWN xxxxxxxxxxxxxxxx HBA 3 N_PORT Bound Existing CRC: 58959ae9, Actual: 58959ae9, CRC Matchs *** Dynamic Binding Table *** Chain 0000: WWN xxxxxxxxxxxxxxxx HBA 0 ID 0 Inx 00:00 Pid 0000 D_ID 0000ef Sys Chain 0016: WWN xxxxxxxxxxxxxxxx HBA 1 ID 1 Inx 01:00 Pid 0016 D_ID 0000ef Sys Chain 0032: WWN xxxxxxxxxxxxxxxx HBA 2 ID 2 Inx 02:00 Pid 0032 D_ID 064700 Non >>>> standby DM is zonned to TLUs Chain 0048: WWN xxxxxxxxxxxxxxxx HBA 3 ID 3 Inx 03:00 Pid 0048 D_ID 054600 Non >>>> standby DM is zonned to TLUs FCP Base Chain: 0 Dump Slot: 3 Dump Chain: 0 16 Adapter Chain Offset 0:0 1:16 2:32 3:48 dumpInit 1 1515252291: ADMIN: 6: Command succeeded: fcp bind show server_3 : commands processed: 1 command(s) succeeded output is complete *** Persistent Binding Table *** Chain 0000: WWN 500xxxxx HBA 0 SP-a10 Bound Chain 0016: WWN 500xxxxx HBA 1 SP-b10 Bound Chain 0032: WWN 500xxxxx HBA 2 N_PORT Bound Chain 0048: WWN 500xxxxx HBA 3 N_PORT Bound Existing CRC: 516986aa, Actual: 516986aa, CRC Matchs *** Dynamic Binding Table *** Chain 0000: WWN 500xxxxx HBA 0 ID 0 Inx 00:00 Pid 0000 D_ID 0000ef Sys Chain 0016: WWN 500xxxxx HBA 1 ID 1 Inx 01:00 Pid 0016 D_ID 0000ef Sys Chain 0032: WWN 500xxxxx HBA 2 ID 2 Inx 02:00 Pid 0032 D_ID 064500 Non Chain 0048: WWN 500xxxxxHBA 3 ID 3 Inx 03:00 Pid 0048 D_ID 054700 Non FCP Base Chain: 0 Dump Slot: 4 Dump Chain: 0 16 Adapter Chain Offset 0:0 1:16 2:32 3:48 dumpInit 1 1515252246: ADMIN: 6: Command succeeded: fcp bind show View all the zoned tapes using, #server_devconfig ALL -l -s -n | more Confirm if any non-disk report from standby DM, if yes then this KBA should be applied.

Cause

The standby Data Mover was zoned for Two (2) way backup. This caused tape contention resulting in the backup to fail.Issue occurs due to tape contention, when Data mover attempts to request tape, the zoned standby DM that is not in use it will not release the tape. The standby Data Mover is only for data mover redundancy and is a temporary solution to run the active data mover on standby However, for any reason, if you have to run on the standby Data Mover, you have to connect the cables, zone them accordingly and configure the same.

Resolution

Unzone the tapes from standby Data Mover. Once complete performing a scsi re-scan on standby DM, then confirm that the tapes have been removed. # server_devconfig ALL -c -s -n List all non-disk again to confirm standby DM does not see the available tapes. # server_devconfig ALL -l -s -n | more The following VNX2 NDMP configuration guide outlines the issue.https://support.emc.com/docu48452_Configuring-NDMP-Backups-on-VNX-8.1.pdf?language=en_USPage 13If an NDMP-Host Data Mover fails over to its standby, you must physically connect the Data Mover's TLU cable to the standby, and restart the backup. You could also have to reconfigure the backup software with the appropriate address. The following Celerra NDMP configuration (legacy VNX array, but same restriction with DART code)https://support.emc.com/docu10558_Configuring-NDMP-Backups-on-Celerra.pdf?language=en_USPage 13Do not connect a standby Data Mover to the TLU by a SCSI connection. This causes the backup to fail and could cause tape contention. The standby DM cannot be zoned as standby Data mover does not have a public network interface to control the Data mover's tape drives. When tapes/TLUs are zoned to standby Data mover, the active data mover is not able to use the tape drive as the standby mover is not listening to scsireservation messages to release or not touch the tapes.

Support Cases