Loading...
Loading...
VMs with a status of Not Detected are failing on Protection Jobs The VM may be part of a Protection Policy Rule The VM asset was deleted from vCenter, but PowerProtect Data Manager Status is Not Detected when it should be Deleted Error Code: ABV0040 but not limited to this code Backups may fail with the following message: VM Proxy BackupJobSession failed -- Unable to find virtual machine with MORef 'vm-5456'. Virtual machine does not exist with MORef 'vm-5456': ServerFaultCode: The object 'vim.VirtualMachine:vm-5456' has already been deleted or has not been completely created
The change: PowerProtect Data Manager 19.18 includes Not Detected VMs in the protection job, but these assets fail. This behavior is expected. Why was this change implemented? In PowerProtect Data Manager 19.17 and earlier versions, backups would not run for Not Detected VMs. This caused a risk of data loss because the Not Detected status could occur unexpectedly and temporarily. When backups failed for a Not Detected VM, there was no alert or failure to highlight the issue for correction. The Issue encountered as a result of this change: When upgrading to 19.18 from earlier versions, there can be several VMs with a Not Detected status resulting in failed protection jobs. These VMs which were deleted from the vCenter but PowerProtect Data Manager have an incorrect status of Not Detected for these assets instead of Deleted. VM deletion events should be received from vCenter as part of the discovery process. However, sometimes the deletion event is missed by PowerProtect Data Manager. If this happens, the next discovery sets the state of the VM status to Not Detected . There are multiple scenarios in PowerProtect Data Manager where deletion events are missed, leading to VM assets being incorrectly marked as Not Detected . These issues are resolved in versions 19.19.0-20 and 19.20.
Option 1 - Recommended Best Practice: The change in 19.18 to run a backup job against any Not Detected VM is the preferred behavior. The Not Detected state could be temporarily or wrongly reported, this could result in data loss. Best practice is to check each Not Detected VM, confirm it was deleted in vCenter, and remove from PowerProtect Data Manager. If the VM is still active in vCenter, investigate why it has a Not Detected Status in PowerProtect Data Manager. View a list of all Not Detected VMs go to Infrastructure > Assets > Filter by Status > Not Detected . The list can be exported using the Export option if required. Select all VMs with a Not Detected status, that no longer exists in vCenter but are still part of a Protection Policy. Click More Actions > Remove Asset. If any assets still have copies present, the following prompt is seen. Choose your wanted option. Option 2 - Return PowerProtect Data Manager to the default behavior prior to 19.18. The following procedure sets the behavior for Not Detected VMs back to being silently excluded from backups for 19.18.0-23 and later. If your PowerProtect Data Manager is an earlier release of 19.18, best practice would be to upgrade and then apply the option below. If this is not possible, contact Dell Support for an earlier 19.18 workaround. Caution: This is not the preferred default behavior as the Not Detected state could be temporarily or wrongly reported, this could result in data loss. Be fully aware of this risk before making any changes. Option 1 is the recommended procedure. Run the following commands from the PowerProtect Data Manager server: Connect to PowerProtect Data Manager using SSH with admin. Set the token variable: Replace the following parameters to match your environment: <your_hostname> with the FQDN or IP of your PowerProtect Data Manager <your_username> with your PowerProtect Data Manager UI username <your_password> with your PowerProtect Data Manager UI password token=`curl -skX POST https://<your_hostname>:8443/api/v2/login -d '{"username":"<your_username>","password":"<your_password>"}' -H "Content-Type: application/json" | json_pp | grep access_token |cut -d "\"" -f4` Issue a GET request to see the current /api/v2/common-settings/JOB_INTERVENTION_SETTING configuration : Replace the following parameters to match your environment: <your_hostname> with the FQDN or IP of your PowerProtect Data Manager curl -k --request GET --url https://<your_hostname>:8443/api/v2/common-settings/JOB_INTERVENTION_SETTING --header "authorization: $token" | json_pp Example output: Create a json file with the payload and paste the payload below. vim job_intervention_setting_change.json Payload: { "properties": [ { "name": "noProtectJobForNotDetectedAssetTypes", "type": "LIST", "value": "VMWARE_VIRTUAL_MACHINE" } ] } Issue a PUT request to update the parameter to exclude Not Detected VMs from Protection Jobs. Replace the following parameters to match your environment: <your_hostname> with the FQDN or IP of your PowerProtect Data Manager curl -k --request PUT --url https://<your_hostname>:8443/api/v2/common-settings/JOB_INTERVENTION_SETTING --header "authorization: $token" -H "Content-Type: application/json" -d @job_intervention_setting_change.json Confirm the change with the same GET request run in step 3. Example output:
Click on a version to see all relevant bugs
Dell Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.