...
Bulk Migration fails at the time of cutover with the following error message.Error message seen in HCX UI:Can not poweroff VM as guest shutdown timed out and forcePowerOff is not setIn the app.log (/common/logs/admin/app.log), a similar error can be found: 2021-03-02 03:12:07.186 UTC [ReplicationTransferService_SvcThread-65273, Ent: HybridityAdmin, , TxId: e0307ba1-99da-4ee6-823d-94f0a4a331d3] ERROR c.v.h.s.r.j.VirtualmachineOperationJob- Job (b6667a7f-3a90-4ee6-a181-175b2cdda7f2) failed with exception Can not poweroff VM as guest shutdown timed out and forcePowerOff is not set java.lang.RuntimeException: Can not poweroff VM as guest shutdown timed out and forcePowerOff is not set at com.vmware.hybridity.service.replicationtransfer.jobs.VirtualmachineOperationJob.doWaitForPowerOffVm(VirtualmachineOperationJob.java:247) at com.vmware.hybridity.service.replicationtransfer.jobs.VirtualmachineOperationJob.run(VirtualmachineOperationJob.java:59) at com.vmware.vchs.hybridity.messaging.LoggingJobWrapper.run(LoggingJobWrapper.java:41) Note: Dates, Times and UUID's in the log line above are examples.
This article provides information on countermeasure for an issue that prevents switchover during bulk migration of HCX.
This issue happens when a guest OS does not reached a fully powered off state within 100 seconds of the initial "Shutdown Guest OS" call as part of bulk migration cutover.
Without "Force Power Off" selected, VMware HCX Bulk Migration utilizes VMware Tools to gracefully shutdown a guest OS. As part of this mechanism, HCX will wait 100 seconds for the guest OS to be completely powered down. If the 100 second period elapses and the virtual machine does not reflect a powered off status, the migration will fail. If VMware Tools is not detected on the guest OS, pre-migration validation will prompt a user to select "Force Power Off".With "Force Power Off" selected, VMware HCX Bulk Migration still utilizes VMware Tools (if present) to gracefully shutdown a guest OS. After 100 seconds has elapsed since the initial "Shutdown Guest OS" call, an immediate power off will be performed (if necessary) on the virtual machine in order to move forward with the migration. From HCX 4.6.1 release onwards, if the virtual machine does not power off after 100 seconds during the Bulk Migration cut-over stage, HCX waits for an additional 100 seconds to handle any delay in synchronizing with the vCenter Server before failing the migration workflow. Refer to HCX Release Notes.
"Install the VMware Tools package inside this virtual machine" message when powering on a VM (2030189)Understanding VMware HCX Bulk Migration