Symptoms

Identified Symptoms The following symptoms have been identified: Cloud tier cleaning is not in progress.A previous cloud tier cleaning attempt was unsuccessful.Cloud unit "cloud-unit" was disconnected.Cloud cleaning was running, and a connection issue with cloud unit caused a disconnect.Cloud unit was left in a CLEANING state, rather than a READY state.Space issues with the cloud unit.Connection issues with the cloud provider or cloud unit. CLI Output adim@localhost## cloud clean start cloud-unit Cloud tier cleaning started for cloud unit "cloud-unit". Use 'cloud clean watch' to monitor progress. adim@localhost## cloud clean watch Cloud tier cleaning is not in progress. adim@localhost## cloud clean status Previous cloud tier cleaning attempt was unsuccessful. Failure reason: cloud unit "cloud-unit" was disconnected. Additional Log Information 08/15 11:32:45.381226 [7fad623ea7e0] INFO: CAL cal_handle_cloud_profile_status:1962 - Processing status event for profile: cloud-unit_profile, status: 1, error code: 0 08/15 11:32:45.381237 [7fad623ea7e0] INFO: CAL cal_handle_cloud_profile_status:1980 - Found cloud unit:cloud-unit using profile:cloud-unit_profile 08/15 11:32:45.381240 [7fad623ea7e0] INFO: CAL cal_handle_cloud_profile_status:2037 - Marking cloud profile:cloud-unit_profile for cloud unit:cloud-unit as UP 08/15 11:32:45.381242 [7fad623ea7e0] INFO: CAL cal_handle_cloud_profile_status:2080 - Cloud unit:cloud-unit already AVAILABLE 08/15 11:35:11.147451 [7f891defd740] INFO: CC-cpm: cp_name cloud-unit, vol_name cloud1, cp_handle 2, part_type CC_TYPE_CLOUD, part_status CLEANING, delete_credit 1005224616626

Cause

Root Cause of the Issue The cloud unit is disconnected.The cloud unit state was left in CLEANING state after the disconnection issue was resolved. Specific Error Messages and Codes Relevant error messages and codes include: errno: 5160, errstr: We encountered an internal error. Please try again. errno: 5005, errstr: Insufficient storage at the cloud provider. External Factors External factors that might have contributed to the cause include: Insufficient storage at the cloud providerConnection issues with the cloud provider or cloud unit adim@localhost## cloud clean status Previous cloud tier cleaning attempt was unsuccessful. Failure reason: cloud unit "cloud-unit" was disconnected. 08/10 06:05:48.963 [7f1098fd619]: Fmig: fmig2_process_cal_event: cloud-unit (path=cloud1/cp1): receiving DDR_EVENT_CAL_UNIT_UNAVAIL event 08/10 06:05:48.964 [7f1098fd619]: Fmig: setting a FMIG restart on CAL_AVAILABLE event registry key 08/10 06:05:48.968 [7f1098fd619]: Fmig: Abort requested to all the sealing jobs on CP cloud-unit (path=cloud1/cp1) 08/11 10:32:02.529772 [7f7ae3047410] INFO: CAL cal_cloudunit_set_unavail:1784 - Marking cloud unit:cloud-unit as UNAVAILABLE with errno: 5160, errstr: We encountered an internal error. Please try again. 08/11 10:44:18.687575 [7f7ab4191570] INFO: CAL cal_cloudunit_set_unavail:1784 - Marking cloud unit:cloud-unit as UNAVAILABLE with errno: 5005, errstr: Insufficient storage at the cloud provider. 08/15 11:35:11.147451 [7f891defd740] INFO: CC-cpm: cp_name cloud-unit, vol_name cloud1, cp_handle 2, part_type CC_TYPE_CLOUD, part_status CLEANING, delete_credit 1005224616626

Resolution

Restart the file system to Resolve the issue To resolve the issue, it is recommended to restart the file system. This should help clean up the current state and allow the cloud unit to return to a ready state. # Run the following command to restart the DDFS service $ filesys restart After restarting the file system, verify that the cloud unit is in a ready state by checking the status of the cloud unit. # Run the following command to check the status of the cloud unit $ cloud unit list $ cloud clean status # Start cloud cleaning is not already running $ cloud clean start

Support Cases