Symptoms

Attempting to create distributed devices or attaching remote mirror to local device failed with the following message: One or more clusters has no room to allocate bitmap log settings for the raid-1 distributed-device. cause: Unable to attach '' to device ''.cause: Unable to attach mirror '' to distributed Device ''.cause: Firmware command error.cause: One or more clusters has no room to allocate bitmap log settings for the raid-1 distributed-device. Attempting to expand a virtual volume failed with the following message: “Failed to expand virtual-volume ‘VPLEX_SLR_MFS05_F’ Unable to find a logging volume at cluster ‘cluster-1’ with enough free space to expand distributed device ‘dd_VPLEX_SLR-MFS05_F’.”

Cause

There is not enough space on the logging volumes.

Resolution

Need to create a new logging volume with higher capacity and move all distributed devices from the existing logging volume to the new logging volume.This procedure has been broken down into three sections: Section-A: Create logging volume with higher capacitySection-B: Move a logging volumeSection-C: Destroy the old logging volume Section-A To create a logging volume with higher capacity. 1. Login into the VPLEX management server using the service account credentials and then access the VPlexcli.2. From the VPlexcli main prompt check for unclaimed storage-volumes using the following command (approximately 10 GB of logging volume space for every 160 TB of distributed devices): ll /**/storage-volumes In the output look in the column "Use" for "unclaimed" for both clusters: Sample Output: VPlexcli:/> ll /**/storage-volumes/clusters/cluster-1/storage-elements/storage-volumes:Name VPD83 ID Capacity Use Vendor IO Type Thin Provision Thin---------------------------------------- ---------------------------------------- -------- --------- ------ Status ----------- Rebuild Type Capable---------------------------------------- ---------------------------------------- -------- --------- ------ ------ ----------- ------- --------- -------C1_Log VPD83T3:60060160133027006cb4ae275e86e611 10G used DGC alive normal true legacy falseC1_VNX1539_DR1_a8f VPD83T3:60060160c9c02c00a8f89be88ac4e711 10G used DGC alive normal true legacy trueC1_VNX1539_DR1_aaf VPD83T3:60060160c9c02c00aaf89be88ac4e711 10G used DGC alive normal true legacy trueC1_thin_rebuild_svol VPD83T3:6006016013302700e0a4f00e6986e611 10G used DGC alive normal true legacy false..VPD83T3:60060160c9c02c00520c47ef8ac4e711 VPD83T3:60060160c9c02c00520c47ef8ac4e711 10G unclaimed DGC alive traditional false legacy trueVPD83T3:60060160c9c02c00805aba254b99e711 VPD83T3:60060160c9c02c00805aba254b99e711 80G claimed DGC alive normal false legacy falseVPD83T3:60060160c9c02c00a4f89be88ac4e711 VPD83T3:60060160c9c02c00a4f89be88ac4e711 10G unclaimed DGC alive traditional false legacy trueVPD83T3:60060160c9c02c00a6f89be88ac4e711 VPD83T3:60060160c9c02c00a6f89be88ac4e711 10G unclaimed DGC alive traditional false legacy trueVPD83T3:60060160c9c02c00c47cb55a4a99e711 VPD83T3:60060160c9c02c00c47cb55a4a99e711 80G meta-data DGC alive traditional false legacy false 3. From each cluster where you wish to claim storage-volumes, change directory (cd) to the storage-volume context level and claim the desired storage-volumes (approximately 10 GB of logging volume space for every 160 TB of distributed devices), using the following the command: storage-volume claim --name -d Sample output: VPlexcli:/clusters/cluster-1/storage-elements/storage-volumes> storage-volume claim --name Test_LogA -d VPD83T3:60060160133027004636740a4e5be611Claimed 1 of 1 storage-volumes. Repeat step 2 for the other desired storage volumes you wish to claim to be used for the new logging volume.4. For those storage volumes you claimed create an extent for each using the following the command: extent create Sample Output: VPlexcli:/clusters/cluster-1/storage-elements/storage-volumes> extent create Test_Log Repeat step 3 for the remaining claimed storage volumes.5. Next create the new logging volume on top of that extents using the following command: logging-volume create -n -g raid-1 e Sample output: VPlexcli:/clusters/cluster-1/storage-elements/storage-volumes> logging-volume create -n c1_New_log -g raid-1 e Test_LogA, Test_LogB 6. List the newly created logging volume using the following command: ll /clusters/cluster-1/system-volumes/ Sample Output: VPlexcli:/clusters/cluster-1/storage-elements/storage-volumes> ll /clusters/*/system-volumes /clusters/cluster-1/system-volumes:Name Volume Type Operational Health Active Ready Geometry Component Block Block Capacity Slots------------------------------- -------------- Status State ------ ----- -------- Count Count Size -------- ------------------------------------ -------------- ----------- ------ ------ ----- -------- --------- -------- ----- -------- -----C1__Logging_vol logging-volume ok ok - - raid-1 1 2621440 4K 10G -C1_New_Log_vol logging-volume ok ok - - raid-1 1 2621440 4K 10G -C1_Meta meta-volume ok ok true true raid-1 2 20971264 4K 80G 64000C1_Meta_backup_2018Apr10_120032 meta-volume ok ok false true raid-1 1 20971264 4K 80G 64000C1_Meta_backup_2018Apr11_120018 meta-volume ok ok false true raid-1 1 20971264 4K 80G 64000Repeat all steps in Section A for cluster-2. Section: BMove a logging volume.The following procedure moves the bit map of distributed-devices from an existing logging volume ( c1_old_vol ) to new logging-volume ( c1_New_log_vol ).1. Follow step #6 in section: A to get the logging volume details: VPlexcli:/clusters/cluster-1/system-volumes> ll 2. Verify that the new logging-volume, c1_New_Log_vol, is not being used by distributed-devices by looking to ensure under the column 'Use' it reads "free': VPlexcli:/clusters/cluster-1/system-volumes/c1_New_log_vol/segments> llName Starting Block Block Count Use------ -------------- ----------- ----free-0 0 2623200 free 3. Verify that the old logging-volume, C1_Logging_vol, is being used by distributed-devices: VPlexcli:/clusters/cluster-1/system-volumes/C1_Logging_vol/segments> ll Sample output: VPlexcli:/clusters/cluster-1/system-volumes/C1_Logging_vol/segments> ll/clusters/cluster-1/system-volumes/C1_Logging_vol/segments:Name Starting Block Use----------------------------------------------------- Block Count ------------------------------------------------------------------------------------------------ -------- ------- -------------------------------------------allocated-device_C1_VNXXX12_DR1_XXf_1 0 80 allocated for device_C1_VNXXX12_DR1_XXf_1allocated-device_C2_VNXXX12_DR1_X2X_12017Nov20_165905 0 80 allocated for device_C2_VNXXX12_DR1_X2X_12017Nov20_165905allocated-device_local_source_12018Mar05_063808 0 80 allocated for device_local_source_12018Mar05_063808allocated-device_remote_target_1 0 80 allocated for device_remote_target_1free-320 4. Choose the new logging volume for all the distributed-devices in the system using the following command: set-log --logging-volumes --distributed-devices * Sample output:VPlexcli:/distributed-storage/distributed-devices> set-log --logging-volumes c1_New_log_vol --distributed-devices * 5. Check that the logging-volume is allocated for the distributed-devices: VPlexcli:/clusters/cluster-1/system-volumes/c1_New_vol/segments> ll 6. Check that the original logging-volume is not being used by any distributed-devices: VPlexcli:/clusters/cluster-1/system-volumes/C1_Logging_vol/segments> llName Starting Block Block Count Use------ -------------- ----------- ----free-0 0 2623200 free Repeat all steps in Section B for cluster-2.Section CTo destroy the old logging volume.1. To see the details of the new logging volume run the following command from the VPlexcli:show-use-hierarchy /clusters/cluster-1/storage-elements/extents/ orshow-use-hierarchy clusters/cluster-1/system-volumes/ Sample output using the extent name: VPlexcli:/> show-use-hierarchy /clusters/cluster-1/storage-elements/extents/extent_C1_Log_1 logging-volume: C1_New_Logging_vol (20G, raid-0, cluster-1) extent: C1_New_Log_1 (10G) storage-volume: C1_New_Log(10G) logical-unit: VPD83T3:60000970000298701473533030354144 storage-array: EMC-SYMMETRIX-123456789 extent: C1_New_Log_2(10G) storage-volume: New_Log2(10G) logical-unit: VPD83T3:60000970000298701473533030354145 storage-array: EMC-SYMMETRIX-987654321 2. Destroy the old logging-volume using the following command:logging-volume destroy --logging-volume Sample output: VPlexcli:/clusters/cluster-1/system-volumes/C1_Log_vol> logging-volume destroy --logging-volume C1_Log_vol Context '/clusters/cluster-1/system-volumes/C1_Log_vol' has been removed. 3. Destroy the extents behind the logging-volume using the following command:extent destroy --extents extent Sample output: VPlexcli:/clusters/cluster-1/storage-elements/extents/C1_Log_1> extent destroy --extents extent_C1_Log_1WARNING: The following items will be destroyed:Context-----------------------------------------------------------------------------------/clusters/cluster-1/storage-elements/extents/extent_C1_Log_1Do you wish to proceed? (Yes/No) YesExtent extent_C1_Log_1' is claimed and may be in use, do you wish to destroy it anyway? (Yes/No) YesContext '/clusters/cluster-1/storage-elements/extents/extent_C1_Log_1' has been removed.Destroyed 1 out of 1 targeted extents. Repeat all steps in Section C for cluster-2.

Support Cases