This procedure describes how to remove OSDs from a Ceph cluster. Once the OSDs are removed, the cluster is also rebalanced to account for the changes. Use this procedure to reduce the size of a cluster or to replace hardware.
This procedure requires administrative privileges.
(ncn-m001#
) Monitor the progress of the OSDs that have been added.
ceph -s
(ncn-m001#
) View the status of each OSD and see where they reside.
ceph osd tree
(ncn-m001#
) Reweigh the OSDs being removed to rebalance the cluster.
The first two substeps below can be skipped if there is a down drive and OSD.
Change the weight of the OSD being removed to 0.
The <OSD_ID>
value should be replaced with the ID of the OSD being removed. For example, if the ID is osd.1
, the <OSD_ID>
value would be 1 in the command below.
ceph osd reweight osd.<OSD_ID> 0
Change the weight in the CRUSH map to 0.
ceph osd crush reweight osd.<OSD_ID> 0
Prevent the removed OSD from getting marked up.
ceph osd set noup
(ncn-m001#
) Remove each OSD after the reweighing work is complete.
Take down the OSD being removed.
ceph osd down osd.<OSD_ID>
Destroy the OSD.
ceph osd destroy osd.<OSD_ID>
Remove the OSD authentication key.
ceph auth rm osd.<OSD_ID>
Remove the OSD.
ceph osd rm osd.<OSD_ID>
Remove the OSD from the CRUSH map.
ceph osd crush rm osd.<OSD_ID>
(ncn-s#
) Remove references to the OSDs on the storage node or nodes they were located on.
The following commands must be run on the storage nodes that held the OSDs being removed.
NOTE
Be sure to replace the <CLUSTER_ID>
field in the following command with the actual Ceph cluster ID.
cd /var/lib/ceph/<CLUSTER_ID>
rm -rf osd.<OSD_ID>
(ncn-m001#
) Clear the flags that were set earlier in the procedure.
ceph osd unset noup
(ncn-m001#
) Monitor the cluster until the rebalancing is complete.
ceph -s