This procedure describes how to remove an OSD(s) from a Ceph cluster. Once the OSD is removed, the cluster is also rebalanced to account for the changes. Use this procedure to reduce the size of a cluster or to replace hardware.
This procedure requires administrative privileges.
Log in as root
on the first master node (ncn-m001
).
Monitor the progress of the OSDs that have been added.
ceph -s
View the status of each OSD and see where they reside.
ceph osd tree
Reweigh the OSD(s) being removed to rebalance the cluster.
The first two substeps below can be skipped if there is a down drive and OSD.
Change the weight of the OSD being removed to 0.
The OSD_ID value should be replaced with the ID of the OSD being removed. For example, if the ID is osd.1, the OSD_ID value would be 1 in the command below.
ceph osd reweight osd.OSD_ID 0
Change the weight in the CRUSH map to 0.
ceph osd crush reweight osd.OSD_ID 0
Prevent the removed OSD from getting marked up.
ceph osd set noup
Remove the OSD after the reweighing work is complete.
Take down the OSD being removed.
ceph osd down osd.OSD_ID
Destroy the OSD.
ceph osd destroy osd.OSD_ID
Remove the OSD authentication key.
ceph auth rm osd.OSD_ID
Remove the OSD.
ceph osd rm osd.OSD_ID
Remove the OSD from the CRUSH map.
ceph osd crush rm osd.OSD_ID
Remove references to the OSDs on the storage node(s) they were located on.
The following commands must be run on the storage node(s) that held the OSDs being removed.
umount /var/lib/ceph/osd/ceph-OSD_ID
rm -rf /var/lib/ceph/osd/ceph-OSD_ID
Clear the flags that were set earlier in the procedure.
ceph osd unset noup
Monitor the cluster until the rebalancing is complete.
ceph -s