Cloudera Docs

Decommissioning ECS Hosts

You can decommission ECS hosts and remove them from the cluster.

  1. Cordon the node. Longhorn will automatically disable the node scheduling when a Kubernetes node is cordoned. Run the following command on any ECS Server host:
    kubectl cordon [***node***]
  2. Drain the node to move the workload to somewhere else. Run the following command on any ECS Server host:
    kubectl drain [***node***] --ignore-daemonsets --pod-selector='app!=csi-attacher,app!=csi-provisioner' --delete-emptydir-data

  3. Detach all the volumes on the node. Navigate to the ECS Service page on Cloudera Manager UI.

    a. In the Web UI dropdown, select Storage UI to open the Longhorn UI.

    b. Under the Volume tab in Longhorn UI, select the volumes on this node. Click Detach and select Yes on the screen prompt.

    If the node has been drained, all the workloads should be migrated to another node already.

    If there are any other volumes remaining attached, detach them before continuing.

  4. Remove the node from Longhorn using the Delete in the Node tab. Or, remove the node from Kubernetes. Run the following command on any ECS Server host: 
    kubectl delete node [***node-name***]

    Longhorn will automatically remove the node from the cluster.

  5. Uninstall ECS and Docker artifacts from the host. Run below commands on the host:
    cd /opt/cloudera/parcels/ECS/bin
./rke2-killall.sh # usually 2 times is sufficient
./rke2-uninstall.sh
rm -rf /ecs/* # assumes the default defaultDataPath and lsoDataPath
rm -rf /var/lib/docker_server/* # deletes the auth and certs
rm -rf /etc/docker/certs.d/* # delete the ca.crt
rm -rf /docker # assumes the default defaultDataPath for docker
  6. Go to the Hosts page for the ECS Cluster, select that host, and under Actions for Selected, click Begin Maintenance (Suppress Alerts/Decommission)