This is the documentation for Cloudera Manager 5.1.x. Documentation for other versions is available at Cloudera Documentation.

Decommissioning and Recommissioning Hosts

Decommissioning a host decommissions and stops all roles on the host without having to go to each service and individually decommission the roles. Decommissioning applies to only to HDFS DataNode, MapReduce TaskTracker, YARN NodeManager, and HBase RegionServer roles. If the host has other roles running on it, those roles are stopped.

Once all roles on the host have been decommissioned and stopped, the host can be removed from service. You can decommission multiple hosts in parallel.

Continue reading:

Decommissioning Hosts

Required Role:

  Note: You cannot decommission a DataNode or a host with a DataNode if the number of DataNodes equals the replication factor (which is by default 3) of any file stored in HDFS. For example, if the replication factor of any file is three, and you have three DataNodes, you cannot decommission a DataNode or a host with a DataNode.

If you are planning to decommission a large number of hosts, see Tuning Prior to Decommissioning a Large Number of Hosts.

To decommission hosts:
  1. Click the Hosts tab.
  2. Select one or more hosts to decommission.
  3. Select Actions for Selected > Decommission.

A confirmation pop-up informs you of the roles that will be decommissioned or stopped on the hosts you have selected. To proceed with the decommissioning, click Confirm.

A Command Details window appears that will show each stop or decommission command as it is run, service by service. You can click one of the decommission links to see the subcommands that are run for decommissioning a given role. Depending on the role, the steps may include adding the host to an "exclusions list" and refreshing the NameNode, JobTracker, or NodeManager, stopping the Balancer (if it is running), and moving data blocks or regions. Roles that do not have specific decommission actions are stopped.

While decommissioning is in progress, the host displays the icon. Once all roles have been decommissioned or stopped, the host displays the icon. If one host in a cluster has been decommissioned, the DECOMMISSIONED facet displays in the Filters on the Hosts page and you can filter the hosts according to their decommission status.

You cannot start roles on a decommissioned host.

Tuning Prior to Decommissioning a Large Number of Hosts

Required Role:

If you are going to decommission a large number of hosts in batches, there is some tuning you should do first to ensure the decommissioning will go smoothly, since replication can be a very time-consuming process.
  1. Raise the heap size of the DataNodes. DataNodes should be configured with at least 4 GB heap size to allow for the increase in iterations and max streams.
    1. Go to the HDFS service page.
    2. Click the Configuration tab.
    3. Under each DataNode role group (DataNode Default Group and any additional DataNode role groups) go to the Resource Management category, and set the Java Heap Size of DataNode in Bytes property as recommended.
  2. Increase the replication work multiplier to a larger number (the default is 2, however 10 is recommended):
    1. Go to the NameNode Default Group > Advanced category.
    2. Search for dfs.namenode.replication and increase the dfs.namenode.replication.work.multiplier.per.iteration property to a value such as 10.
    3. Click Save Changes.
  3. Set the replication maximum streams to a value such as 100.
    1. Go to the Service-Wide > Advanced category.
    2. In the HDFS Service Configuration Advanced Configuration Snippet for hdfs-site.xml, paste in the following:
      <property>
  <name>dfs.namenode.replication.max-streams</name>
  <value>100</value>
</property
    3. Click Save Changes.
  4. Restart the HDFS service.

Recommissioning Hosts

Required Role:

Only hosts that are decommissioned using Cloudera Manager can be recommissioned.

  1. Click the Hosts tab.
  2. Select one or more hosts to recommission.
  3. Select Actions for Selected > Recommission.

The icon is removed from the host and from the roles that reside on the host. However, the roles themselves are not restarted.

Restarting All The Roles on a Recommissioned Host

  1. Click the Hosts tab.
  2. Select one or more hosts on which to start recommissioned roles.
  3. Select Actions for Selected > Start All Roles.
Page generated September 3, 2015.