Cloudera Docs

Enabling node labels on a cluster to configure partition

You can configure partitions on a cluster by making configuration changes on the YARN ResourceManager host.

  1. Create a Label Directory in HDFS.
    1. Use the following commands to create a node-labels directory in which to store the Node Labels in HDFS: 
      sudo su hdfs
hadoop fs -mkdir -p /yarn/node-labels
hadoop fs -chown -R yarn:yarn /yarn
hadoop fs -chmod -R 700 /yarn

      Where -chmod -R 700 specifies that only the yarn user can access the node-labels directory.

    2. Use the following command to confirm that the directory was created in HDFS: 
      hadoop fs -ls /yarn
      The new node-labels directory should appear. The owner should be yarn, and the permission should be drwx: 
      Found 1 items
drwx------ - yarn yarn 0 2014-11-24 13:09 /yarn/node-labels
  2. In Cloudera Manager, navigate to YARN > Configuration.
  3. Search for node label.
  4. Find the Node Labels property and ensure that it is selected (enabled).
  5. Find the Node Labels Directory property and set its value to reference the HDFS node label directory.
    For example: hdfs://node-1.example.com:8020/yarn/node-labels/.

    HDFS paths are automatically translated, however, any other file system requires a full path to the Node Labels Directory.

    The default value of the Node Labels Directory property is /yarn/node-labels.

  6. Start or Restart the YARN ResourceManager.
  7. Restart YARN Queue Manager.
  8. Refresh the Yarn QueueManager UI.