YARN Post-Upgrade Task

Considering logical processors in the calculation

The yarn.nodemanager.resource.count-logical-processors-as-cores property was not present in CDH 5. In CDH 6 it is set to false by default, meaning that YARN does not consider logical processors in the calculation which can results in a 2x performance hit if Linux Container Executor and CGroups are enabled. To solve this issue, do the following:
  1. In Cloudera Manager, navigate to YARN > Configuration.
  2. Find the YARN Service Advanced Configuration Snippet (Safety Valve) for yarn-site.xml property.
  3. Add the following configuration:
    yarn.nodemanager.resource.count-logical-processors-as-cores=true

    Using this configuration snippet ensures that all nodes that need the configuration receive it. This also ensures different NodeManager groups are consistently configured.

  4. Restart the NodeManager.