Cloudera Docs

YARN Capacity Scheduler Sidecar migration from HDP to CDP

To migrate YARN from a HDP cluster you must migrate the Capacity Scheduler to CDP. The migration includes converting Capacity Scheduler placement rules to the new JSON format.

To migrate the Capacity scheduler from HDP to CDP:​​
  1. Get and copy the capacity-scheduler.xml configuration file from Ambari.
  2. Log in to Cloudera Manager in the destination cluster and ensure that the YARN Queue Manager is not started and that the YARN service is installed and running.
  3. Go to the YARN service.
  4. Click the Configurations tab.
  5. Search for the following property: Capacity Scheduler Configuration Advanced Configuration Snippet (Safety Valve)
  6. Copy and paste the contents of the capacity-scheduler.xml configuration file you copied from Ambari into the text box.
  7. Restart YARN. (Click Actions > Restart)
  8. Open the YARN Queue Manager UI (Click Clusters > [Cluster Name] > YARN Queue Manager UI)
  9. Queue Manager reads the configuration from the safety-valve and if it detects that the placement rules are in the format used by the source cluster, it automatically converts it into the new JSON-based placement rules format (this format was introduced in CDP Private Cloud Base 7.1.6).
  10. Verify that Queue Manager correctly converted the configuration. You can test the conversions by going to the Placement Rules tab and changing the value of a property temporarily. If you see an “out of sync” error message, then the configurations are not in sync.

From now on, Queue Manage can be used for configuration.

The capactiy-scheduler.xml Advanced Configuration Snippet works differently than the usual Advanced Configuration Snippet. Normally, Advanced Configuration Snippets are used as a powerful tool to edit configuration. In the case of the YARN Queue Manager UI, Queue Manager continuously writes into the Advanced Configuration snippet as the user changes the configuration via the YARN Queue Manager UI. Therefore the configuration set in the Advanced Configuration Snippet cannot differ from the Queue Manager configuration. If you experience out-of-sync errors, restart the YARN and Queue Manager services, which will override the Advanced Configuration Snippet with the latest settings applied in Queue Manager.