Cloudera Docs

Use the fs2cs conversion utility

You can use the fs2cs conversion utility to automatically convert certain Fair Scheduler configuration to Capacity Scheduler configuration.

From the CDP Private Cloud Base 7.1 release, Cloudera provides a conversion tool, called fs2cs conversion utility. This utility is a CLI application that is part of the yarn CLI command. It generates capacity-scheduler.xml and yarn-site.xml as output files.

  • Be aware of the Fair Scheduler properties that are auto-converted, those that require manual configuration, and those that do not have an equivalent feature in Capacity Scheduler.
  • You must have downloaded and distributed parcels for the target version of CDP.
  • In VPC, to use your current Compute Cluster queue configurations in your new installation after the upgrade, you must have manually saved them before starting the update process and then added the configurations to your new installation. Else, your Compute Cluster queue configurations will be lost because the Upgrade Wizard transitions only the queues from your Base Cluster.
    1. In Cloudera Manager, navigate to Host > All Hosts.
    2. Find the host with the ResourceManager role and click the YARN ResourceManager role.
    3. Click the Processes tab.
    4. Find and save the fair-scheduler.xml and yarn-site.xml configuration files for future reference.
  1. Download the Fair Scheduler configuration files from the Cloudera Manager data store
    1. In Cloudera Manager, navigate to Host > All Hosts.
    2. Find the host with the ResourceManager role and click the YARN ResourceManager role.
    3. Click the Process tab.
    4. Find and save the fair-scheduler.xml and yarn-site.xml configuration files for future reference.
  2. Use the fs2cs conversion utility
    1. Log in to the host machine where you downloaded the fair-scheduler.xml and yarn-site.xml files using ssh.
    2. Create a new directory to save the capacity-scheduler.xml file that is generated by the fs2cs conversion utility: 
      $ mkdir -p output
    3. Use the fs2cs conversion utility to auto-convert the structure of resource pools. Options listed between braces [] are optional: 
      $ yarn fs2cs [--cluster-resource <vcores/memory>][--no-terminal-rule-check] --yarnsiteconfig </path/to/yarn-site.xml> [--fsconfig </path/to/fair-scheduler.xml>] --output-directory </output/path/> [--print] [--skip-verification]
  3. Provide the generated Capacity Scheduler configuration in Cloudera Manager.
    1. In Cloudera Manager, select the YARN service.
    2. Click the Configuration tab.
    3. Search for capacity-scheduler and find the Capacity Scheduler Configuration Advanced Configuration Snipper (Safety Valves).
    4. Click View as XML and insert the full content of the capacity-scheduler.xml file that was generated by the converter tool.
    5. Click Save Changes.
    6. Search for yarn-site and find the YARN Service Advanced Configuration Snippet (Safety Valve) for yarn-site.xml.
    7. Click View as XML and insert the full content of the yarn-site.xml file that was generated by the converted tool.
    8. Manually remove the following invalid tags:
      • header
      • configuration tags
      • final tags
      • source tags
    9. Click Save Changes.
  4. Restart the YARN and Queue Manager services.
Proceed with the migration to CDP.

After the migration is completed, manually tune the configuration generated by the fs2cs conversion utility using Queue Manager UI and Cloudera Manager Advanced configuration snippet (Safety Valves).