Converting Fair Scheduler to Capacity Scheduler using the fs2cs conversion utility

You can use the fs2cs conversion utility to automatically convert certain Fair Scheduler configuration to Capacity Scheduler configuration.

From the Cloudera Base on premises 7.1 release, Cloudera provides a conversion tool, called fs2cs conversion utility. This utility is a CLI application that is part of the yarn CLI command. It generates capacity-scheduler.xml and yarn-site.xml as output files.

From Cloudera Base on premises 7.1.6, the fs2cs conversion utility converts the scheduler configuration from weight resource allocation mode in Fair Scheduler to weight resource allocation mode in Capacity Scheduler. However, this can be changed by using the -pc command line option. If -pc is used, the fs2cs conversion utility converts from weight resource allocation mode to relative (percentage) mode.

When using the fs2cs utility to automatically convert your Fair Scheduler into a Capacity Scheduler, you have to perform the following steps:

Be aware of the Fair Scheduler properties that are auto-converted, those that require manual configuration, and those that do not have an equivalent feature in Capacity Scheduler. To learn about how properties are converted, see Plan your scheduler migration.
You must have downloaded and distributed parcels for the target version of Cloudera.

Download the Fair Scheduler configuration files from the Cloudera Manager data store:
1. In Cloudera Manager for the CDH cluster, navigate to Host > All Hosts.
2. Find the host with the ResourceManager role and click the YARN ResourceManager role.
3. Click Process.
4. Find and save the fair-scheduler.xml and yarn-site.xml configuration files for future reference.
Use the fs2cs conversion utility:
1. Log in to the host machine which is part of the Cloudera cluster using ssh.
2. Copy the downloaded XML files to this host using scp.
3. Create a new directory to save the capacity-scheduler.xml file that is generated by the fs2cs conversion utility:
```
$ mkdir -p output
```
4. Use the fs2cs conversion utility to auto-convert the structure of resource pools:
```
$ yarn fs2cs [--cluster-resource <vcores/memory>][--no-terminal-rule-check] --yarnsiteconfig </path/to/yarn-site.xml> [--fsconfig </path/to/fair-scheduler.xml>] --output-directory </output/path/> [--print] [--skip-verification]  
```
For example, if ou copied the XML files to /home/migration, then you can use the following command:
```
$ yarn fs2cs --yarnsiteconfig /home/migration/yarn-site.xml --fsconfig /home/migration/fair-scheduler.xml --output-directory /home/migration/output --no-terminal-rule-check
```
For a detailed list of switches supported by the fs2cs conversion utility, use yarn fs2cs --help.

note
You have to provide an absolute path for the yarn-site.xml and the fair-scheduler.xml configuration file. If only the file names are provided the command fails.
Provide the generated Capacity Scheduler configuration in Cloudera Manager:
1. Remove the following invalid tags from the fs2cs conversion utility generated yarn-site.xml file:
  - header
  - configuration tags
  - final tags
  - source tags
2. In Cloudera Manager for the Cloudera cluster, select the YARN service.
3. Click Configuration.
4. Search for capacity-scheduler and find the Capacity Scheduler Configuration Advanced Configuration Snipper (Safety Valves).
5. Click View as XML and insert the full content of the capacity-scheduler.xml file that was generated by the converter tool.
6. Click Save Changes.
7. Search for yarn-site and find the YARN Service Advanced Configuration Snippet (Safety Valve) for yarn-site.xml.
8. Click View as XML and insert the full content of the yarn-site.xml file that was generated by the converted tool.
9. Click Save Changes.
Restart the YARN and Queue Manager services.

If the fs2cs conversion utility command fails, check if you provided the correct absolute path for the yarn-site.xml and the fair-scheduler.xml configuration file.

Proceed with the migration to CDP.

After the migration is completed, manually tune the configuration generated by the fs2cs conversion utility using Queue Manager UI and Cloudera Manager Advanced configuration snippet (Safety Valves).