CLI options of the fs2cs conversion tool
Before generating the scheduler output files, you must understand the CLI options available for the fs2cs conversion tool. These options help you to complete the scheduler conversion.
Option | Description |
---|---|
-c,--cluster-resource <arg> |
Needs to be specified if maxResources is defined as percentages for any queue,
otherwise this parameter can be omitted. The cluster resource setting is optional,
but it can be necessary in some cases. In Fair Scheduler, you can define the maximum
capacity of the queue as a percentage of the total cluster resource. Capacity
scheduler does not accept a vector of capacities (work is ongoing under YARN-9936), therefore, percentages are converted to
absolute resources. The acceptable formats
are:
|
-d,--dry-run
|
Performs a dry-run of the conversion. Outputs whether the conversion is possible or not. |
-f,--fsconfig <arg> |
Absolute path to a valid By default, |
-h,--help |
Displays the list of options |
-o,--output-directory <arg> |
Output directory for yarn-site.xml and capacity-scheduler.xml files. Must have write permission for the user who is running this script. If |
-p,--print |
If defined, the converted configuration will only be emitted to the console. If -p or --print is specified, the xml files are emitted to the standard output, so the -o / --output-directory is ignored. |
-r,--rulesconfig <arg> |
Optional parameter. If specified, should point to a valid path to the conversion rules file (property format). |
-s, --skip-verification |
It does not validate the converted Capacity Scheduler configuration. By default, the utility starts an internal Capacity Scheduler instance to see whether it can start up properly or not. This switch disables this behaviour. |
-t,--no-terminal-rule-check |
Disables checking whether a placement rule is terminal to maintain backward compatibility with configs that were made before YARN-8967. By default, Fair Scheduler performs a strict check of whether a placement rule is
terminal or not. This means that if you use a <reject> rule which is followed by
a <specified> rule, then this is not allowed, because the latter is unreachable.
However, before YARN-8967, Fair Scheduler was more lenient
and allowed certain sequence of rules that are no longer valid. Inside the tool, a
Fair Scheduler instance is instantiated to read and parse the allocation file. To
have Fair Scheduler accept such configurations, the |
-y,--yarnsiteconfig <arg> |
Path to a valid |
Option | Description |
---|---|
-c,--cluster-resource <arg> |
Needs to be specified if maxResources is defined as percentages for any queue,
otherwise this parameter can be omitted. The cluster resource setting is optional,
but it can be necessary in some cases. In Fair Scheduler, you can define the maximum
capacity of the queue as a percentage of the total cluster resource. Capacity
scheduler does not accept a vector of capacities (work is ongoing under YARN-9936), therefore, percentages are converted to
absolute resources. The acceptable formats
are:
|
-d,--dry-run
|
Performs a dry-run of the conversion. Outputs whether the conversion is possible or not. |
-f,--fsconfig <arg> |
Absolute path to a valid By default, |
-h,--help |
Displays the list of options |
-o,--output-directory <arg> |
Output directory for yarn-site.xml and capacity-scheduler.xml files. Must have write permission for the user who is running this script. If |
-p,--print |
If defined, the converted configuration will only be emitted to the console. If -p or --print is specified, the xml files are emitted to the standard output, so the -o / --output-directory is ignored. |
-pc,-percentage |
By default the fs2cs conversion utility converts into weight mode. Using -pc you can change it to relative (percentage) mode. Note that there are some scheduler transition limitations that are resolved in weight mode but not in relative (percentage) mode. Relative mode can be considered the “legacy” mode of Capacity Scheduler, where capacities are expressed in percentages. |
-r,--rulesconfig <arg> |
Optional parameter. If specified, should point to a valid path to the conversion rules file (property format). |
-s, --skip-verification |
It does not validate the converted Capacity Scheduler configuration. By default, the utility starts an internal Capacity Scheduler instance to see whether it can start up properly or not. This switch disables this behaviour. |
-t,--no-terminal-rule-check |
Disables checking whether a placement rule is terminal to maintain backward compatibility with configs that were made before YARN-8967. By default, Fair Scheduler performs a strict check of whether a placement rule is
terminal or not. This means that if you use a <reject> rule which is followed by
a <specified> rule, then this is not allowed, because the latter is unreachable.
However, before YARN-8967, Fair Scheduler was more lenient
and allowed certain sequence of rules that are no longer valid. Inside the tool, a
Fair Scheduler instance is instantiated to read and parse the allocation file. To
have Fair Scheduler accept such configurations, the |
-y,--yarnsiteconfig <arg> |
Path to a valid |