Auto-converted Fair Scheduler properties

The fs2cs conversion utility automatically converts certain Fair Scheduler properties into Capacity Scheduler properties. Reviewing the list of auto-converted properties enables you to verify the conversion and plan the manual fine-tuning that requires to be done after the upgrade is completed.

Table 1. Queue resource-quota related features
Property Description

Pre-created hierarchical queues.

The same queue hierarchy is achieved after conversion.

<weight>

Weight: The steady fair share of a queue.

The queue.capacity property will be set with the same ratio.

<maxAMShare>

Maximum AM share: Limits the fraction of the queue’s fair share that can be used to run application masters

<maxRunningApps>

Maximum running apps: Limits the number of apps from the queue to run at once

<maxContainerAllocation>

Maximum container allocation: Maximum amount of resources a queue can allocate for a single container.

<schedulingPolicy>

Scheduling policy of a queue (for example, how submitted applications are ordered over time).

It is converted with some limitations. For more information, see Fair Scheduler features and the conversion details.

<aclSubmitApps> <aclAdministerApps>

ACL settings: List of users and/or groups that can submit apps to the queue or can administer a queue.

maximum-applications

Specifies the maximum number of concurrent active applications at any one time in the queue.

maximum-am-resource-percent

Specifies the maximum percentage of resources in the cluster which can be used to run application masters for the queue.

acl_submit_applications

Specifies the ACL which controls who can submit applications to the given queue.

acl_administer_queue

Specifies the ACL which controls who can administer applications in the given queue.

ordering-policy

Specifies the queue ordering policies to FIFO or fair on the given queue.

Table 2. Global scheduling settings
Property Description
yarn.scheduler.fair.allow-undeclared-pools

Allow undeclared pools.

Sets whether new queues can be created at application submission time.

yarn.scheduler.fair.sizebasedweight

Size based weight.

Whether to assign shares to individual apps based on their size, rather than providing an equal share to all apps regardless of size.

<queueMaxAppsDefault>

Queue max apps default: Sets the default running app limit for all queues.

<queueMaxAMShareDefault>

Default max AM share: Sets the default AM resource limit for queue.

yarn.scheduler.fair.locality.threshold.node

Locality threshold node: For applications that request containers on particular nodes, the number of scheduling opportunities since the last container assignment to wait before accepting a placement on another node.

yarn.scheduler.fair.locality.threshold.rack

Locality threshold rack: For applications that request containers on particular racks, the number of scheduling opportunities since the last container assignment to wait before accepting a placement on another rack.

yarn.scheduler.fair.max.assign

Maximum assignments: If assignmultiple is true and dynamic.max.assign is false, the maximum amount of containers that can be assigned in one heartbeat.

yarn.scheduler.fair.assignmultiple

Assign multiple: Whether to allow multiple container assignments in one heartbeat.

yarn.resourcemanager.scheduler.monitor.enable

Allows higher-priority applications to preempt lower-priority applications.

yarn.scheduler.capacity.maximum-am-resource-percent

Specifies the maximum percentage of resources in the cluster which can be used to run application masters.

Table 3. Preemption
Property Description
yarn.scheduler.fair.preemption

Fair Scheduler preemption turned on.

After the conversion capacity Scheduler preemption is turned on by default using the default values.

<allowPreemptionFrom>

Per-queue preemption disabled.

After the conversion the same queue preemption disabled by default.

yarn.scheduler.fair.waitTimeBeforeKill

Wait time before killing a container

disable_preemption

Disables preemption of application containers submitted to a given queue.