YARN Resource Management
Also available as:
PDF
loading table of contents...

Dominant Resource Fairness (DRF)

The Dominant Resource Calculator (DominantResourceCalculator) is used to e nable CPU scheduling. The Dominant Resource Calculator is based on the Dominant Resource Fairness (DRF) model of resource allocation.

DRF uses the concept of the dominant resource to compare multi-dimensional resources. The idea is that in a multi-resource environment, resource allocation should be determined by the dominant share of an entity (user or queue), which is the maximum share that the entity has been allocated of any resource (memory or CPU). Essentially, the DRF seeks to maximize the minimum dominant share across all entities.

For example, if user A runs CPU-heavy tasks and user B runs memory-heavy tasks, the DRF attempts to equalize CPU share of user A with the memory share of user B. In this case, the DRF would allocate more CPU and less memory to the tasks run by user A, and allocate less CPU and more memory to the tasks run by user B. In the single resource case -- where all jobs are requesting the same resources -- the DRF reduces to max-min fairness for that resource.

For more information about DRF, see Dominant Resource Fairness: Fair Allocation of Mulitple Resources.