YARN (Yet Another Resource Negotiator)
A general architecture for running distributed applications. YARN specifies the following components:
- ResourceManager - Manages the global assignment of compute resources to applications.
- ApplicationMaster - Manages the lifecycle of applications.
- NodeManager - Launches and monitors the compute containers on machines in the cluster.
- JobHistory Server - Keeps track of completed applications.
The ApplicationMaster negotiates with the ResourceManager for cluster resources—described in terms of a number of containers, each with a certain memory limit—and then runs
application-specific processes in those containers. The containers are overseen by NodeManagers running on cluster nodes, which ensure that the application does not use more resources than it has
MapReduce v2 (MRv2) is implemented as a YARN application.