List of Pages in Category YARN (45 pages)

YARN (Yet Another Resource Negotiator)

A general architecture for running distributed applications. YARN specifies the following components:

  • ResourceManager - A master daemon that authorizes submitted jobs to run, assigns an ApplicationMaster to them, and enforces resource limits.
  • ApplicationMaster - A supervisory task that requests the resources needed for executor tasks. An ApplicationMaster runs on a different NodeManager for each application. The ApplicationMaster requests containers, which are sized by the resources a task requires to run.
  • NodeManager - A worker daemon that launches and monitors the ApplicationMaster and task containers.
  • JobHistory Server - Keeps track of completed applications.

The ApplicationMaster negotiates with the ResourceManager for cluster resources—described in terms of a number of containers, each with a certain memory limit—and then runs application-specific processes in those containers. The containers are overseen by NodeManagers running on cluster nodes, which ensure that the application does not use more resources than it has been allocated.

MapReduce v2 (MRv2) is implemented as a YARN application.


C D E I J M N R S T V W Y

*

C

D

E

I

J

M

N

R

S

T

V

W

Y