YARN application execution consists of the following steps:
Application submission.
Bootstrapping the Application Master instance for the application.
Application execution managed by the Application Master instance.
Let’s walk through an application execution sequence (steps are illustrated in the diagram):
A client program submits the application, including the necessary specifications to launch the application-specific Application Master.
The Resource Manager assumes responsibility for negotiating a specified Container in which to start the Application Master, and then launches the Application Master.
On boot-up, the Application Master registers with the Resource Manager. The registration allows the client program to query the Resource Manager for details, which allows it to directly communicate with its own Application Master.
During normal operation, the Application Master negotiates appropriate resource Containers via the resource-request protocol.
Upon successful Container allocations, the Application Master launches the Container by providing the Container launch specification to the Node Manager. The launch specification typically includes the necessary information to allow the Container to communicate with the Application Master.
The application code executing within the Container then provides necessary information (progress, status etc.) to its Application Master via an application-specific protocol.
During the application execution, the client that submitted the program communicates directly with the Application Master to get status, progress updates, etc. via an application-specific protocol.
Once the application is complete, the Application Master de-registers with the Resource Manager and shuts down, allowing its own Container to be repurposed.
