Architecture of Timeline Service 2.0

YARN Timeline Service 2.0 uses a set of collectors or writers to write data to the back-end storage. The collectors are present at the level of individual applications or the ResourceManager.

The application-level collectors are distributed and co-located with dedicated ApplicationManagers. These collectors collect data that belongs to the applications. For a given application, the ApplicationMaster can write data to the co-located timeline collector. In addition, NodeManagers of other nodes that are running the containers for the application can write data to the timeline collector on the node that is running the ApplicationMaster.

The collector associated with the ResourceManager collects generic life cycle events.

In addition to the collectors, there are separate daemons called timeline readers that are dedicated to serving queries through REST APIs.