Application Information for Timeline Service 2.0

Timeline Entity Objects

You can provide application information through the following fields in timeline entity objects:

events: A set of timeline events, ordered by the timestamp of the events in descending order. Each event contains an ID and a map to store related information and is associated with one timestamp.
configs: A map from a configuration name to a configuration value and representing all the configurations associated with the entity. Users can post the entire configuration or a part of it in this field. The field is supported for application and generic entities.
metrics: A set of metrics related to a particular entity. There are two types of metrics: single value metric and time series metric. Each metric item contains the metric name, value, and the type of aggregation operation to be performed in the metric. The field is supported for flow run, application and generic entities.
info: A map from an info key name to an (info value object that holds related information for the entity. The field is supported for application and generic entities.
isrelatedtoEntities and relatestoEntities: Each entity contains relatestoEntities and isrelatedtoEntities fields to represent relationships with other entities. Both the fields are represented by a map from a relationship name string to a timeline entity.

Timeline Metrics

When posting timeline metrics, you can select how each metric is aggregated. The term aggregate means applying a TimelineMetricOperation for a set of entities. Timeline Service 2.0 enables aggregating metrics from different timeline entities within one YARN application. The following are the types of operations supported in TimelineMetricOperation:

MAX: Retrieves the maximum value among all TimelineMetric objects.
SUM: Retrieves the sum of all TimelineMetric objects.

The default value of NOP means that no real-time aggregation operation is performed.

Flow Context

Application frameworks must set the flow context whenever possible in order to take advantage of the flow support that Timeline Service 2.0 provides. The flow context consists of the following:

Flow Name: A string that identifies the high-level flow; for example, distributed grep or any identifiable name that can uniquely represent the application. The default value is the name of the application or the ID of the application if the name is not set.
Flow Run ID: A monotonically increasing sequence of numbers that distinguish different runs of the same flow. The default value is the application time in UNIX time (milliseconds).
Flow Version: This optional value is a string identifier that denotes a version of the flow. Flow version can be used to identify changes in the flows, such as code changes or script changes. The default version number is 1.

You can provide the flow context through YARN application tags, as shown in the following example:

ApplicationSubmissionContext appContext = app.getApplicationSubmissionContext();

// set the flow context as YARN application tags
Set<String> tags = new HashSet<>();
tags.add(TimelineUtils.generateFlowNameTag("distributed grep"));
tags.add(Timelineutils.generateFlowVersionTag
("3df8b0d6100530080d2e0decf9e528e57c42a90a"));
tags.add(TimelineUtils.generateFlowRunIdTag(System.currentTimeMillis()));

appContext.setApplicationTags(tags);

	Note
	The ResourceManager converts YARN application tags to lower case values before storing them. Therefore, you must convert flow names and versions to lower case before using them in REST API queries.