Timeline Server 1.5 Overview
In HDP 2.2.9, Timeline Server 1.5 includes changes and fixes that improve scalability.
Version 1.5 incorporates changes and improvements to the whole stack; it introduces new client
APIs, a new storage architecture (the EntityGroupFSTimelineStore), and supports a
“plugin” mechanism to work with YARN applications. The most significant change, on the YARN
side, is the EntityGroupFSTimelineStore. This storage system has a reader part
and a writer part. To work with this new version 1.5 architecture, applications can use the
newly introduced client APIs to write to version 1.5, and provide a plugin that assists
version 1.5 on the reader part. Tez currently provides this support and can work with version
1.5 natively.
RollingLevelDbTimelineStoreimproves the performance for ATS (Application Timeline Server) operations that need to purge data regarding obsolete entries. Instead of deleting one row at a time, the store is organized as a list of databases so that an entire database can be dropped when all the entries contained within it are obsolete.EntityGroupFSTimelineStoreenables a completely different reader-writer architecture for the ATS. It has two parts, a writer part and a reader part. The writer part involves configuring Tez with the ATSV15Plugin. The reader part involves specifying the configurations below. The writer in this instance writes summary information to the ATS and the bulk of the data to the filesystem location specified by the configuration path. Before this store can be used the directory has to be created with the correct permissions.
Changed properties:
Property:
yarn.timeline-service.store-classOld value:
org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStoreNew value:
org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore
Property:
yarn.timeline-service.leveldb-timeline-store.pathOld value:
${yarn.log.dir}/timelineNew value:
<Data disk>/ats/leveldb/
New properties:
tez.history.logging.service.classyarn.timeline-service.entity-group-fs-store.active-diryarn.timeline-service.entity-group-fs-store.cleaner-interval-secondsyarn.timeline-service.entity-group-fs-store.done-diryarn.timeline-service.entity-group-fs-store.group-id-plugin-classesyarn.timeline-service.entity-group-fs-store.retain-secondsyarn.timeline-service.entity-group-fs-store.scan-interval-secondsyarn.timeline-service.entity-group-fs-store.summary-storeyarn.timeline-service.version