YARN Resource Management
Also available as:
PDF
loading table of contents...

Upgrading Timeline Server 1.0 to 1.5

[Warning]Warning

Upgrading from ATS v1 to v1.5 may cause data stored in ATS v1.0 storage to be inaccessible.

The upgrade process is not rolling: applications may lose previous data stored in ATS v1.0 during upgrade.

Before launching ATS v1.5, ensure your system is compliant with the following:

  • yarn.timeline-service.entity-group-fs-store.active-dir and yarn.timeline-service.entity-group-fs-store.done-dir must exist on the cluster on HDFS. Active-dir should have permission 01777, owned by YARN, group admin-group. Done-dir should have permission 0700, owned by yarn, group admin-group.

  • You must manually create yarn.timeline-service.entity-group-fs-store.active-dir in HDFS before you start RM and ATS.

  • The Tez cache plugin class must be in the timeline server’s classpath.

  • yarn.timeline-service.leveldb-timeline-store.path must be a path on the local filesystem and must be created with permissions 755. The owner must be the same user that timeline server is running as.

  1. To upgrade to ATS 1.5, configure the following:

     <property>
    	<name>yarn.timeline-service.version</name>
    	<value>1.5</value>
    <description>Timeline service version we’re currently using.</description>
      </property>
    
      <property>
    	<name>yarn.timeline-service.store-class</name>
    	<value>org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore</value>
    <description>Main storage class for YARN timeline server. </description>
      </property>
    
      <property>
    	<name>yarn.timeline-service.entity-group-fs-store.active-dir</name>
    	<value>/ats/active/</value>
    <description>DFS path to store active application’s timeline data</description>
      </property>
    
      <property>
    	<name>yarn.timeline-service.entity-group-fs-store.done-dir</name>
    	<value>/ats/done/</value>
    <description>DFS path to store done application’s timeline data</description>
      </property>
    
  2. The following configuration exists in ATS v1.0, ensure it is pointing to the correct directory:

      <property>
    	<name>yarn.timeline-service.leveldb-timeline-store.path</name>
    	<value><Data disk>/ats/leveldb/</value>
    <description>Local FS path to store the leveldbs that hold the application summary data</description>
      </property>
    
  3. If you have Tez enabled, perform this additional step:

      <property>
        <name>yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes</name>
        <value>org.apache.tez.dag.history.logging.ats.TimelineCachePluginImpl</value>
        <description>Plugins that can translate a timeline entity read request into a list of timeline cache ids, separated by commas. </description>
      </property>
    
  4. Configure the yarn.timeline-service.entity-group-fs-store.summary-store property:

    <property>
         <name>yarn.timeline-service.entity-group-fs-store.summary-store</name> 
         <description>Summary storage for ATS v1.5</description>
         <value>org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore</value>
    </property>