3. Configuring Generic Data Collection

Enabling the Collection of Generic Data

  • yarn.timeline-service.generic-application-history.enabled

    This property indicates to the ResourceManager, as well as to clients, whether or not the Generic History Service (GHS) is enabled. If the GHS is enabled, the ResourceManager begins recording historical data that the GHS can consume, and clients can redirect to the GHS when applications finish running.

    Example:

    <property>
         <description>Enable or disable the GHS</description>
         <name>yarn.timeline-service.generic-application-history.enabled</name>
         <value>true</value>
    </property>

Configuring the Store for Generic Data

  • yarn.timeline-service.generic-application-history.store-class

    The store class name for the history store. Defaults to the file system store.

    Example:

    <property>
         <description>Store class name for history store, defaulting to file system store</description>
         <name>yarn.timeline-service.generic-application-history.store-class</name>
         <value>org.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStore</value>
    </property>

  • yarn.timeline-service.generic-application-history.fs-history-store.uri

    The URI pointing to the FileSystem path location where the history will be persisted. This must be supplied when using org.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStore as the value for yarn.timeline-service.generic-application-history.store-class.

    Example:

    <property>
         <description>URI pointing to the location of the FileSystem path where the history will be persisted.</description>
         <name>yarn.timeline-service.generic-application-history.fs-history-store.uri</name>
         <value>${hadoop.log.dir}/yarn/system/history</value>
    </property>

  • yarn.timeline-service.generic-application-history.fs-history-store.compression-type

    The T-file compression types used to compress history data. The available values are:

    • "none" -- No compression

    • "lzo" -- LZO compression

    • "gz" -- GZIP compression

    Example:

    <property>
         <description>T-file compression types used to compress history data.</description>
         <name>yarn.timeline-service.generic-application-history.fs-history-store.compression-type</name>
         <value>none</value>
    </property>


loading table of contents...