2. Aggregated Metrics TTL Settings

AMS provides configurable Time To Live configuration for aggregated metrics. The TTL settings are available in Ambari Metrics > Configs > Advanced ams-site and have the “.ttl” suffix. Each property name is self explanatory and controls the amount of time to keep metrics at the specified aggregation level before they are purged. The values for these TTL’s are set in seconds. In an example where you are running a single-node sandbox and want to ensure that no values are stored for more than 7 days to save on local disk space, you would set any property ending in “.ttl” that has a value greater than 604800, 7 days in seconds, to 604800. That would ensure that properties such as timeline.metrics.cluster.aggregator.daily.ttl that controls the daily aggregation TTL, which by by default stores data for 2 years, will only store daily aggregations for 604800 seconds, or 7 days. Reducing the TTL values helps significantly reduce the total amount of storage used for metric storage. Those that matter most for reducing the total amount of disk space used for AMS are:

It’s important to note that these settings should be set during installation. If these settings need to be changed post-installation, they have to be set using the HBase shell. This is a current limitation imposed by Phoenix that is resolved in Ambari 2.1.2. To change these TTL settings in Ambari 2.1.1 and earlier, the HBase shell is used to connect to the embedded HBase instance that is part of AMS. Run the following command from the Collector host.

/usr/lib/ams-hbase/bin/hbase --config /etc/ams-hbase/conf shell

Once connected to HBase each of the following tables needs to be updated with the appropriate TTL values that are being changed. The table below maps a specific ".ttl" property in Ambari Metrics > Configs > Advanced ams-site to the actual HBase table.

For each table that needs to be updated, alter the TTL value as follows:

hbase(main):000:0> alter 'METRIC_RECORD_DAILY', { NAME => '0', TTL => 604800}