Ambari-Metrics
To restore the backed-up Ambari-Metrics data, downgrade the ambari-metrics packages on all hosts and restore the backed-up-ambari-metrics data.
-
Downgrade ambari-metrics packages on all hosts where they are installed.
On every host in your cluster running a Metrics Monitor, run the following commands:
yum downgrade ambari-metrics-monitor ambari-metrics-hadoop-sink
Run the following command on all hosts running the Metrics Collector:yum downgrade ambari-metrics-collector
Run the following command on the host running the Grafana component:yum downgrade ambari-metrics-grafana
-
Restore backed-up ambari-metrics data on the host where the
ambari-metrics-collector is installed
rm -rf /var/lib/ambari-metrics-collector tar xf ams-backup.tar.gz -C /
-
Start Ambari-Metics in Ambari UI.
Known Issue Description: AMS crashes after metrics DB rollback.Workaround:
- Stop Metrics Collector via ambari UI.
- Disable auto-start for Metrics Collector via ambari UI.
- SSH to metrics collector host.
- Find all processes run under AMS user and kill them.
- Remove PID files on the host (pid files are in /var/run/ambari-metrics-collector/ and /var/run/ams-hbase/ directories).
- Remove the previous content of hbase.rootdir and restore the back-up of metrics DB again.
- Remove the directory specified in AMS hbase.zookeeper.property.dataDir property.
- Remove the directory specified in AMS ams-hbase-site/phoenix.spool.directory property.
- Start Metrics Collector via ambari UI.
- Enable auto-start for Metrics Collector via ambari UI.