Metric sources sent to Cloudera Observability

Describes the resources from which Telemetry Publisher and Databus WXM Client collects diagnostic metrics.

Telemetry Publisher and Databus WXM Client collect metrics, as well as configuration and log files, from Impala, Oozie, Hive, YARN, and Spark services for jobs running on your clusters and transmits this information to Cloudera Observability.

The following example, describes how metrics are collected from a Data Hub Public Cloud environment:

  • Pull — Telemetry Publisher pulls diagnostic metrics from Oozie, YARN, and Spark periodically (by default, once per minute).
  • Push — An Agent pushes diagnostic data from Hive and Impala to Telemetry Publisher within 5 seconds after a job finishes.
The following diagram, shows an example of a Data Hub Public Cloud environment:


After the diagnostic data reaches Telemetry Publisher or Databus WXM Client, it is stored temporarily in its data directory and periodically (once per minute) exported to Cloudera Observability.