Workload Analytics is a tool that provides information and insight that can be used for troubleshooting failed jobs and optimizing slow jobs that run on Cloudera Altus clusters. Workload Analytics can be used with any job that can run on an Altus cluster. After a job ends but before the cluster terminates, information about job execution is automatically sent to Workload Analytics with the Telemetry Publisher, a role in the Cloudera Manager Management Service.
Workload Analytics uses the information to display metrics about the performance of a job. Additionally, Workload Analytics compares the current run of a job to previous runs of the same job by creating baselines. You can use the knowledge gained from this information to identify and address abnormal or degraded performance or potential performance improvements.
To use Workload Analytics, the environment that a cluster belongs to must have Workload Analytics enabled, which can be done when you create the environment. For more information about how to enable Workload Analytics, see Environment Options.
View the video in full-screen mode on Vimeo here: Workload Analytics Overview.
When you enable Workload Analytics for an environment, the Cloudera Management Service starts the Telemetry Publisher. The Telemetry Publisher collects and transmits metrics as well as configuration and log files from the Oozie, Hive, YARN, and Spark services for the job to Workload Analytics.
The Telemetry Publisher collects data for all clusters created using environments that have the Workload Analytics option enabled.
Accessing Workload Analytics
Workload Analytics is enabled at the environment level and must be enabled when the environment is created.
- Sign in to the Cloudera Altus console: https://console.altus.cloudera.com/.
- On the side navigation panel, click Jobs.
- Select the job you want to analyze.
- Click on Analytics.
From Analytics, you can access Health Checks, Execution Details, Baseline, and Trend.
There can be a delay from job completion to when the job is available in Workload Analytics. Large jobs can take up to 10 minutes to display in Workload Analytics.
- In the left navigation panel, select Environments.
- Select the environment that the job ran in and check if Workload Analytics is enabled.