Monitoring and Troubleshooting Cloudera Data EngineeringPDF version

Enabling deep analysis on a Cloudera Data Engineering job from the Cloudera Data Engineering web UI

Using the Cloudera Data Engineering web UI you can enable an on-demand deep analysis on a Cloudera Data Engineering job run to generate detailed information, including memory utilization and stage-level analysis.

Because deep analysis consumes cluster resources by running an internal Cloudera Data Engineering job, you must enable and run it manually for any job run you want to analyze or troubleshoot.

  1. From the Cloudera Data Engineering Overview page, select the Cloudera Data Engineering service for the job you want to troubleshoot or analyze.
  2. In the Virtual Clusters column, click the View Jobs icon for the cluster containing the job.
  3. Select the job you want to analyze.
  4. Select the Configuration tab and click Edit.
  5. Select the Spark Analysis option.
    This enables collecting metrics during future runs of the job that you want to investigate.
  6. Click Update.
  7. Click Actions > Run Now to run the job.
    Metrics are collected for deep analysis.