Cloudera Docs

Enabling deep analysis on a Cloudera Data Engineering job from the CDE web UI

Using the Cloudera Data Engineering (CDE) web UI you can enable an on-demand deep analysis on a CDE job run to generate detailed information, including memory utilization and stage-level analysis.

Because deep analysis consumes cluster resources by running an internal CDE job, you must enable and run it manually for any job run you want to analyze or troubleshoot.

  1. From the CDE Overview page, select the CDE service for the job you want to troubleshoot or analyze.
  2. In the Virtual Clusters column, click the View Jobs icon for the cluster containing the job.
  3. Select the job you want to analyze.
  4. Select the Configuration tab and click Edit.
  5. Select the Spark Analysis option.
    This enables collecting metrics during future runs of the job that you want to investigate.
  6. Click Update.
  7. Click Actions > Run Now to run the job.
    Metrics are collected for deep analysis.