Enabling deep analysis on a Cloudera Data Engineering job run using the CLI
Using the Cloudera Data Engineering (CDE) CLI you can enable an on-demand deep analysis on a CDE job run to generate detailed information, including memory utilization and stage-level analysis.
You can enable deep analysis for a job run by using the cde job run or cde spark submit commands with the --enable-analysis flag.
cde job run:
cde job run --name [***JOB NAME***] --enable-analysis
cde job run --name test_job --enable-analysis
cde spark submit:
cde spark submit [***JAR/PY FILE***] --enable-analysis
cde spark submit test_job.jar --enable-analysis
Metrics are collected for deep analysis.