Cloudera Docs

Creating an ad-hoc job in Cloudera Data Engineering

Ad-hoc runs mimic the behavior of the traditional spark-submit or single execution of an Airflow DAG, where the job runs once. These runs will not establish a permanent job definition. You can use the ad-hoc job runs for log analysis and future reference.

Before you begin

  • Ensure that you have a Virtual Cluster that is ready to use.
  1. In the Cloudera Data Platform (CDP) console, click the Data Engineering tile. The Home page displays.
  2. In the Jobs section under Spark, click Ad-hoc Run.
  3. Select a Virtual Cluster.
  4. Enter a Job Name.
  5. Upload an Application File or enter the Application File’s External URL.
  6. Enter a Main Class.
  7. Enter Arguments and Configurations.
  8. Select a Python Environment
  1. In the Cloudera Data Platform (CDP) console, click the Data Engineering tile. The Home page displays.
  2. In the Jobs section under Airflow, and click Ad-hoc Run.
  3. Select a Virtual Cluster.
  4. Enter a Job Name.
  5. Upload a DAG file.
  6. Click Create and Run.