Cloudera Docs

Clearing and rerunning failed DAG tasks

Learn how to re-execute specific failed or successful tasks using the Clear task option within the Cloudera Data Engineering Airflow UI. This functionality allows Cloudera Data Engineering Admins to resume workflows by triggering retries for manual jobs or creating new run attempts for scheduled jobs without restarting the entire DAG.

The Clear task option is only enabled for the following Cloudera Data Engineering Admin roles:

  • DE Admin
  • Service Admin
  • VC Admin

Make sure that you have one of these roles assigned to perform this action.

  1. In the Cloudera console, click the Data Engineering tile. The Cloudera Data Engineering Home page displays.
  2. In the left navigation menu, click Jobs. The Jobs page opens and displays all the jobs created in the virtual cluster.
  3. Click the name of the job for which you want to clear and rerun the failed DAG task.
  4. Click Airflow UI > Graph tab.
  5. In the graph, click the task that you want to rerun.
  6. Click Clear task button. The Clear and Retry dialog box opens.
  7. Click Clear button.
    • If the job is a manually triggered job, then the task status changes to up_for_retry. You can view the logs, but no new Cloudera Data Engineering run is created.
    • If the job is a scheduled job, a new run attempt is created. This new attempt includes the logs from the previous attempt as well as the new execution logs.