Orchestrating workflows and pipelinesPDF version

Supporting Airflow operators and hooks

Apache Airflow Python DAG files can be used to automate workflows or data pipelines in Cloudera Data Engineering (CDE). CDE currently supports a specified list of Airflow operators and hooks that can be used.

CDE supports the following Airflow operators:

  • airflow.operators.bash
  • airflow.operators.branch
  • airflow.operators.datetime
  • airflow.operators.dummy
  • airflow.operators.email
  • airflow.operators.generic_transfer
  • airflow.operators.latest_only
  • airflow.operators.python
  • airflow.operators.sql
  • airflow.operators.subdag
  • airflow.operators.trigger_dagrun

CDE supports the following Airflow hooks:

  • airflow.hooks.filesystem
  • airflow.hooks.subprocess