Supported Airflow operators and hooks
Apache Airflow Python DAG files can be used to automate workflows or data pipelines in Cloudera Data Engineering. Cloudera Data Engineering currently supports a specified list of Airflow operators and hooks that can be used.
Cloudera Data Engineering 1.5.5 SP3
Airflow operators:
Cloudera Data Engineering supports the following Airflow operators:
- airflow.operators.bash
- airflow.operators.branch
- airflow.operators.datetime
- airflow.operators.dummy
- airflow.operators.generic_transfer
- airflow.operators.latest_only
- airflow.operators.python
- airflow.providers.common.sql.operators.sql
- airflow.providers.apache.sqoop.operators.sqoop
- airflow.operators.trigger_dagrun
- airflow.operators.email
Additional operators:
Starting from Cloudera Data Engineering 1.5.5 SP1 release onwards, Cloudera Data Engineering supports the following additional operators:
- Hive
- Impala
- Sqoop
Airflow hooks:
Cloudera Data Engineering supports the following Airflow hooks:
- airflow.hooks.filesystem
- airflow.hooks.subprocess
Airflow provider packages:
- apache-airflow-providers-amazon 9.22.0
- apache-airflow-providers-apache-hive 9.3.0
- apache-airflow-providers-apache-impala 1.9.0
- apache-airflow-providers-apache-sqoop 4.2.1
- apache-airflow-providers-celery 3.15.2
- apache-airflow-providers-cncf-kubernetes 10.13.0
- apache-airflow-providers-common-compat 1.14.0
- apache-airflow-providers-common-io 1.7.1
- apache-airflow-providers-common-sql 1.32.0
- apache-airflow-providers-docker 4.5.2
- apache-airflow-providers-elasticsearch 6.5.0
- apache-airflow-providers-fab 1.5.4
- apache-airflow-providers-ftp 3.14.1
- apache-airflow-providers-grpc 3.9.2
- apache-airflow-providers-hashicorp 4.5.0
- apache-airflow-providers-http 6.0.0
- apache-airflow-providers-imap 3.11.0
- apache-airflow-providers-microsoft-azure 13.0.0
- apache-airflow-providers-mysql 6.5.0
- apache-airflow-providers-odbc 4.12.0
- apache-airflow-providers-openlineage 2.11.0
- apache-airflow-providers-postgres 6.6.0
- apache-airflow-providers-redis 4.4.2
- apache-airflow-providers-sendgrid 4.2.1
- apache-airflow-providers-sftp 5.7.0
- apache-airflow-providers-slack 9.7.0
- apache-airflow-providers-smtp 2.4.2
- apache-airflow-providers-sqlite 4.3.0
- apache-airflow-providers-ssh 4.3.1
- apache-airflow-providers-trino 6.5.0
- cloudera-airflow-provider 2.1.4
- google-cloud-orchestration-airflow 1.19.0
Cloudera Data Engineering 1.5.5 SP2 and earlier releases
Airflow operators:
Cloudera Data Engineering supports the following Airflow operators:
- airflow.operators.bash
- airflow.operators.branch
- airflow.operators.datetime
- airflow.operators.dummy
- airflow.operators.generic_transfer
- airflow.operators.latest_only
- airflow.operators.python
- airflow.providers.common.sql.operators.sql
- airflow.providers.apache.sqoop.operators.sqoop
- airflow.operators.trigger_dagrun
- airflow.operators.email
Additional operators:
Starting from Cloudera Data Engineering 1.5.5 SP1 release onwards, Cloudera Data Engineering supports the following additional operators:
- Hive
- Impala
- Sqoop
Airflow hooks:
Cloudera Data Engineering supports the following Airflow hooks:
- airflow.hooks.filesystem
- airflow.hooks.subprocess
Airflow provider packages:
- apache-airflow-providers-amazon 9.1.0
- apache-airflow-providers-apache-hive 8.2.1
- apache-airflow-providers-apache-impala 1.5.2
- apache-airflow-providers-celery 3.8.5
- apache-airflow-providers-cncf-kubernetes 10.0.1
- apache-airflow-providers-common-compat 1.2.2
- apache-airflow-providers-common-io 1.4.2
- apache-airflow-providers-common-sql 1.20.0
- apache-airflow-providers-docker 3.14.1
- apache-airflow-providers-elasticsearch 5.5.3
- apache-airflow-providers-fab 1.5.1
- apache-airflow-providers-ftp 3.11.1
- apache-airflow-providers-google 11.0.0
- apache-airflow-providers-grpc 3.6.0
- apache-airflow-providers-hashicorp 3.8.0
- apache-airflow-providers-http 4.13.3
- apache-airflow-providers-imap 3.7.0
- apache-airflow-providers-microsoft-azure 11.1.0
- apache-airflow-providers-mysql 5.7.4
- apache-airflow-providers-odbc 4.8.1
- apache-airflow-providers-openlineage 1.14.0
- apache-airflow-providers-postgres 5.14.0
- apache-airflow-providers-redis 3.8.0
- apache-airflow-providers-sendgrid 3.6.0
- apache-airflow-providers-sftp 4.11.1
- apache-airflow-providers-slack 8.9.2
- apache-airflow-providers-smtp 1.8.1
- apache-airflow-providers-snowflake 5.8.1
- apache-airflow-providers-sqlite 3.9.1
- apache-airflow-providers-ssh 3.14.0
- cloudera-airflow-provider 2.1.4
- cloudera_cdp_airflow_internal 1.0.0
- google-cloud-orchestration-airflow 1.15.1
