Running a Airflow job using the CLI
The following example demonstrates how to run a Cloudera Data Engineering (CDE) Airflow job using the command line interface (CLI).
Make sure that the job has been created and all necessary resources have been created and uploaded.
Using the cde job run requires more preparation on the target
environment compared to the cde airflow submit command. Whereas
cde airflow submit is a quick and efficient way of testing an
Airflow job during development, cde job run is suited for
production environments where a job is to be run multiple times, therefore removing
resources and job definitions after every job run is neither necessary, nor
viable.
cde job run --name <job name> [--config <key=value>]* [--wait]Airflow configs provided at job run time will override the corresponding job configs.
By default the command returns the job run ID as soon as the job has been submitted.
Optionally, you can use the --wait switch to wait until the
job run ends and returns a non-zero exit code if the job run was not
successful.
