General known issues with Cloudera Data Engineering

Learn about the general known issues with the Cloudera Data Engineering (CDE) service on public clouds, the impact or changes to the functionality, and the workaround.

DXE-2402 : The option to “Force Disable” a CDE Service will not work when the associated CDP Environment backing the CDE Service has been deleted

A failed CDE Service has the option to “Force Disable” to remove it from the UI Overview page. This will not work when the associated CDP Environment backing the CDE Service has been deleted.

DEX-2576 : A scheduled Airflow DAG with start_date and end_date parameters is not correctly displayed in the CDE UI within the Jobs Schedule page and the Airflow UI within the DAG details

If you define the start_date and end_date parameters in the default arguments of the Airflow DAG, they are not updated in the Airflow UI under the DAG Details section, but are displayed in the CDE UI under the Jobs Schedule page. If you define the start_date and end_date parameters only in the DAG definition block, they are displayed in the Airflow UI under the DAG Details section but are not displayed in the Jobs Schedule page.

Workaround: Define the start_date and end_date parameters in both the default_args and the DAG definition block.

DEX-2592 : Deleting the Airflow Job while it is in running state causes incorrect reporting of the Job status

Deleting the Airflow Job while it is in running state will cause the Job status to remain in running state indefinitely, both in the UI and through the API/CLI. The actual Airflow tasks are terminated when the corresponding Airflow Job is deleted, either through the UI or API/CLI.

DEX-2585 : Jobs with a daily schedule can fail to run

Jobs that are configured to run on a daily schedule (@daily) can fail to run at the scheduled time.

DEX-2482 - Patch CVE-2020-8554

New admission controller to patch a vulnerability affecting multitenant clusters. If a potential attacker can already create or edit services and pods, then they may be able to intercept traffic from other pods (or nodes) in the cluster.

DEX-2675 - Patch CVE-2021-3167

This issue impacts only CDE 1.3.0.

CDE authentication tokens are exposed in pod and archived logs. Cloudera Data Engineering (CDE) uses short-lived JWTs to authenticate end users. In virtual clusters created in a CDE 1.3.0 service, these tokens are logged by the server as part of the HTTP request headers in plain text in the Jobs API server (at DEBUG level, which is the default level). A malicious user who has administrator access to the Kubernetes cluster or the archived log files in the CDP environment telemetry log bucket on S3 could copy the JWT and, before its 1 hour expiry, use it to impersonate a user to create and run jobs, and perform other actions in a virtual cluster in the same CDE service.