Release NotesPDF version

July 23, 2024

This release (1.22.0) of the Cloudera Data Engineering (CDE) service on CDP Public Cloud introduces the following changes.

AWS Graviton is a general purpose, ARM-based processor family. From the AWS Graviton family, CDE supports AWS Graviton 3. AWS Graviton delivers currently the best price performance for cloud workloads running in AWS EC2. With AWS Graviton, you can optimize costs and achieve better performance.

For more information, see AWS Graviton instances in CDE.

If you use services that run on a private network, (for example, Private EKS), your CDE service is eligible for an in-place upgrade.

For more information, see In-place upgrade with Apache Airflow Operators and Libraries

The Kubernetes version that CDE uses is upgraded to 1.28. For more information, see Compatibility for Cloudera Data Engineering and Runtime components.

The Iceberg version that CDE uses is upgraded to 1.4.3. For more information, see Compatibility for Cloudera Data Engineering and Runtime components.

CDE supports Spark version 3.5.1. For more information, see Compatibility for Cloudera Data Engineering and Runtime components.

With the VC-level Spark configurations option, you can create and update Spark configurations that, by default, apply to all Spark jobs that are run in a VC.

For more information, see Managing Virtual Cluster-level Spark configurations

With the Azure private DNS zones feature, you can optimize costs by leveraging existing Azure private DNS zone resources. When creating the CDE service, you can use an Azure private DNS zone for the Azure Kubernetes Service (AKS), the Storage Account File Share, and for the database.

For more information, see Azure private DNS zones in a CDE service.

  • DEX-12763 - Upgraded CDE Service to 1.20.3 and above with SSD Instances unable to run jobs
  • DEX-13755 - Azure and AWS service B&R does not restore all the jobs
  • DEX-13151 - UI keeps loading when jobs are unpaused if the user has readonly access
  • DEX-12634 - VC Security/Access Control allows for duplicate user record
  • DEX-14080 - Remove VC cluster maintenance mode configuration on restore
  • DEX-13154 - View log for VC will not redirect to Logs tab
  • DEX-13742 - UI should send allp details for previous Service version
  • DEX-13262 - UI: editing a schedule for the example job fails to send a request
  • DEX-13385 - UI does not indicate user does not have access to CLI tool
  • DEX-13639 - Without All Purpose details we are showing All Purpose OnDemand and Spot instances on Service Config page
  • DEX-13692 - Include sub seconds into execution date for manually triggered Airflow runs
  • DEX-13958 - In place upgrade fails from 1.19/1.20 -> 1.21/1.22 if custom runtime images are used with public docker registry