Cloudera Docs

October 12, 2022

This release (1.17-h1) of the Cloudera Data Engineering Service on CDP Public Cloud introduces the following changes.

Support for Iceberg 0.14

When you upgrade to CDE 1.17-h1, ensure that you also upgrade to Iceberg 0.14. For more information, see Using Apache Iceberg in Cloudera Data Engineering. The following features are included with Iceberg 0.14:
  • MERGE operations allow for bulk updates and DELETES.
  • CDE Azure deployments are now able to leverage Iceberg for Lakehouse architecture.
  • [Technical Preview] Iceberg table format version 2 (v2) is the latest specification available and includes the following key updates:
    • UPDATE and DELETE operations follow the Iceberg format v2 row-level position delete specification and enforce snapshot isolation.
    • DELETES, UPDATES, and MERGE operations use the merge-on-read function by default. Merge-on-read is more efficient than the copy-on-write function because it does not rewrite file data.

Set default values for the variables in CDE job specification

Using [--default-variable] flags you can now replace strings in job values. For more information, see Creating and updating Apache Spark jobs using the CLI.