Cloudera Docs

Iceberg changelog for CDW Public Cloud

Review the changes introduced in Iceberg for Cloudera Data Warehouse Public Cloud.

2024.0.18.4 (December 5, 2024) GA

  • CDPD-68442: IMPALA-12970: Fix ConcurrentModificationException for Iceberg table scans
  • IMPALA-13432: test_migrated_table_field_id_resolution_complex can fail with different Iceberg version
  • CDPD-74017: IMPALA-13364: Schema resolution doesn't work for migrated partitioned Iceberg tables that have complex types
  • CDPD-75084: HIVE-28570: Iceberg: Aggregate queries on translated iceberg table fails post rename.

2024.0.18.2-4 (October 4, 2024)

Hive - Iceberg changes
  • DWX-18477: HIVE-28282: Addendum: Iceberg: Merge query failures with reserved keywords as column names

2024.0.18.1-1 (August 15, 2024)

No new features or fixes.

2024.0.18.0-206 (July 26, 2024)

Hive - Iceberg changes
  • DWX-13719: Iceberg positional deletes cache support
  • DWX-18477: HIVE-28282: Merging into iceberg table fails with copy on write when values clause has a function call
  • CDPD-69704: HIVE-28266: Iceberg: select count(*) from data_files metadata tables gives wrong result
  • CDPD-71800: Iceberg: Use 'merge-on-read' as default write mode for v2 tables
  • CDPD-70188: HIVE-28278: Iceberg: Stats: IllegalStateException Invalid file: file length 0
  • DWX-18213: NFQE SMG Iceberg: Hive select after deletes on table returning inconsistent data
  • CDPD-68088: HIVE-28127,HIVE-28203: Exception when rebuilding materialized view with calculated columns on iceberg sources
  • CDPD-68087: HIVE-28126: Use added record count in cost model when rebuilding materialized view stored by iceberg
  • CDPD-68654: HIVE-28172: Iceberg: Truncate should work for V1 tables with partition evolution.
  • CDPD-68285: HIVE-28166: Iceberg: Truncate on branch operates on the main table.
  • DWX-17402: HIVE-28069: Iceberg: Implement Merge task functionality for Iceberg tables - Add serde properties for CTAS and CMV
  • CDPD-67867: HIVE-28130: Iceberg: Add support for Rename Branch.
  • DWX-17957: HIVE-28167: Full table deletion fails when converting to truncate for Iceberg and ACID tables
  • DWX-17926: HIVE-28020: Iceberg: Upgrade iceberg version to 1.4.3
  • DWX-17926: HIVE-27819: Iceberg: Upgrade iceberg version to 1.4.2
  • CDPD-66779: HIVE-28087: Iceberg: Timestamp partition columns with transforms are not correctly sorted during insert
  • DWX-17402: HIVE-28069: Iceberg: Implement Merge task functionality for Iceberg tables
  • CDPD-66865: HIVE-28119: Iceberg: Allow insert clause with a column list in Merge query not_matched condition
  • DWX-16464: HIVE-27653: Iceberg: Add conflictDetectionFilter to validate concurrently added data and delete files
  • CDPD-67008: HIVE-28021: Iceberg: Attempting to create a table with a percent symbol fails
  • CDPD-67521: HIVE-28015: Iceberg: Add identifier-field-ids support in Hive
  • CDPD-67286: HIVE-28102: Iceberg: Invoke validateDataFilesExist for RowDelta operations.
  • CDPD-67120: HIVE-28084: Iceberg: COW fix for Merge operation
  • DWX-15939: HIVE-27850: Iceberg: Addendum 2: Set runAs user in CompactionInfo
  • CDPD-66414: HIVE-28057: Iceberg: Branches with non-lowercase characters can't be accessed.
  • DWX-17513: HIVE-27980: Iceberg: Compaction: Add support for OPTIMIZE TABLE syntax
  • DWX-17274: Cache Iceberg delete files in LLAP cache
  • DWX-15939: HIVE-27850: Iceberg: Addendum: Use fully qualified table name in compaction query
  • CDPD-65978: HIVE-27409: Iceberg: table with EXTERNAL type can not use statistics to optimize the query
  • CDPD-65863: HIVE-27593: Iceberg: Keep Iceberg properties in sync with HMS
  • CDPD-65867: HIVE-27360: Iceberg: Don't create the redundant MANAGED location when creating table without EXTERNAL keyword
  • DWX-17270: HIVE-27991: Utilise FanoutWriters when inserting records in an Iceberg table when the records are unsorted
  • CDPD-65654: Move the iceberg version dependency to parent pom
  • CDPD-65755: HIVE-28016: Iceberg: NULL column values handling in COW mode
  • DWX-13719: Iceberg positional deletes cache support
Impala - Iceberg changes
  • IMPALA-13046: Update Iceberg mixed format deletes test
  • IMPALA-13002: Iceberg V2 tables with Avro delete files aren't read properly
  • IMPALA-12991: Eliminate unnecessary SORT for Iceberg DELETEs
  • IMPALA-13003: Handle Iceberg AlreadyExistsException
  • IMPALA-12612: SELECT * queries expand complex type columns from Iceberg metadata tables
  • IMPALA-12893: Update tests for Iceberg 1.4.3
  • IMPALA-12611: Add support to MAP type Iceberg Metadata table columns
  • IMPALA-12600: Schema evolution with equality delete files
  • IMPALA-12609: Implement SHOW METADATA TABLES IN statement to list Iceberg Metadata tables
  • IMPALA-12809: Iceberg metadata table scanner should always be scheduled to the coordinator
  • IMPALA-12894: Fix optimized count(*) for Iceberg tables with dangling delete files
  • IMPALA-12729: Allow creating primary keys for Iceberg tables
  • IMPALA-12879: Conjunct not referring to table field causes ERROR for Iceberg table
  • IMPALA-12893: Specify 'format-version' explicitly in Iceberg tests
  • IMPALA-12894: Turn off the count(*) optimisation for V2 Iceberg tables
  • IMPALA-12845: Crash with DESCRIBE on a complex type from an Iceberg table
  • IMPALA-12610: Support reading ARRAY columns for Iceberg Metadata tables
  • IMPALA-12598: Allow multiple equality field id lists for Iceberg tables
  • IMPALA-12412: Support partition evolution in OPTIMIZE statement
  • IMPALA-12808: test_iceberg_deletes_and_updates failed by timeout error
  • IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile
  • IMPALA-12787: Concurrent DELETE and UPDATE operations on Iceberg tables can be problematic
  • IMPALA-12764: Fix Iceberg metadata table scan's LIMIT clause
  • IMPALA-12742: Add interop tests for DATE-partitioned Iceberg tables
  • IMPALA-12765: Balance consecutive partitions better for Iceberg tables
  • IMPALA-12742: DELETE/UPDATE Iceberg table partitioned by DATE fails with error
  • IMPALA-12708: An UPDATE creates 2 new snapshots in Iceberg tables
  • IMPALA-11710: Table properties are not updated in Iceberg metadata files
  • IMPALA-12704: Fix NPE when quering empty iceberg table's metadata
  • IMPALA-12706: Fix nested struct querying for Iceberg metadata tables
  • IMPALA-12495: Describe statement for Iceberg metadata tables

For more information about the core Apache Iceberg 1.4.3 changes, see the Apache Iceberg Release Notes.