Apache Iceberg in Cloudera
Apache Iceberg is a cloud-native, high-performance open table format for organizing petabyte-scale analytic datasets on a file system or object store. Combined with Cloudera, users can build an open data lakehouse architecture for multi-function analytics and to deploy large scale end-to-end pipelines.
Open data lakehouse on Cloudera simplifies advanced analytics on all data with a unified platform for structured and unstructured data and integrated data services to enable any analytics use case from ML, BI to stream analytics and real-time analytics. Apache Iceberg is the secret sauce of the open lakehouse.
The following table shows the support for Iceberg in Cloudera and below the table Iceberg versions v1 and v2 are defined:
Release |
Iceberg support level | SQL Engine | ||||
---|---|---|---|---|---|---|
Impala | Hive | Spark | NiFi | Flink | ||
Cloudera Public Cloud Data Services | GA | v1, v2: create table, read, insert, delete, update | v1, v2: create table, read, insert, update, and delete | v1, v2: create table, read, insert, update, and delete | v1, v2: read and insert | N/A |
Cloudera Data Hub 7.2.16.2 | GA | v1, v2: create table, read | v1: create table, read, insert, update, delete | v1, v2: create table, read, insert, update, and delete | v1, v2: read and insert | v1: create table, read, and insert |
Cloudera Data Hub 7.2.17 | GA | v1, v2: create table, read, insert, | v1, v2: create table, read, insert, update, delete | v1, v2: create table, read, insert, update, and delete | v1, v2: read and insert | v1, v2: create table, read, append, overwrite *** |
Cloudera Data Hub 7.2.18 | GA | v1, v2: create table, read, insert, update, and delete | v1, v2: create table, read, insert, update, delete | v1, v2: create table, read, insert, update, and delete | v1, v2: read and insert | v1, v2: create table, read, append, overwrite *** |
Cloudera Private Cloud Data Services 1.5.1 2023.0.13.0-20 | Technical Preview (Cloudera Private Cloud Base 7.1.7, 7.1.8) | v1, v2: create table, read | v1, v2: create table, read, insert, update, and delete | v1, v2: create table, read, insert, update, and delete | No Private Cloud support | No Private Cloud support |
Cloudera Private Cloud Data Services 1.5.2 | GA (7.1.9 Base) Technical Preview (Cloudera Private Cloud Base 7.1.7, 7.1.8) | v1, v2: create table, read, insert, and delete | v1, v2: create table, read, insert, update, and delete | v1, v2: create table, read, insert, update, and delete | v1, v2: read and insert (7.1.9 Base) | v1, v2: create table, read, and insert (7.1.9 Base) |
Cloudera Private Cloud Data Services 1.5.3 | GA (Cloudera Private Cloud Base 7.1.9) Technical Preview (Cloudera Private Cloud Base 7.1.7, 7.1.8) | v1, v2: create table, read, insert, update, and delete | v1, v2: create table, read, insert, update, and delete | v1, v2: create table, read, insert, update, and delete | v1, v2: read and insert (7.1.9 Base) | v1, v2: create table, read, and insert (7.1.9 Base) |
Cloudera Private Cloud Base 7.1.7 SP2, 7.1.8 | No Iceberg support | |||||
Cloudera Private Cloud Base 7.1.9 | GA | v1, v2: create table, read, insert | No Iceberg support | v1, v2: create table, read, insert, update, and delete |
v1, v2: read and insert |
v1, v2: create table, read, and insert |
** The support for delete operations, except from Flink, shown in this table is limited to position deletes. Equality deletes are not supported in these releases except from Flink.
*** Iceberg v2 updates and deletes from Flink are a technical preview in Cloudera Public Cloud 7.2.17.
- v1
Defines large analytic data tables using open format files.
- v2
Specifies ACID compliant tables including row-level deletes and updates.