Iceberg tables and connectors in Cloudera SQL Stream Builder

Prerequisites

Iceberg, Hive, Hadoop, and Kudu connectors ship in the alternative Flink image. Enable it during install as described in Installation overview. Cloudera SQL Stream Builder exposes Iceberg as a first-class catalog type under Data Sources and Catalogs.

Kafka to Iceberg live upsert

Streaming upsert into Iceberg tables requires exactly-once checkpointing. Without checkpointing, the Iceberg sink cannot commit incremental files reliably and the job may fail or produce duplicates.

Enable checkpointing in the Flink configuration for the job, for example:

execution.checkpointing.interval: 60s
state.checkpoints.dir: s3://<bucket>/checkpoints/
state.savepoints.dir: s3://<bucket>/savepoints/

Configure durable checkpoint storage through ssb.storageConfiguration or per-job Advanced Settings before you start a Kafka to Iceberg upsert pipeline.

Iceberg table and catalog properties

When you create an Iceberg table in Cloudera SQL Stream Builder, set catalog and table options that match your object store and metastore:

• For S3-backed catalogs, use the alternative image and ensure Hadoop S3A settings are available through storageConfiguration or mounted core-site.xml.
• Specify the Iceberg catalog type (for example Hive or Hadoop) and warehouse location in the catalog DDL.
• For upsert mode, define primary key columns and set the Iceberg write mode to upsert in the table DDL or sink configuration.

Iceberg catalogs that use s3a paths resolve the filesystem implementation correctly when the alternative image is enabled.