ReadyFlow Overview: Kafka to Iceberg

You can use the Kafka to Apache Iceberg ReadyFlow to move your data from a Cloudera-managed Kafka topic to a Cloudera-managed Apache Iceberg table in Cloudera Data Warehouse.

This ReadyFlow consumes JSON, CSV or Avro data from a Kafka topic, parses the schema by looking up the schema name in the Cloudera Schema Registry and ingests it into an Iceberg table. The flow writes out a file every time its size has either reached 100MB or five minutes have passed. Files can reach a maximum size of 1GB. You can specify the topic you want to read from as well as the target Iceberg table. Failed ingestion operations are retried automatically to handle transient issues. Define a KPI on the failure_WriteToIceberg connection to monitor failed write operations.