Cloudera Docs

List of required configuration parameters for the Kafka to Iceberg ReadyFlow

When deploying the Kafka to Iceberg ReadyFlow, you have to provide the following parameters. Use the information you collected in Prerequisites.

Table 1. Kafka to Iceberg ReadyFlow configuration parameters
Parameter Name Description
CDP Workload User Specify the CDP machine user or workload username that you want to use to authenticate to Kafka and the object store. Ensure this user has the appropriate access rights in Ranger for the Kafka topic and Ranger or IDBroker for Hive access.
CDP Workload User Password Specify the CDP machine user or workload username that you want to use to authenticate to Kafka and Hive. Ensure this user has the appropriate access rights in Ranger for the Kafka topic and ID Broker for Hive access.
CDPEnvironment The CDP Environment configuration resources.
CSV Delimiter If your source data is CSV, specify the delimiter here.
Data Input Format Specify the format of your input data. Possible values are:
  • CSV
  • JSON
  • AVRO
Hive Catalog Namespace Specify the Hive Catalog Namespace. The default value is default, which references the database catalog created automatically when you activate an environment in Cloudera Data Warehouse.
Iceberg Table Name Specify the Iceberg table that you want to write to, running in your Hive Virtual Warehouse.
Kafka Broker Endpoint Specify the Kafka bootstrap servers string as a comma separated list.
Kafka Consumer Group ID The name of the consumer group used for the source topic you are consuming from.
Kafka Source Topic Specify a topic name that you want to read from.
Schema Name Specify the schema name to be looked up in the Schema Registry for the source Kafka topic.
Schema Registry Hostname Specify the hostname of the Schema Registry you want to connect to. This must be the direct hostname of the Schema Registry itself, not the Knox Endpoint.