List of required configuration parameters for the Kafka to ADLS Avro ReadyFlow

When deploying the Kafka to ADLS Avro ReadyFlow, you have to provide the following parameters. Use the information you collected in Prerequisites.

Table 1. Kafka to ADLS Avro ReadyFlow configuration parameters
Parameter Name	Description
ADLS File System	Specify the name of the ADLS data container you want to write to. The full path will be constructed from: abfs://{ADLS File System}@#{ADLS Storage Account}.dfs.core.windows.net/#{ADLS Path}/${Kafka.topic}
ADLS Path	Specify the path within the ADLS data container where you want to write to without any leading characters. The full path will be constructed from: abfs://{ADLS File System}@#{ADLS Storage Account}.dfs.core.windows.net/#{ADLS Path}/${Kafka.topic}
ADLS Storage Account	Specify the storage account name you want to write to. The full ADLS data container path will be constructed from: abfs://{ADLS File System}@#{ADLS Storage Account}.dfs.core.windows.net/#{ADLS Path}/${Kafka.topic}
CDP Workload User	Specify the CDP machine user or workload username that you want to use to authenticate to Kafka and the object store. Ensure this user has the appropriate access rights in Ranger for the Kafka topic and Ranger or IDBroker for object store access.
CDP Workload User Password	Specify the password of the CDP machine user or workload user you are using to authenticate against Kafka and the object store.
CSV Delimiter	If your source data is CSV, specify the delimiter here.
Data Input Format	Specify the format of your input data. CSV JSON AVRO
Kafka Broker Endpoint	Specify the Kafka bootstrap servers string as a comma separated list.
Kafka Consumer Group ID	The name of the consumer group used for the source topic you are consuming from.
Kafka Source Topic	Specify a topic name that you want to read from.
Schema Name	Specify the schema name to be looked up in the Schema Registry for the source Kafka topic.
Schema Registry Hostname	Specify the hostname of the Schema Registry you want to connect to. This must be the direct hostname of the Schema Registry itself, not the Knox Endpoint.