Ingesting data into Kafka You can use an Apache NiFi data flow to ingest data into Apache Kafka by following these steps. Understand the use caseLearn how to use NiFi to move data from a range of locations into Kafka in CDP Public Cloud. Meet the prerequisitesUse this checklist to make sure that you meet all the requirements before you start building your data flow.Build the data flowLearn how to create NiFi data flows easily with a number of processors and flow objectives to choose from.Create controller services for your data flowLearn how to create and configure controller services for the Kafka ingest data flow.Configure the processor for your data sourceLearn how to configure a data source processor for the Kafka ingest data flow.Configure the processor for your data targetLearn how to configure a data target processor for the Kafka ingest data flow.Start the data flowLearn how to start your Kafka ingest data flow.Verify data flow operationLearn how you can verify the operation of your Kafka ingest data flow.Monitoring end to end latency for Kafka topicTo monitor end-to-end latency of Kafka topics, you can monitor the count of consumed messages across all Kafka consumer groups and the time taken by all Kafka consumer groups to consume messages that are produced in a Kafka topic, in a graphical way. You can also monitor the same for each Kafka consumer group, each client in a Kafka consumer group, and each partition in a Kafka topic.
