Understand the use case

Learn how to use NiFi to move data from a range of locations into Kafka in CDP Public Cloud.

Apache NiFi as a part of Cloudera Data Flow offers a scalable way of managing data flows with guaranteed delivery, data buffering / pressure release, and prioritized queuing.

You can use it to ingest data into Apache Kafka by simply dragging and dropping a series of processors on the NiFi user interface. In addition to facilitating the data flow into Kafka, the processors provide an out-of-the-box integration with Streams Messaging Manager (SMM) by configuring the metrics interceptors.

When the data flow is ready, you can visually monitor and control the pipeline you have built.

This use case walks you through the steps of creating a data flow that generates FlowFiles with random CSV data and writes this data to Apache Kafka in CDP Public Cloud. This gets you started with creating a Kafka ingest data flow. If you want to use a data source other than generated FlowFiles, see the Getting Started with Apache NiFi for information about how to build a data flow, and about get and consume data processor options.