Cloudera Docs

What is Cloudera Flow Management?

Cloudera Flow Management (CFM) powered by Apache NiFi and NiFi Registry is a key part of the Cloudera DataFlow (CDF) platform. You can use CFM to ingest and manage data without writing a single line of code. It helps you to capture any type of data from any source and move it into the enterprise securely, while you transform and dynamically route this data in between.

You can build complex and scalable data flows, and you can manage, edit, and monitor these flows in real time with CFM. You can also adopt a DevOps style flow development lifecycle to deliver your flow applications faster by using templates, version control, collaboration, and various deployment options.

Cloudera Flow Management integrates seamlessly with CDF components such as Kafka, MiNiFi, Flink, and Spark Streaming, or platform components such as HBase, Hive, Impala, and Kudu.

CFM has two main components:

  • Apache NiFi – the core data ingestion engine with a no-code graphical user interface. It supports processors for connectivity, transformation, and content routing.

  • Apache NiFi Registry – the companion to NiFi that enables DevOps style development of flows. It supports versioning flows, promoting flows, and deploying flow applications across environments.

What is NiFi?

Apache NiFi is a powerful, flexible, extensible, and reliable system to distribute and process data. It is data source agnostic and supports sources of different formats, schemas, protocols, speeds, and sizes. You can design and automate complex data flows on a configurable platform using 300+ processors. You can build your data flow by simply dragging and dropping a series of processors using the intuitive graphical user interface of Apache NiFi. When the data flow is ready, you can monitor and control the pipeline in real time. For more information on Apache NiFi, see the Apache NiFi User Guide.

What is NiFi Registry?

Registry is a complimentary application that provides a central location for storing and managing shared and versioned data flow resources. It helps you with the full flow life cycle development. It integrates with NiFi to allow storing, retrieving, and upgrading versioned flows. When designing a data flow in NiFi, you may need to define and test different approaches and work incrementally towards your final goal. NiFi Registry helps you to keep track of several flow versions and navigate between them easily. For more information, see the Apache NiFi Registry User Guide.