What is Cloudera Flow Management?

Learn about Cloudera Flow Management, a no-code data ingestion and management solution powered by Apache NiFi and NiFi Registry.

Cloudera Flow Management (CFM) is a key part of the Cloudera DataFlow (CDF) platform. It helps you to capture any type of data from any source and move it into the enterprise securely, while transforming and dynamically routing this data in between. It is suitable for highly scalable data movement, transformation, and management. CFM integrates seamlessly with CDF components such as Kafka, MiNiFi, Flink, and Spark Streaming, or platform components such as HBase, Hive, Impala, and Kudu.

With CFM, you can build complex data flows, manage, edit, and monitor these flows in real time. You can also adopt a DevOps style flow development lifecycle to deliver your flow applications faster by using templates, version control, collaboration, and various deployment options.

CFM has two main components:

  • Apache NiFi – the core data ingestion engine with no-code graphical user interface. It supports processors for connectivity, transformation, and content routing.

  • Apache NiFi Registry – the companion to NiFi that enables DevOps style development of flows. It supports versioning flows, promoting flows and deploying flow applications across environments.

What is NiFi?

Apache NiFi is a powerful, flexible, extensible, and reliable system to distribute and process data. It is data source agnostic and supports sources of different formats, schemas, protocols, speeds, and sizes. You can design and automate complex data flows on a configurable platform using 300+ processors. You can build your data flow by simply dragging and dropping a series of processors using the intuitive graphical user interface of Apache NiFi. When the data flow is ready, you can monitor and control the pipeline in real time. For more information on Apache NiFi, see the Apache NiFi User Guide.

What is NiFi Registry?

Registry is a complementary application that provides a central location for storing and managing shared and versioned data flow resources. It helps you with the full flow life cycle development. It integrates with NiFi to allow storing, retrieving, and upgrading versioned flows. When designing a data flow in NiFi, you may need to define and test different approaches and work incrementally towards your final goal. NiFi Registry helps you to keep track of several flow versions and navigate between them easily. For more information, see the Apache NiFi Registry User Guide.