Homepage
/
Cloudera Flow Management
4.0.0
(on premises • technical preview)
Search Documentation
▶︎
Cloudera
Reference Architectures
▶︎
Cloudera on cloud
Getting Started
Patterns
Preview Features
Data Catalog
Data Engineering
DataFlow
Data Hub
Data Warehouse
Data Warehouse Runtime
Cloudera AI
Management Console
Operational Database
Replication Manager
DataFlow for Data Hub
Runtime
▼
Cloudera on premises
Data Services
Getting Started
Cloudera Manager
Management Console
Replication Manager
Data Catalog
Data Engineering
Data Warehouse
Data Warehouse Runtime
Cloudera AI
Base
Getting Started
Runtime
Upgrade
Storage
Flow Management
Streaming Analytics
Flow Management Operator
Streaming Analytics Operator
Streams Messaging Operator
▶︎
Cloudera Manager
Cloudera Manager
▶︎
Applications
Cloudera Streaming Community Edition
Data Science Workbench
Data Visualization
Edge Management
Observability SaaS
Observability on premises
Workload XM On-Prem
▶︎
Legacy
Cloudera Enterprise
Flow Management
Stream Processing
HDP
HDF
Streams Messaging Manager
Streams Replication Manager
▶︎
Data Services
Getting Started
Cloudera Manager
Management Console
Replication Manager
Data Catalog
Data Engineering
Data Warehouse
Data Warehouse Runtime
Cloudera AI
Base
Getting Started
Runtime
Upgrade
Storage
Flow Management
Streaming Analytics
Flow Management Operator
Streaming Analytics Operator
Streams Messaging Operator
«
Filter topics
Cloudera Flow Management
▶︎
Release Notes
What's new
▶︎
Support matrix
Component versions
System requirements
Supported operating systems
Supported NiFi Registry databases
Supported NiFi processors
Supported NiFi controller services
Supported NiFi reporting tasks
Supported NiFi parameter providers
Supported NiFi flow analysis rules
Supported Cloudera exclusive components
Download locations
Unsupported features
Behavioral changes
Known issues
▶︎
Overview
What is NiFi?
What is NiFi Registry?
Cloudera Manager integration
▶︎
Planning
Data flow design
Cluster layout
Disk configuration
Resource-intensive processors
Sizing recommendations
▶︎
Installation & Upgrade
▶︎
Installation
Cloudera Flow Management installation workflow
▶︎
Preparing your Cloudera Base on premises cluster
Installing and configuring Python
Installing and configuring JDK
▶︎
Installing and configuring a database for NiFi Registry
Installing MySQL
Configuring NiFi Registry Metadata Stores in MySQL
Installing PostgreSQL
Configuring NiFi Registry Metadata Stores in PostgreSQL
Installing Cloudera Manager and a Cloudera Base on premises cluster
Installing the Cloudera Flow Management parcel from the repository
Downloading Custom Service Descriptor files
▶︎
Installing NiFi and NiFi Registry on your Base cluster
Adding the NiFi service
Adding the NiFi Registry service
Connecting NiFi to NiFi Registry
Adding users or groups to Ranger policies
Deselecting unwanted NiFi Registry dependencies
▶︎
Installing NiFi and NiFi Registry on a Compute cluster
Creating a Shared Data Context
Adding the NiFi and NiFi Registry groups to Ranger in the Base cluster
Creating a Compute cluster
Adding the NiFi and NiFi Registry services to a Compute cluster
Connecting NiFi to NiFi Registry
Adding users or groups to Ranger policies
Deselecting unwanted NiFi Registry dependencies
▶︎
Installing NiFi on a Compute cluster and NiFi Registry on a Base cluster
Installing NiFi Registry on your Base cluster
Creating a Shared Data Context
Adding the NiFi and NiFi Registry groups to Ranger in the Base cluster
Creating Compute cluster
Adding NiFi to the Compute cluster
Setting up NiFi node users in NiFi Registry
Connecting NiFi to NiFi Registry
Adding users or groups to Ranger policies
Deselecting unwanted NiFi Registry dependencies
▶︎
Upgrade considerations
Cloudera Flow Management 4.0.0 Technical Preview limitations
Breaking changes in NiFi 2
Pre-upgrade check for NiFi
▶︎
Cloudera Flow Management Migration Tool
Migration Tool Release Notes
Migration Tool overview
Recommended migration workflow
Migration best practices
▶︎
Migrating a data flow
▶︎
Preparing for migration
Setting up your environment
Configuring the Migration Tool
▶︎
Migrating a flow
Migrating templates
Migrating variables
Migrating components
Using migrate-all
▶︎
Cloudera Flow Management Migration Tool Reference
Commands
Activity Log
Troubleshooting flow migration issues
▶︎
Getting Started
What is Cloudera Flow Management?
▶︎
Setting up Cloudera Flow Management
Download NiFi and NiFi Registry
Install NiFi and NiFi Registry
Start NiFi and NiFi Registry
Building your first data flow
Connecting NiFi to NiFi Registry
Creating a bucket in NiFi Registry
▶︎
Versioning data flows
Start version control
Commit local changes
Change the version of your data flow
Manage versioned data flows
▶︎
How To
▶︎
Managing NiFi in Cloudera Manager
Using the Actions menu
Managing NiFi nodes
▶︎
Working with dataflows
▶︎
Tuning your dataflow
Tuning your data flow
▶︎
Timer and event driven thread pools
Viewing the total number of threads in a cluster
Viewing the total number of active threads
Viewing the number of cores
Configuring thread pool size
Concurrent tasks
Run duration
Recommendations
▶︎
Using Parameter Context Inheritance
What is Parameter Context inheritance?
▶︎
Example for configuring Parameter Context inheritance
Creating the basic Parameter Contexts
Setting up Parameter Context inheritance
Parameter overriding
▶︎
Using Parameter Providers
What are Parameter Providers?
▶︎
Example for using Parameter Providers
Creating and configuring a Parameter Provider
Fetching Parameters
Creating a Parameter Context from a Parameter Group
Updating Parameter sensitivity
Updating Parameter Context when the external source has changed
Using Parameter Context inheritance to combine Parameters
▶︎
Using DataFlow Catalog Registry Client
Creating a Machine User
Adding a new Registry Client
Checking out a ReadyFlow
Checking out your flows
Versioning a flow in the Catalog
▶︎
Using Flow Library Registry Client
Adding Cloudera Flow Library Registry Client
Checking out a flow
▶︎
Accessing NiFi Registry API Swagger UI
Authentication and permissions
Access and usage
▶︎
Moving data using NiFi site-to-site
▶︎
Moving data from Cloudera on premises to Cloudera on cloud with NiFi site-to-site
Understanding the use case
Preparing your clusters
Setting up your network configuration
Configuring your truststores
Defining your Cloudera on cloud data flow
Configuring Ranger policies for site-to-site communication
Defining your Cloudera Base on premises data flow
▶︎
Moving data between Cloudera and other systems
▶︎
Moving data in and out of Snowflake
Pushing data to and moving data from Snowflake using NiFi
▶︎
Moving data out of Snowflake
Before you begin
Downloading the Snowflake JDBC driver JAR file
Adding Snowflake CA certificates to NiFi truststore
Building your data flow
Creating Controller Services for your data flow
Configuring your source processor
Configuring your target processor
Confirming your data flow success
▶︎
Pushing data into Snowflake
Before you begin
Adding Snowflake CA certificates to NiFi truststore
Building your dataflow
Configuring your Controller Services
Configure your source processor
Configuring your target processor
Confirming your data flow success
Next steps
▶︎
Moving data from Salesforce
▶︎
Moving data out of Salesforce using Apache NiFi
▶︎
Moving data out of Salesforce
Before you begin
Configuring Salesforce
Building your dataflow
▶︎
Creating Controller Services for your dataflow
StandardOauth2AccessTokenProvider configurations
Configuring your source processor
Configuring your target processor
Confirming your dataflow success
▶︎
Moving data to Ozone
▶︎
Pushing data to Ozone using NiFi
Before you begin
Building your dataflow
Configuring your source processor
Configuring your target processor
Starting your dataflow
▶︎
Moving data to Cloudera Data Warehouse using Iceberg
▶︎
Moving data into Cloudera Data Warehouse on Cloudera Data Services on premises using Iceberg table format
Understanding the use case
Prerequisites
Creating Iceberg target table
Building data flow
Creating and configuring controller services
Configuring processor for data source
Configuring processor for data target
Starting the data flow
▶︎
Security
Recommendations
Identity and policies
▶︎
TLS/SSL configuration
Enabling Auto-TLS
▶︎
Configuring TLS/SSL manually
Requirements and recommendations
Configuring TLS/SSL encryption manually
NiFi TLS/SSL properties
NiFi Registry TLS/SSL properties
▶︎
Authentication
▶︎
Kerberos authentication
Customizing Kerberos principal
LDAP authentication
SAML authentication
OpenID Connect authentication
Identity mapping properties
▶︎
Authorization
▶︎
User group providers
▶︎
LDAP integration
LDAP and Ranger policies
LDAP and file-based policies
LDAP User Group Provider properties
Pairing LDAP with a Composite Group Provider
▶︎
Access policies providers
▶︎
Ranger authorization
Understanding Ranger authorization
Ranger-based NiFi policy descriptions
Predefined Ranger access policies for NiFi
Predefined Ranger access policies for NiFi Registry
Predefined component-level policies for NiFi
NiFi restricted components
Adding user to a pre-defined Ranger access policy
Creating a custom Ranger access policy
Authorization example
Enabling access to Knox and NiFi
File-based authorization
▶︎
Migrating file-based authorization to Ranger
Migrating NiFi file-based authorization to Ranger
Migrating NiFi Registry file-based authorization to Ranger
▶︎
Environment variables
Kerberos credentials
Local file system access
▶︎
Network
Default ports for NiFi and NiFi Registry
▶︎
Cryptography
▶︎
FIPS 140-2 compliance
Encrypting NiFi sensitive properties with FIPS 140-2 approved algorithm
Configuring NiFi to use GCM ciphers
▶︎
Integrations
▶︎
Integrating NiFi and Atlas
Manually integrating with Atlas when Auto-TLS is not enabled
Manually integrating with Atlas when Auto-TLS is enabled
Integrating NiFi and NiFi Registry with Knox
Customizing properties in Cloudera Manager
▼
Apache NiFi and NiFi Registry documentation
Getting started with Apache NiFi
Using Apache NiFi
Apache NiFi Expression Language Guide
Apache NiFi RecordPath Guide
Apache NiFi System Administrator's Guide
Using Apache NiFi Toolkit
Apache NiFi Developer Guide
Getting started with Apache NiFi Registry
Using Apache NiFi Registry
Apache NiFi Registry System Administrator's Guide
Apache NiFi Registry REST API
Access and usage
Access policies providers
Accessing NiFi Registry API Swagger UI
Activity Log
Adding a new Registry Client
Adding Cloudera Flow Library Registry Client
Adding NiFi to the Compute cluster
Adding Snowflake CA certificates to NiFi truststore
Adding Snowflake CA certificates to NiFi truststore
Adding the NiFi and NiFi Registry groups to Ranger in the Base cluster
Adding the NiFi and NiFi Registry groups to Ranger in the Base cluster
Adding the NiFi and NiFi Registry services to a Compute cluster
Adding the NiFi Registry service
Adding the NiFi service
Adding user to a pre-defined Ranger access policy
Adding users or groups to Ranger policies
Adding users or groups to Ranger policies
Adding users or groups to Ranger policies
Apache NiFi and NiFi Registry documentation
Apache NiFi Developer Guide
Apache NiFi Expression Language Guide
Apache NiFi RecordPath Guide
Apache NiFi Registry REST API
Apache NiFi Registry System Administrator's Guide
Apache NiFi System Administrator's Guide
Authentication
Authentication and permissions
Authorization
Authorization example
Before you begin
Before you begin
Before you begin
Before you begin
Behavioral changes
Breaking changes in NiFi 2
Building data flow
Building your data flow
Building your dataflow
Building your dataflow
Building your dataflow
Building your first data flow
Change the version of your data flow
Checking out a flow
Checking out a ReadyFlow
Checking out your flows
Cloudera Flow Management
Cloudera Flow Management 4.0.0 Technical Preview limitations
Cloudera Flow Management installation workflow
Cloudera Flow Management Migration Tool
Cloudera Flow Management Migration Tool Reference
Cloudera Manager integration
Cluster layout
Commands
Commit local changes
Component versions
Concurrent tasks
Configure your source processor
Configuring NiFi Registry Metadata Stores in MySQL
Configuring NiFi Registry Metadata Stores in PostgreSQL
Configuring NiFi to use GCM ciphers
Configuring processor for data source
Configuring processor for data target
Configuring Ranger policies for site-to-site communication
Configuring Salesforce
Configuring the Migration Tool
Configuring thread pool size
Configuring TLS/SSL encryption manually
Configuring TLS/SSL manually
Configuring your Controller Services
Configuring your source processor
Configuring your source processor
Configuring your source processor
Configuring your target processor
Configuring your target processor
Configuring your target processor
Configuring your target processor
Configuring your truststores
Confirming your data flow success
Confirming your data flow success
Confirming your dataflow success
Connecting NiFi to NiFi Registry
Connecting NiFi to NiFi Registry
Connecting NiFi to NiFi Registry
Connecting NiFi to NiFi Registry
Creating a bucket in NiFi Registry
Creating a Compute cluster
Creating a custom Ranger access policy
Creating a Machine User
Creating a Parameter Context from a Parameter Group
Creating a Shared Data Context
Creating a Shared Data Context
Creating and configuring a Parameter Provider
Creating and configuring controller services
Creating Compute cluster
Creating Controller Services for your data flow
Creating Controller Services for your dataflow
Creating Iceberg target table
Creating the basic Parameter Contexts
Cryptography
Customizing Kerberos principal
Customizing properties in Cloudera Manager
Data flow design
Default ports for NiFi and NiFi Registry
Defining your Cloudera Base on premises data flow
Defining your Cloudera on cloud data flow
Deselecting unwanted NiFi Registry dependencies
Deselecting unwanted NiFi Registry dependencies
Deselecting unwanted NiFi Registry dependencies
Disk configuration
Download locations
Download NiFi and NiFi Registry
Downloading Custom Service Descriptor files
Downloading the Snowflake JDBC driver JAR file
Enabling access to Knox and NiFi
Enabling Auto-TLS
Encrypting NiFi sensitive properties with FIPS 140-2 approved algorithm
Environment variables
Example for configuring Parameter Context inheritance
Example for using Parameter Providers
Fetching Parameters
File-based authorization
FIPS 140-2 compliance
Getting Started
Getting started with Apache NiFi
Getting started with Apache NiFi Registry
Identity and policies
Identity mapping properties
Install NiFi and NiFi Registry
Installation
Installing and configuring a database for NiFi Registry
Installing and configuring JDK
Installing and configuring Python
Installing Cloudera Manager and a Cloudera Base on premises cluster
Installing MySQL
Installing NiFi and NiFi Registry on a Compute cluster
Installing NiFi and NiFi Registry on your Base cluster
Installing NiFi on a Compute cluster and NiFi Registry on a Base cluster
Installing NiFi Registry on your Base cluster
Installing PostgreSQL
Installing the Cloudera Flow Management parcel from the repository
Integrating NiFi and Atlas
Integrating NiFi and NiFi Registry with Knox
Integrations
Kerberos authentication
Kerberos credentials
Known issues
LDAP and file-based policies
LDAP and Ranger policies
LDAP authentication
LDAP integration
LDAP User Group Provider properties
Local file system access
Manage versioned data flows
Managing NiFi in Cloudera Manager
Managing NiFi nodes
Manually integrating with Atlas when Auto-TLS is enabled
Manually integrating with Atlas when Auto-TLS is not enabled
Migrating a data flow
Migrating a flow
Migrating components
Migrating file-based authorization to Ranger
Migrating NiFi file-based authorization to Ranger
Migrating NiFi Registry file-based authorization to Ranger
Migrating templates
Migrating variables
Migration best practices
Migration Tool overview
Migration Tool Release Notes
Moving data between Cloudera and other systems
Moving data from Cloudera on premises to Cloudera on cloud with NiFi site-to-site
Moving data from Salesforce
Moving data in and out of Snowflake
Moving data into Cloudera Data Warehouse on Cloudera Data Services on premises using Iceberg table format
Moving data out of Salesforce
Moving data out of Salesforce using Apache NiFi
Moving data out of Snowflake
Moving data to Cloudera Data Warehouse using Iceberg
Moving data to Ozone
Moving data using NiFi site-to-site
Network
Next steps
NiFi Registry TLS/SSL properties
NiFi restricted components
NiFi TLS/SSL properties
OpenID Connect authentication
Overview
Pairing LDAP with a Composite Group Provider
Parameter overriding
Planning
Pre-upgrade check for NiFi
Predefined component-level policies for NiFi
Predefined Ranger access policies for NiFi
Predefined Ranger access policies for NiFi Registry
Preparing for migration
Preparing your Cloudera Base on premises cluster
Preparing your clusters
Prerequisites
Pushing data into Snowflake
Pushing data to and moving data from Snowflake using NiFi
Pushing data to Ozone using NiFi
Ranger authorization
Ranger-based NiFi policy descriptions
Recommendations
Recommendations
Recommended migration workflow
Release Notes
Requirements and recommendations
Resource-intensive processors
Run duration
SAML authentication
Security
Setting up Cloudera Flow Management
Setting up NiFi node users in NiFi Registry
Setting up Parameter Context inheritance
Setting up your environment
Setting up your network configuration
Sizing recommendations
StandardOauth2AccessTokenProvider configurations
Start NiFi and NiFi Registry
Start version control
Starting the data flow
Starting your dataflow
Support matrix
Supported Cloudera exclusive components
Supported NiFi controller services
Supported NiFi flow analysis rules
Supported NiFi parameter providers
Supported NiFi processors
Supported NiFi Registry databases
Supported NiFi reporting tasks
Supported operating systems
System requirements
Timer and event driven thread pools
TLS/SSL configuration
Troubleshooting flow migration issues
Tuning your data flow
Tuning your dataflow
Understanding Ranger authorization
Understanding the use case
Understanding the use case
Unsupported features
Updating Parameter Context when the external source has changed
Updating Parameter sensitivity
Upgrade considerations
User group providers
Using Apache NiFi
Using Apache NiFi Registry
Using Apache NiFi Toolkit
Using DataFlow Catalog Registry Client
Using Flow Library Registry Client
Using migrate-all
Using Parameter Context Inheritance
Using Parameter Context inheritance to combine Parameters
Using Parameter Providers
Using the Actions menu
Versioning a flow in the Catalog
Versioning data flows
Viewing the number of cores
Viewing the total number of active threads
Viewing the total number of threads in a cluster
What are Parameter Providers?
What is Cloudera Flow Management?
What is NiFi Registry?
What is NiFi?
What is Parameter Context inheritance?
What's new
Working with dataflows
«
Filter topics
Getting started with Apache NiFi
Getting started with Apache NiFi
Using Apache NiFi
Apache NiFi Expression Language Guide
Apache NiFi RecordPath Guide
Apache NiFi System Administrator's Guide
Using Apache NiFi Toolkit
Apache NiFi Developer Guide
Getting started with Apache NiFi Registry
Using Apache NiFi Registry
Apache NiFi Registry System Administrator's Guide
Apache NiFi Registry REST API
»
Apache NiFi and NiFi Registry documentation
Getting started with Apache NiFi
For more information, see the
Apache NiFi Getting Started Guide
.
Feedback
We want your opinion
How can we improve this page?
What kind of feedback do you have?
I like something
I have an idea
Something's not working
Can we contact you for follow-up on this?
Back
Submit
OK
This site uses cookies and related technologies, as described in our
privacy policy
, for purposes that may include site operation, analytics, enhanced user experience, or advertising. You may choose to consent to our use of these technologies, or
manage your own preferences.
Accept all
4.0.0
2.1
2.1.7
2.1.6
2.1.5
2.1.4
2.1.3
2.1.2
2.1.1
2.0
2.0.4
2.0.1
