Homepage
/
Cloudera DataFlow for Data Hub
7.2.9
(Public Cloud)
Search Documentation
▶︎
Cloudera
Reference Architectures
▼
Cloudera Public Cloud
Getting Started
Patterns
Preview Features
Data Catalog
Data Engineering
DataFlow
Data Hub
Data Warehouse
Data Warehouse Runtime
Cloudera AI
Management Console
Operational Database
Replication Manager
Cloudera Manager
CDF for Data Hub
Runtime
▶︎
Cloudera Private Cloud
Data Services
Getting Started
Cloudera Manager
Management Console
Replication Manager
Data Catalog
Data Engineering
Data Warehouse
Data Warehouse Runtime
Cloudera AI
Base
Getting Started
Runtime
Upgrade
Storage
Flow Management
Streaming Analytics
Flow Management Operator
Streaming Analytics Operator
Streams Messaging Operator
▶︎
Cloudera Manager
Cloudera Manager
▶︎
Applications
Cloudera Streaming Community Edition
Data Science Workbench
Data Visualization
Edge Management
Observability SaaS
Observability on premises
Workload XM On-Prem
▶︎
Legacy
Cloudera Enterprise
Flow Management
Stream Processing
HDP
HDF
Streams Messaging Manager
Streams Replication Manager
▶︎
Getting Started
Patterns
Preview Features
Data Catalog
Data Engineering
DataFlow
Data Hub
Data Warehouse
Data Warehouse Runtime
Cloudera AI
Management Console
Operational Database
Replication Manager
Cloudera Manager
CDF for Data Hub
Runtime
«
Filter topics
CDF for Data Hub
▶︎
Release Notes
▶︎
What's New in Cloudera DataFlow for Data Hub 7.2.9
What's New in Flow Management
What's New in Streams Messaging
What's New in Streaming Analytics
Component Support in Cloudera DataFlow for Data Hub 7.2.9
▶︎
Supported NiFi Extensions
Supported NiFi Processors
Supported NiFi Controller Services
Supported NiFi Reporting Tasks
▶︎
Unsupported Features in Cloudera DataFlow for Data Hub 7.2.9
Unsupported Flow Management features
Unsupported Streams Messaging features
Unsupported Streaming Analytics features
▶︎
Apache Patch Information in Cloudera DataFlow for Data Hub 7.2.9
NiFi patches
NiFi Registry patches
▶︎
Known Issues In Cloudera DataFlow for Data Hub 7.2.9
Known Issues in Flow Management
Known Issues in Streams Messaging
Known Issues in Streaming Analytics
▶︎
Fixed Issues in Cloudera DataFlow for Data Hub 7.2.9
Fixed Issues in Flow Management
Fixed Issues in Streams Messaging
Fixed Issues in Streaming Analytics
▶︎
Concepts
▶︎
Apache NiFi Overview
What is Apache NiFi?
The core concepts of NiFi
NiFi Architecture
Performance Expectations and Characteristics of NiFi
High Level Overview of Key NiFi Features
▶︎
Streams Messaging
▶︎
Apache Kafka Overview
Kafka Introduction
▶︎
Kafka Architecture
Brokers
Topics
Records
Partitions
Record order and assignment
Logs and log segments
Kafka brokers and Zookeeper
Leader positions and in-sync replicas
▶︎
Kafka FAQ
Basics
Use cases
▶︎
Schema Registry Overview
▶︎
Schema Registry Overview
Examples of Interacting with Schema Registry
▶︎
Schema Registry Use Cases
Use Case 1: Registering and Querying a Schema for a Kafka Topic
Use Case 2: Reading/Deserializing and Writing/Serializing Data from and to a Kafka Topic
Use Case 3: Dataflow Management with Schema-based Routing
Schema Registry Component Architecture
▶︎
Schema Registry Concepts
Schema Entities
Compatibility Policies
▶︎
Streams Messaging Manager Overview
Introduction to Streams Messaging Manager
▶︎
Streams Replication Manager Overview
Overview
Key Features
Main Use Cases
▶︎
Use Case Architectures
▶︎
Highly Available Kafka Architectures
Active / Stand-by Architecture
Active / Active Architecture
Cross Data Center Replication
▶︎
Cluster Migration Architectures
On-premise to Cloud and Kafka Version Upgrade
Aggregation for Analytics
▶︎
Streams Replication Manager Architecture
▶︎
Streams Replication Manager Driver
Connect workers
Connectors
Task architecture and load-balancing
Driver inter-node coordination
Streams Replication Manager Service
▶︎
Understanding Replication Flows
Replication Flows Overview
Remote Topics
Bi-directional Replication Flows
Fan-in and Fan-out Replication Flows
▶︎
Apache Flink Overview
What is Apache Flink?
Streaming use cases with Flink
▶︎
Flink Streaming Applications
Handling state in Flink
Event-driven applications with Flink
Sophisticated windowing in Flink
Using watermark in Flink
Creating checkpoints and savepoints in Flink
▶︎
Planning
▶︎
Planning your Flow Management deployment
Deployment scenarios
Flow Management cluster definitions
Flow Management cluster layout
▶︎
Planning your Streams Messaging deployment
Deployment scenarios
Data Hub cluster definitions
Streams Messaging cluster layout
▶︎
Planning your Streaming Analytics deployment
Streaming Analytics deployment scenarios
Streaming Analytics Data Hub cluster definitons
Streaming Analytics cluster layout
▶︎
How To: Flow Management
▶︎
Creating your First Flow Management Cluster in CDP Public Cloud
▶︎
Creating your first Flow Management cluster
Meet the prerequisites
Create your cluster
Give users access to your cluster
Next steps
▶︎
Authorizing Flow Management Cluster Access in CDP Public Cloud
Security for Flow Management Clusters and Users in CDP Public Cloud
▶︎
User Authorization
Authorization workflow
Assigning administrator level permissions
▶︎
Assigning selective permissions to user
Assign the EnvironmentUser role
Add the user to predefined Ranger access policies
Create a custom access policy
Authorization example
Predefined Ranger Access Policies for Apache NiFi
Predefined Ranger Access Policies for Apache NiFi Registry
▶︎
Moving Data using NiFi Site-to-Site
▶︎
Moving data from CDP Private Cloud Base to Public Cloud with NiFi site-to-site
Understand the use case
Prepare your clusters
Set up your network configuration
Configure your truststores
Define your CDP Public Cloud dataflow
Configure Ranger policies for site-to-site communication
Define your CDP Private Cloud Base dataflow
▶︎
Moving Data in and Out of Snowflake
Pushing data to and moving data from Snowflake using Apache NiFi
▶︎
Moving data out of Snowflake
Before you begin
Downloading the Snowflake JDBC driver jar file
Adding Snowflake CA certificates to NiFi truststore
Building your dataflow
Creating Controller Services for your dataflow
Configuring your source processor
Configuring your target processor
Confirming your dataflow success
▶︎
Pushing data into Snowflake
Before you begin
Adding Snowflake CA certificates to NiFi truststore
Building your dataflow
Configuring your Controller Services
Configure your source processor
Configuring your target processor
Confirming your dataflow success
Next steps
▶︎
Ingesting Data into CDP Public Cloud
▶︎
Ingesting Data into Apache Kafka in CDP Public Cloud
▶︎
Ingesting data into Apache Kafka
Understand the use case
Meet the prerequisites
Build the data flow
Create controller services for your data flow
Configure the processor for your data source
Configure the processor for your data target
Start the data flow
Verify data flow operation
Monitoring your data flow
Next steps
Appendix - Schema example
▶︎
Ingesting Data into Apache Hive in CDP Public Cloud
▶︎
Ingesting Data into Apache Hive in CDP Public Cloud
Understand the use case
Meet the prerequisites
Configure the service account
Create IDBroker mapping
Create the Hive target table
Add Ranger policies
Obtain Hive connection details
Build the data flow
Configure the controller services
Configure the processor for your data source
Configure the processor for your data target
Start your data flow
Verify your data flow
Next steps
▶︎
Ingesting Data into Apache HBase in CDP Public Cloud
▶︎
Ingesting Data into Apache HBase in CDP Cloud
Understand the use case
Meet the prerequisites
Create the HBase target table
Add Ranger policies
Obtain HBase connection details
Build the data flow
Configure the HBase client service
Configure the processor for your data source
Configure the processor for your data target
Start your data flow
Verify your data flow
Next steps
▶︎
Ingesting Data into Apache Kudu in CDP Public Cloud
▶︎
Ingesting Data into Apache Kudu in CDP Public Cloud
Understand the use case
Meet the prerequisites
Create the Kudu target table
Build the data flow
Configure the Controller Service
Configure the processor for your data source
Configure the processor for your data target
Start your data flow
Verify that you can write data to Kudu
Next steps
▶︎
Ingesting Data into Apache Solr in CDP Public Cloud
▶︎
Ingesting data into Apache Solr
Understand the use case
Meet the prerequisites
Create Solr target collection
Build the data flow
Create controller services for your data flow
Configure the processor for your data source
Configure the processor for your data target
Start the data flow
Verify data flow operation
Monitoring your data flow
Next steps
▶︎
Ingesting Data into Amazon S3 Buckets
▶︎
Ingesting data into Amazon S3
Understand the use case
Meet the prerequisites
Build the data flow
Set up AWS for your ingest data flow
Create IDBroker mapping
Create controller services for your data flow
Configure the processor for your data source
Configure the processor for merging records
Configure the processor for your data target
Start the data flow
Verify data flow operation
Monitoring your data flow
Next steps
▶︎
Ingesting Data into Azure Data Lake Storage
▶︎
Ingesting data into Azure Data Lake Storage
Understand the use case
Meet the prerequisites
Build the data flow
Create IDBroker mapping
Create controller services for your data flow
Configure the processor for your data source
Configure the processor for merging records
Configure the processor for your data target
Start the data flow
Verify data flow operation
Monitoring your data flow
Next steps
▶︎
Ingesting Data into Google Cloud Storage
▶︎
Ingesting data into Google Cloud Storage
Understand the use case
Meet the prerequisites
Build the data flow
Create IDBroker mapping
Create controller services for your data flow
Configure the processor for your data source
Configure the processor for merging records
Configure the processor for your data target
Start the data flow
Verify data flow operation
Monitoring your data flow
Viewing data lineage in Apache Atlas
Next steps
▶︎
Apache NiFi
▶︎
Using Apache NiFi
Introduction
▶︎
Browser Support
Unsupported Browsers
Viewing the UI in Variably Sized Browsers
Terminology
NiFi User Interface
Accessing the UI with Multi-Tenant Authorization
Logging In
▶︎
Building a DataFlow
Adding Components to the Canvas
▶︎
Component Versions
Sorting and Filtering Components
Changing Component Versions
Understanding Version Dependencies
▶︎
Configuring a Processor
Settings Tab
Scheduling Tab
Properties Tab
Comments Tab
Additional Help
▶︎
Configuring a Process Group
General Tab
Controller Services
▶︎
Parameters
Parameter Contexts
Adding a Parameter to a Parameter Context
Parameters and Expression Language
Assigning a Parameter Context to a Process Group
Referencing Parameters
Accessing Parameters
▶︎
Using Custom Properties with Expression Language
Variables
Referencing Custom Properties via nifi.properties
▶︎
Controller Services
Adding Controller Services for Reporting Tasks
Adding Controller Services for Dataflows
Enabling/Disabling Controller Services
Reporting Tasks
▶︎
Connecting Components
Details Tab
Settings
Changing Configuration and Context Menu Options
Bending Connections
Processor Validation
▶︎
Site-to-Site
Configure Site-to-Site client NiFi instance
Configure Site-to-Site Server NiFi Instance
Example Dataflow
▶︎
Command and Control of the DataFlow
Starting a Component
Stopping a Component
Terminating a Component's tasks
Enabling/Disabling a Component
▶︎
Remote Process Group Transmission
Individual Port Transmission
▶︎
Navigating within a DataFlow
Component Linking
▶︎
Component Alignment
Align Vertically
Align Horizontally
▶︎
Search Components in DataFlow
Filters
Keywords
▶︎
Monitoring of DataFlow
Anatomy of a Processor
Anatomy of a Process Group
Anatomy of a Remote Process Group
Queue Interaction
Summary Page
Historical Statistics of a Component
▶︎
Versioning a DataFlow
Connecting to a NiFi Registry
Version States
Import a Versioned Flow
Start Version Control
▶︎
Managing Local Changes
Show Local Changes
Revert Local Changes
Commit Local Changes
Change Version
Stop Version Control
Nested Versioned Flows
Parameters in Versioned Flows
Variables in Versioned Flows
▶︎
Restricted Components in Versioned Flows
Restricted Controller Service Created in Root Process Group
Restricted Controller Service Created in Process Group
▶︎
Templates
Creating a Template
Importing a Template
Instantiating a Template
▶︎
Managing Templates
Exporting a Template
Removing a Template
▶︎
Data Provenance
Provenance Events
Searching for Events
Details of an Event
Replaying a FlowFile
▶︎
Viewing FlowFile Lineage
Find Parents
Expanding an Event
▶︎
Write Ahead Provenance Repository
Backwards Compatibility
Older Existing NiFi Version
Bootstrap.conf
System Properties
Encrypted Provenance Considerations
▶︎
Encrypted Provenance Repository
What is it?
How does it work?
Writing and Reading Event Records
Potential Issues
▶︎
Encrypted Content Repository
What is it?
▶︎
How does it work?
StaticKeyProvider
FileBasedKeyProvider
Key Rotation
Writing and Reading Content Claims
Potential Issues
▶︎
Encrypted FlowFile Repository
What is it?
▶︎
How does it work?
StaticKeyProvider
FileBasedKeyProvider
Key Rotation
Writing and Reading FlowFiles
Potential Issues
Experimental Warning
Other Management Features
▶︎
Using the Apache NiFi Toolkit
▶︎
Overview
Prerequisites for Running in a Secure Environment
▶︎
NiFi CLI
Usage
Property/Argument Handling
▶︎
Security Configuration
Example - Secure NiFi Registry without Proxied-Entity
Example - Secure NiFi Registry with Proxied-Entity
Interactive Usage
Output
Back-Referencing
Adding Commands
▶︎
Encrypt-Config Tool
▶︎
Usage
NiFi
NiFi Registry
▶︎
Examples
NiFi
NiFi Registry
Sensitive Property Key Migration
▶︎
File Manager
Usage
▶︎
Expected Behavior
Backup
Install
Restore
▶︎
Flow Analyzer
Usage
▶︎
Node Manager
Usage
▶︎
Expected Behavior
Status
Disconnect
Connect
Remove
▶︎
Notify
Usage
▶︎
S2S
Usage
▶︎
TLS Toolkit
▶︎
Wildcard Certificates
Potential issues with wildcard certificates
▶︎
Operation Modes
Standalone
Client/Server
▶︎
Using An Existing Intermediate Certificate Authority (CA)
nifi-cert.pem
nifi-key.key
Signing with Externally-signed CA Certificates
Additional Certificate Commands
▶︎
ZooKeeper Migrator
Usage
▶︎
Migrating Between Source and Destination ZooKeepers
ZooKeeper Migration Steps
▶︎
Adding Functionality to Apache NiFi
NiFi Developer Guide Introduction
NiFi Components
▶︎
Processor API
▶︎
Supporting API
FlowFile
ProcessSession
ProcessContext
PropertyDescriptor
Validator
ValidationContext
PropertyValue
Relationship
StateManager
ProcessorInitializationContext
ComponentLog
▶︎
AbstractProcessor API
Processor Initialization
Exposing Processor's Relationships
Exposing Processor Properties
Validating Processor Properties
Responding to Changes in Configuration
Performing the Work
When Processors are Triggered
▶︎
Component Lifecycle
@OnAdded
@OnEnabled
@OnRemoved
@OnScheduled
@OnUnscheduled
@OnStopped
@OnShutdown
▶︎
Component Notification
@OnPrimaryNodeStateChange
Restricted
▶︎
State Manager
Scope
Storing and Retrieving State
Unit Tests
Reporting Processor Activity
▶︎
Documenting a Component
Documenting Properties
Documenting Relationships
Documenting Capability and Keywords
Documenting FlowFile Attribute Interaction
Documenting Related Components
Advanced Documentation
Provenance Events
▶︎
Common Processor Patterns
Data Ingress
Data Egress
Route Based on Content (One-to-One)
Route Based on Content (One-to-Many)
Route Streams Based on Content (One-to-Many)
Route Based on Attributes
Split Content (One-to-Many)
Update Attributes Based on Content
Enrich/Modify Content
▶︎
Error Handling
Exceptions within the Processor
Exceptions within a callback: IOException, RuntimeException
Penalization vs. Yielding
Session Rollback
▶︎
General Design Considerations
Consider the User
Cohesion and Reusability
Naming Conventions
Processor Behavior Annotations
Data Buffering
▶︎
Controller Services
Developing a ControllerService
Interacting with a ControllerService
▶︎
Reporting Tasks
Developing a Reporting Task
▶︎
UI Extensions
Custom Processor UIs
Content Viewers
Command Line Tools
▶︎
Testing
Instantiate TestRunner
Add ControllerServices
Set Property Values
Enqueue FlowFiles
Run the Processor
Validate Output
Mocking External Resources
Additional Testing Capabilities
NiFi Archives (NARs)
Per-Instance ClassLoading
Deprecating a Component
▶︎
How to contribute to Apache NiFi
Technologies
Where to Start?
Supplying a contribution
Contact Us
▶︎
Using Apache NiFi Registry
Introduction
▶︎
Browser Support
Unsupported Browsers
Viewing the UI in Variably Sized Browsers
Terminology
NiFi Registry User Interface
Logging In
▶︎
Manage Flows
▶︎
View a Flow
Sorting & Filtering Flows
Delete a Flow
▶︎
Manage Buckets
Sorting & Filtering Buckets
Create a Bucket
Delete a Bucket
Delete Multiple Buckets
Edit a Bucket Name
Make a Bucket Publicly Visible
Allow Bundles in a Bucket to be Overwritten
▶︎
Bucket Policies
Create a Bucket Policy
Delete a Bucket Policy
▶︎
Manage Users & Groups
Sorting & Filtering Users/Groups
Add a User
Delete a User
Delete Multiple Users
Edit a User Name
▶︎
Special Privileges
Grant Special Privileges to a User
▶︎
Manage Groups
Add an Empty Group
Add User to a Group
Create a New Group with Selected Users
▶︎
Remove a User from a Group
User Window
Group Window
Other Group Level Actions
▶︎
Manage Bundles
Upload Bundle
▶︎
Download Bundle
Bundle Coordinates
Bundle Id
Additional Actions
▶︎
How To: Streams Messaging
▶︎
Creating your First Streams Messaging Cluster in CDP Public Cloud
▶︎
Creating your first Streams Messaging cluster
Meet the prerequisites
Create your cluster
Give users access to your cluster
Next steps
▶︎
Connecting Kafka Clients to CDP Public Cloud Clusters
Connecting Kafka clients to Data Hub provisioned clusters
▶︎
Apache Kafka
▶︎
Configuring Apache Kafka
Operating system requirements
Performance considerations
Quotas
▶︎
JBOD
JBOD setup
JBOD Disk migration
Setting user limits for Kafka
Connecting Kafka clients to Data Hub provisioned clusters
Configuring Kafka ZooKeeper chroot
Rack awareness
▶︎
Securing Apache Kafka
▶︎
Channel encryption
Configure Kafka brokers
Configure Kafka clients
Configure Kafka MirrorMaker
Configure Zookeeper TLS/SSL support for Kafka
▶︎
Authentication
▶︎
TLS/SSL client authentication
Configure Kafka brokers
Configure Kafka clients
Principal name mapping
Kerberos authentication
▶︎
Delegation token based authentication
Enable or disable authentication with delegation tokens
Manage individual delegation tokens
Rotate the master key/secret
▶︎
Client authentication using delegation tokens
Configure clients on a producer or consumer level
Configure clients on an application level
▶︎
Kafka security hardening with Zookeeper ACLs
Restrict access to Kafka metadata in Zookeeper
Unlock Kafka metadata in Zookeeper
▶︎
LDAP authentication
Configure Kafka brokers
Configure Kafka clients
▶︎
PAM authentication
Configure Kafka brokers
Configure Kafka clients
▶︎
Authorization
▶︎
Ranger
Enable authorization in Kafka with Ranger
Configure the resource-based Ranger service used for authorization
▶︎
Governance
Configuring the Atlas hook in Kafka
Inter-broker security
Configuring multiple listeners
▶︎
Tuning Apache Kafka Performance
Handling large messages
▶︎
Cluster sizing
Sizing estimation based on network and disk message throughput
Choosing the number of partitions for a topic
▶︎
Broker Tuning
JVM and garbage collection
Network and I/O threads
ISR management
Log cleaner
▶︎
System Level Broker Tuning
File descriptor limits
Filesystems
Virtual memory handling
Networking parameters
Configure JMX ephemeral ports
Kafka-ZooKeeper performance tuning
▶︎
Managing Apache Kafka
▶︎
Management basics
Broker log management
Record management
Broker garbage log collection and log rotation
Client and broker compatibility across Kafka versions
▶︎
Managing topics across multiple Kafka clusters
Set up MirrorMaker in Cloudera Manager
Settings to avoid data loss
▶︎
Broker migration
Migrate brokers by modifying broker IDs in meta.properties
Use rsync to copy files from one broker to another
▶︎
Disk management
Monitoring
▶︎
Handling disk failures
Disk Replacement
Disk Removal
Reassigning replicas between log directories
Retrieving log directory replica assignment information
▶︎
Metrics
Building Cloudera Manager charts with Kafka metrics
Essential metrics to monitor
▶︎
Command Line Tools
Unsupported command line tools
kafka-topics
kafka-configs
kafka-console-producer
kafka-console-consumer
kafka-consumer-groups
▶︎
kafka-reassign-partitions
Tool usage
Reassignment examples
kafka-log-dirs
zookeeper-security-migration
kafka-delegation-tokens
kafka-*-perf-test
Configuring log levels for command line tools
Understanding the kafka-run-class Bash Script
▶︎
Developing Apache Kafka Applications
Kafka producers
▶︎
Kafka consumers
Subscribing to a topic
Groups and fetching
Protocol between consumer and broker
Rebalancing partitions
Retries
Kafka clients and ZooKeeper
▶︎
Java client
▶︎
Client examples
Simple Java consumer
Simple Java producer
Security examples
▶︎
.NET client
▶︎
Client examples
Simple .NET consumer
Simple .NET producer
Performant .NET producer
Security examples
Kafka Streams
Kafka public APIs
Recommendations for client development
▶︎
Schema Registry
▶︎
Integrating with Schema Registry
▶︎
Integrating with NiFi
Understand the NiFi Record Based Processors and Controller Services
Set up the HortonworksSchemaRegistry Controller Service
Adding and Configuring Record Reader and Writer Controller Services
Using Record-Enabled Processors
▶︎
Integrating with Kafka
Integrate Kafka and Schema Registry using NiFi Processors
Integrate Kafka and Schema Registry
Improve Performance in Schema Registry
▶︎
Using Schema Registry
Adding a new schema
Querying a schema
Evolving a schema
Deleting a schema
Importing Confluent Schema Registry schemas into Cloudera Schema Registry
▶︎
Securing Schema Registry
▶︎
Schema Registry Authorization through Ranger Access Policies
Pre-defined Access Policies for Schema Registry
Add the user or group to a pre-defined access policy
Create a Custom Access Policy
▶︎
Streams Messaging Manager
▶︎
Monitoring Kafka Clusters
Monitoring Kafka clusters
Monitoring Kafka producers
Monitoring Kafka topics
Monitoring Kafka brokers
Monitoring Kafka consumers
▶︎
Managing Alert Policies
Introduction to alert policies in Streams Messaging Manager
Component types and metrics for alert policies
Notifiers
▶︎
Managing alert policies and notifiers in SMM
Creating a notifier
Updating a notifier
Deleting a notifier
Creating an alert policy
Updating an alert policy
Enabling an alert policy
Disabling an alert policy
Deleting an alert policy
▶︎
Managing Kafka Topics
Creating a Kafka topic
Modifying a Kafka topic
Deleting a Kafka topic
▶︎
Monitoring End to End Latency
End to end latency overview
Granularity of metrics for end-to-end latency
Enabling interceptors
Monitoring end to end latency for Kafka topic
End to end latency use case
▶︎
Monitoring Kafka Cluster Replications using Streams Messaging Manager
Introduction to monitoring Kafka cluster replications in SMM
Configuring SMM for monitoring Kafka cluster replications
▶︎
Viewing Kafka cluster replication details
Searching Kafka cluster replications by source
Monitoring Kafka cluster replications by quick ranges
Monitoring status of the clusters to be replicated
▶︎
Monitoring topics to be replicated
Searching by topic name
Monitoring throughput for cluster replication
Monitoring replication latency for cluster replication
Monitoring checkpoint latency for cluster replication
Monitoring replication throughput and latency by values
▶︎
Securing Streams Messaging Manager
Securing Streams Messaging Manager
Verifying the setup
▶︎
Streams Replication Manager
▶︎
Planning for Streams Replication Manager
Streams Replication Manager requirements
Recommended deployment architecture
▶︎
Configuring Streams Replication Manager
Enable high availability
Configuring clusters and replications
Configuring the driver role target clusters
Configuring the service role target cluster
Configuring properties not exposed in Cloudera Manager
Configuring replication specific REST servers
Configuring automatic group offset synchronization
New topic and consumer group discovery
▶︎
Configuration examples
Bidirectional replication example of two active clusters
Cross data center replication example of multiple clusters
▶︎
Using Streams Replication Manager
▶︎
SRM Command Line Tools
▶︎
srm-control
▶︎
Configuring srm-control
Configure srm-control for unsecured environments using Cloudera Manager
Configure srm-control for unsecured environments using environment variables
Configure srm-control for secured environments using Cloudera Manager
Configure srm-control for secured environments using environment variables
Topics and Groups Subcommand
Offsets Subcommand
Monitoring Replication with Streams Messaging Manager
Replicating Data
▶︎
How to Set up Failover and Failback
Configure SRM for Failover and Failback
Migrating Consumer Groups Between Clusters
▶︎
Securing Streams Replication Manager
Security overview
SRM security example for a cluster environment managed by a single Cloudera Manager instance
SRM security example for a cluster environment managed by multiple Cloudera Manager instances
▶︎
Use cases for Streams Replication Manager in CDP Public Cloud
Using SRM in CDP Public Cloud overview
Replicating data from PvC Base to Data Hub with on-prem SRM
Replicating data from PvC Base to Data Hub with cloud SRM
Replicate data between Data Hub clusters with cloud SRM
▶︎
How To: Streaming Analytics
▶︎
Creating your First Streaming Analytics Cluster in CDP Public Cloud
▶︎
Creating your first Streaming Analytics cluster
Meet the prerequisites
Create your cluster
Give users access to your cluster
Next steps
▶︎
Analyzing data with Apache Kafka in CDP Public Cloud
Understand the use case
▶︎
Prepare your environment
Assign resource roles
Create IDBroker mapping
Set workload password
Create your streaming clusters
Set Ranger policies
Retrieve keytab file
Create Atlas entity type definitions
▶︎
Analyzing your data with Kafka
Job monitoring with Flink Dashboard
Metadata governance with Atlas
Data querying with SQL Client
▶︎
Analyzing data with Apache HBase in CDP Public Cloud
▶︎
Analyzing your data with HBase
Job monitoring with Flink Dashboard
Metadata governance with Atlas
▶︎
Analyzing data with Apache Kudu in CDP Public Cloud
▶︎
Analyzing your data with Kudu
Job monitoring with Flink Dashboard
Metadata governance with Atlas
▼
Reference
▶︎
Apache NiFi REST API Reference
Apache NiFi REST API Reference
▶︎
Apache NiFi Record Path Reference
Apache NiFi RecordPath Overview
Structure of a RecordPath
Child Operator
Descendant Operator
Filters
▶︎
Function Usage
Arrays
Maps
Predicates
Functions
▶︎
Standalone Functions
substring
substringAfter
substringAfterLast
substringBefore
substringBeforeLast
replace
replaceRegex
concat
fieldName
toDate
toString
toBytes
coalesce
format
trim
toUpperCase
toLowerCase
base64Encode
base64Decode
hash
padLeft
padRight
uuid5
▶︎
Filter Functions
contains
matchesRegex
startsWith
endsWith
not
isEmpty
isBlank
▶︎
Apache NiFi Expression Language Guide
Apache NiFi Expression Language Overview
▶︎
Structure of a NiFi Expression
Expression Language Hierarchy
▶︎
Expression Language in the Application
Escaping Expression Language
Expression Language Editor
▶︎
Functions
Data Types
Boolean Logic
String Manipulation
Encode/Decode Functions
Searching
Mathematical Operations and Numeric Manipulation
Date Manipulation
Type Coercion
Subjectless Functions
Evaluating Multiple Attributes
▶︎
Apache NiFi Admin Guide
System Requirements
How to install and start NiFi
▶︎
Port Configuration
NiFi
Embedded ZooKeeper
Configuration Best Practices
Recommended Antivirus Exclusions
▶︎
Security Configuration
TLS Generation Toolkit
▶︎
User Authentication
Lightweight Directory Access Protocol (LDAP)
Kerberos
OpenId Connect
SAML
Apache Knox
▶︎
Multi-Tenant Authorization
Authorizer Configuration
▶︎
Authorizers.xml Setup
FileUserGroupProvider
LdapUserGroupProvider
ShellUserGroupProvider
Composite Implementations
FileAccessPolicyProvider
StandardManagedAuthorizer
FileAuthorizer
Initial Admin Identity (New NiFi Instance)
Legacy Authorized Users (NiFi Instance Upgrade)
Cluster Node Identities
▶︎
Configuring Users & Access Policies
Creating Users and Groups
Access Policies
Viewing Policies on Users
Access Policy Configuration Examples
▶︎
Encryption Configuration
▶︎
Key Derivation Functions
NiFi Legacy KDF
OpenSSL PKCS#5 v1.5 EVP_BytesToKey
Bcrypt
Scrypt
PBKDF2
None
Argon2
▶︎
Salt and IV Encoding
NiFi Legacy
OpenSSL PKCS#5 v1.5 EVP_BytesToKey
Bcrypt, Scrypt, PBKDF2, Argon2
Java Cryptography Extension (JCE) Limited Strength Jurisdiction Policies
Allow Insecure Cryptographic Modes
Encrypted Passwords in Flows
Encrypted Passwords in Configuration Files
NiFi Toolkit Administrative Tools
▶︎
Clustering Configuration
Zero-Leader Clustering
Why Cluster?
Terminology
Communication within the Cluster
▶︎
Managing Nodes
Disconnect Nodes
Offload Nodes
Delete Nodes
Decommission Nodes
NiFi CLI Node Commands
Flow Election
Basic Cluster Setup
Troubleshooting
▶︎
State Management
Configuring State Providers
Embedded ZooKeeper Server
ZooKeeper Access Control
▶︎
Securing ZooKeeper with Kerberos
Kerberizing Embedded ZooKeeper Server
Kerberizing NiFi's ZooKeeper Client
Troubleshooting Kerberos Configuration
Securing ZooKeeper with TLS
Embedded ZooKeeper with TLS
ZooKeeper Migrator
Bootstrap Properties
▶︎
Notification Services
Email Notification Service
HTTP Notification Service
Proxy Configuration
▶︎
Kerberos Service
Notes
Analytics Framework
▶︎
System Properties
Upgrade Recommendations
Core Properties
State Management
H2 Settings
FlowFile Repository
Write Ahead FlowFile Repository
Encrypted Write Ahead FlowFile Repository Properties
Volatile FlowFile Repository
RocksDB FlowFile Repository
Swap Management
Content Repository
File System Content Repository Properties
Encrypted File System Content Repository Properties
Volatile Content Repository Properties
Provenance Repository
Write Ahead Provenance Repository Properties
Encrypted Write Ahead Provenance Repository Properties
Persistent Provenance Repository Properties
Volatile Provenance Repository Properties
Status History Repository
Site to Site Properties
▶︎
Site to Site Routing Properties for Reverse Proxies
Site to Site protocol sequence
Reverse Proxy Configurations
Site to Site and Reverse Proxy Examples
Web Properties
Security Properties
Identity Mapping Properties
Cluster Common Properties
Cluster Node Properties
ZooKeeper Properties
Kerberos Properties
Analytics Properties
Referencing Custom Properties via nifi.properties
▶︎
Upgrading NiFi
Preserve Custom Processors
Preserve Modified NARs
Clear Activity and Shutdown Existing NiFi
Install the new NiFi Version
▶︎
Update the Configuration Files for Your New NiFi Installation
Migrating a Flow with Sensitive Properties
Start New NiFi
▶︎
Processor Locations
Available Configuration Options
Installing Custom Processors
Autoloading Custom Processors
▶︎
Apache NiFi Registry Admin Guide
System Requirements
How to install and start NiFi Registry
Recommended Antivirus Exclusions
Security Configuration
▶︎
User Authentication
Lightweight Directory Access Protocol (LDAP)
Kerberos
▶︎
Authorization
Authorizer Configuration
▶︎
Authorizers.xml Setup
StandardManagedAuthorizer
UserGroupProvider
AccessPolicyProvider
Initial Admin Identity (New NiFi Registry Instance)
▶︎
Access Policies
Bucket Policies
Special Privilege Policies
▶︎
Encrypted Passwords in Configuration Files
Encrypt-Config Tool
Sensitive Property Key Migration
Bootstrap Properties
Proxy Configuration
▶︎
Kerberos Service
Notes
▶︎
System Properties
Web Properties
Security Properties
Identity Mapping Properties
Providers Properties
Alias Properties
Database Properties
Extension Directories
Kerberos Properties
▶︎
Metadata Database
H2
Postgres
MySQL
Schema Differences & Limitations
▶︎
Persistence Providers
▶︎
Flow Persistence Providers
FileSystemFlowPersistenceProvider
GitFlowPersistenceProvider
DatabaseFlowPersistenceProvider
Switching from other Flow Persistence Provider
Data model version of serialized Flow snapshots
▶︎
Bundle Persistence Providers
FileSystemBundlePersistenceProvider
S3BundlePersistenceProvider
▶︎
Event Hooks
Shared Event Hook Properties
ScriptEventHookProvider
LoggingEventHookProvider
URL Aliasing
▶︎
Backup & Recovery
Metadata Database
▶︎
Persistence Providers
Flow Persistence
Bundle Persistence
Configuration Files
▼
Apache NiFi Registry REST API Reference
▼
1. Overview
1.1. Version information
1.2. Contact information
1.3. License information
1.4. URI scheme
1.5. Tags
▶︎
2. Security
2.1. Authorization
2.2. BasicAuth
▶︎
3. Resources
▶︎
3.1. Access
3.1.1. Get access status
3.1.2. Performs a logout for other providers that have been issued a JWT.
3.1.3. Redirect/callback URI for processing the result of the OpenId Connect login sequence.
3.1.4. Retrieves a JWT following a successful login sequence using the configured OpenId Connect provider.
3.1.5. Performs a logout in the OpenId Provider.
3.1.6. Initiates a request to authenticate through the configured OpenId Connect provider.
3.1.7. Create token trying all providers
3.1.8. Create token using identity provider
3.1.9. Test identity provider
3.1.10. Get identity provider usage
3.1.11. Create token using kerberos
3.1.12. Create token using basic auth
▶︎
3.2. Bucket Bundles
3.2.1. Get extension bundles by bucket
3.2.2. Create extension bundle version
▶︎
3.3. Bucket Flows
3.3.1. Create flow
3.3.2. Get bucket flows
3.3.3. Get bucket flow
3.3.4. Update bucket flow
3.3.5. Delete bucket flow
3.3.6. Get bucket flow diff
3.3.7. Create flow version
3.3.8. Get bucket flow versions
3.3.9. Get latest bucket flow version content
3.3.10. Get latest bucket flow version metadata
3.3.11. Get bucket flow version
▶︎
3.4. Buckets
3.4.1. Create bucket
3.4.2. Get all buckets
3.4.3. Get bucket fields
3.4.4. Get bucket
3.4.5. Update bucket
3.4.6. Delete bucket
▶︎
3.5. Bundles
3.5.1. Get all bundles
3.5.2. Get all bundle versions
3.5.3. Get bundle
3.5.4. Delete bundle
3.5.5. Get bundle versions
3.5.6. Get bundle version
3.5.7. Delete bundle version
3.5.8. Get bundle version content
3.5.9. Get bundle version extensions
3.5.10. Get bundle version extension
3.5.11. Get bundle version extension docs
3.5.12. Get bundle version extension docs details
▶︎
3.6. Config
3.6.1. Get configration
▶︎
3.7. Extension Repository
3.7.1. Get extension repo buckets
3.7.2. Get extension repo groups
3.7.3. Get extension repo artifacts
3.7.4. Get extension repo versions
3.7.5. Get extension repo version
3.7.6. Get extension repo version content
3.7.7. Get extension repo extensions
3.7.8. Get extension repo extension
3.7.9. Get extension repo extension docs
3.7.10. Get extension repo extension details
3.7.11. Get extension repo version checksum
3.7.12. Get global extension repo version checksum
▶︎
3.8. Extensions
3.8.1. Get all extensions
3.8.2. Get extensions providing service API
3.8.3. Get extension tags
▶︎
3.9. Flows
3.9.1. Get flow fields
3.9.2. Get flow
3.9.3. Get flow versions
3.9.4. Get latest flow version
3.9.5. Get latest flow version metadata
3.9.6. Get flow version
▶︎
3.10. Items
3.10.1. Get all items
3.10.2. Get item fields
3.10.3. Get bucket items
▶︎
3.11. Policies
3.11.1. Create access policy
3.11.2. Get all access policies
3.11.3. Get available resources
3.11.4. Get access policy for resource
3.11.5. Get access policy
3.11.6. Update access policy
3.11.7. Delete access policy
▶︎
3.12. Tenants
3.12.1. Create user group
3.12.2. Get user groups
3.12.3. Get user group
3.12.4. Update user group
3.12.5. Delete user group
3.12.6. Create user
3.12.7. Get all users
3.12.8. Get user
3.12.9. Update user
3.12.10. Delete user
▶︎
4. Definitions
4.1. AccessPolicy
4.2. AccessPolicySummary
4.3. AllowableValue
4.4. Attribute
4.5. BatchSize
4.6. Bucket
4.7. BucketItem
4.8. BuildInfo
4.9. Bundle
4.10. BundleInfo
4.11. BundleVersion
4.12. BundleVersionDependency
4.13. BundleVersionMetadata
4.14. ComponentDifference
4.15. ComponentDifferenceGroup
4.16. ConnectableComponent
4.17. ControllerServiceAPI
4.18. ControllerServiceDefinition
4.19. CurrentUser
4.20. DeprecationNotice
4.21. DynamicProperty
4.22. DynamicRelationship
4.23. Extension
4.24. ExtensionBundle
4.25. ExtensionFilterParams
4.26. ExtensionMetadata
4.27. ExtensionMetadataContainer
4.28. ExtensionRepoArtifact
4.29. ExtensionRepoBucket
4.30. ExtensionRepoGroup
4.31. ExtensionRepoVersion
4.32. ExtensionRepoVersionSummary
4.33. ExternalControllerServiceReference
4.34. Fields
4.35. JaxbLink
4.36. Permissions
4.37. Position
4.38. Property
4.39. ProvidedServiceAPI
4.40. RegistryConfiguration
4.41. Relationship
4.42. Resource
4.43. ResourcePermissions
4.44. Restricted
4.45. Restriction
4.46. RevisionInfo
4.47. Stateful
4.48. SystemResourceConsideration
4.49. TagCount
4.50. Tenant
4.51. User
4.52. UserGroup
4.53. VersionedConnection
4.54. VersionedControllerService
4.55. VersionedFlow
4.56. VersionedFlowCoordinates
4.57. VersionedFlowDifference
4.58. VersionedFlowSnapshot
4.59. VersionedFlowSnapshotMetadata
4.60. VersionedFunnel
4.61. VersionedLabel
4.62. VersionedParameter
4.63. VersionedParameterContext
4.64. VersionedPort
4.65. VersionedProcessGroup
4.66. VersionedProcessor
4.67. VersionedPropertyDescriptor
4.68. VersionedRemoteGroupPort
4.69. VersionedRemoteProcessGroup
Streams Messaging Manager REST API Reference
▶︎
Streams Replication Manager Reference
srm-control Options Reference
Configuration Properties Reference for Properties not Available in Cloudera Manager
Streams Replication Manager REST API Reference
▶︎
Learning & Training
▶︎
Getting Started with Streams Messaging Clusters on CDP Public Cloud
Introducing streams messaging cluster on CDP Public Cloud
Meet the prerequisites to create streams messaging cluster
Creating Machine User
Granting Machine User access to environment
Creating Kafka topic
▶︎
Create Ranger policies for Machine User account
Create topic policy
Create consumer group policy
▶︎
Produce data to Kafka topic
Setting workload password
Connecting to Kafka host
Configuring LDAP authentication
Producing data to Kafka topic
Consuming data from Kafka topic
▶︎
Use Kerberos authentication
Kerberos authentication using the ticket cache
Kerberos authentication using a keytab
Monitoring Kafka activity in Streams Messaging Manager
▶︎
Use Schema Registry
▶︎
Gather configuration information
Finding list of brokers
Finding Schema Registry endpoint
Creating TLS truststore
Defining Schema Registry access policies
Producing data in Avro format
Checking schema registration
Checking producer activity
Consuming data from Kafka topics using stored schemas
▶︎
Monitor end-to-end latency
Setting up authorization policies
Enabling end-to-end latency monitoring
▶︎
Evolve your schema
Reconfiguring the Kafka consumer
Reconfiguring the Kafka producer
What to do next
▶︎
Getting Started with Apache NiFi Registry
Who is This Guide For?
Terminology Used in This Guide
Downloading and Installing NiFi Registry
▶︎
Starting NiFi Registry
For Linux/Unix/Mac OS X users
Installing as a Service
▶︎
I Started NiFi Registry. Now What?
Create a Bucket
Connect NiFi to the Registry
Start Version Control on a Process Group
Save Changes to a Versioned Flow
Import a Versioned Flow
Where To Go For More Information
.NET client
1. Overview
1.1. Version information
1.2. Contact information
1.3. License information
1.4. URI scheme
1.5. Tags
2. Security
2.1. Authorization
2.2. BasicAuth
3. Resources
3.1. Access
3.1.1. Get access status
3.1.10. Get identity provider usage
3.1.11. Create token using kerberos
3.1.12. Create token using basic auth
3.1.2. Performs a logout for other providers that have been issued a JWT.
3.1.3. Redirect/callback URI for processing the result of the OpenId Connect login sequence.
3.1.4. Retrieves a JWT following a successful login sequence using the configured OpenId Connect provider.
3.1.5. Performs a logout in the OpenId Provider.
3.1.6. Initiates a request to authenticate through the configured OpenId Connect provider.
3.1.7. Create token trying all providers
3.1.8. Create token using identity provider
3.1.9. Test identity provider
3.10. Items
3.10.1. Get all items
3.10.2. Get item fields
3.10.3. Get bucket items
3.11. Policies
3.11.1. Create access policy
3.11.2. Get all access policies
3.11.3. Get available resources
3.11.4. Get access policy for resource
3.11.5. Get access policy
3.11.6. Update access policy
3.11.7. Delete access policy
3.12. Tenants
3.12.1. Create user group
3.12.10. Delete user
3.12.2. Get user groups
3.12.3. Get user group
3.12.4. Update user group
3.12.5. Delete user group
3.12.6. Create user
3.12.7. Get all users
3.12.8. Get user
3.12.9. Update user
3.2. Bucket Bundles
3.2.1. Get extension bundles by bucket
3.2.2. Create extension bundle version
3.3. Bucket Flows
3.3.1. Create flow
3.3.10. Get latest bucket flow version metadata
3.3.11. Get bucket flow version
3.3.2. Get bucket flows
3.3.3. Get bucket flow
3.3.4. Update bucket flow
3.3.5. Delete bucket flow
3.3.6. Get bucket flow diff
3.3.7. Create flow version
3.3.8. Get bucket flow versions
3.3.9. Get latest bucket flow version content
3.4. Buckets
3.4.1. Create bucket
3.4.2. Get all buckets
3.4.3. Get bucket fields
3.4.4. Get bucket
3.4.5. Update bucket
3.4.6. Delete bucket
3.5. Bundles
3.5.1. Get all bundles
3.5.10. Get bundle version extension
3.5.11. Get bundle version extension docs
3.5.12. Get bundle version extension docs details
3.5.2. Get all bundle versions
3.5.3. Get bundle
3.5.4. Delete bundle
3.5.5. Get bundle versions
3.5.6. Get bundle version
3.5.7. Delete bundle version
3.5.8. Get bundle version content
3.5.9. Get bundle version extensions
3.6. Config
3.6.1. Get configration
3.7. Extension Repository
3.7.1. Get extension repo buckets
3.7.10. Get extension repo extension details
3.7.11. Get extension repo version checksum
3.7.12. Get global extension repo version checksum
3.7.2. Get extension repo groups
3.7.3. Get extension repo artifacts
3.7.4. Get extension repo versions
3.7.5. Get extension repo version
3.7.6. Get extension repo version content
3.7.7. Get extension repo extensions
3.7.8. Get extension repo extension
3.7.9. Get extension repo extension docs
3.8. Extensions
3.8.1. Get all extensions
3.8.2. Get extensions providing service API
3.8.3. Get extension tags
3.9. Flows
3.9.1. Get flow fields
3.9.2. Get flow
3.9.3. Get flow versions
3.9.4. Get latest flow version
3.9.5. Get latest flow version metadata
3.9.6. Get flow version
4. Definitions
4.1. AccessPolicy
4.10. BundleInfo
4.11. BundleVersion
4.12. BundleVersionDependency
4.13. BundleVersionMetadata
4.14. ComponentDifference
4.15. ComponentDifferenceGroup
4.16. ConnectableComponent
4.17. ControllerServiceAPI
4.18. ControllerServiceDefinition
4.19. CurrentUser
4.2. AccessPolicySummary
4.20. DeprecationNotice
4.21. DynamicProperty
4.22. DynamicRelationship
4.23. Extension
4.24. ExtensionBundle
4.25. ExtensionFilterParams
4.26. ExtensionMetadata
4.27. ExtensionMetadataContainer
4.28. ExtensionRepoArtifact
4.29. ExtensionRepoBucket
4.3. AllowableValue
4.30. ExtensionRepoGroup
4.31. ExtensionRepoVersion
4.32. ExtensionRepoVersionSummary
4.33. ExternalControllerServiceReference
4.34. Fields
4.35. JaxbLink
4.36. Permissions
4.37. Position
4.38. Property
4.39. ProvidedServiceAPI
4.4. Attribute
4.40. RegistryConfiguration
4.41. Relationship
4.42. Resource
4.43. ResourcePermissions
4.44. Restricted
4.45. Restriction
4.46. RevisionInfo
4.47. Stateful
4.48. SystemResourceConsideration
4.49. TagCount
4.5. BatchSize
4.50. Tenant
4.51. User
4.52. UserGroup
4.53. VersionedConnection
4.54. VersionedControllerService
4.55. VersionedFlow
4.56. VersionedFlowCoordinates
4.57. VersionedFlowDifference
4.58. VersionedFlowSnapshot
4.59. VersionedFlowSnapshotMetadata
4.6. Bucket
4.60. VersionedFunnel
4.61. VersionedLabel
4.62. VersionedParameter
4.63. VersionedParameterContext
4.64. VersionedPort
4.65. VersionedProcessGroup
4.66. VersionedProcessor
4.67. VersionedPropertyDescriptor
4.68. VersionedRemoteGroupPort
4.69. VersionedRemoteProcessGroup
4.7. BucketItem
4.8. BuildInfo
4.9. Bundle
@OnAdded
@OnEnabled
@OnPrimaryNodeStateChange
@OnRemoved
@OnScheduled
@OnShutdown
@OnStopped
@OnUnscheduled
AbstractProcessor API
Access Policies
Access Policies
Access Policy Configuration Examples
Accessing Parameters
Accessing the UI with Multi-Tenant Authorization
AccessPolicyProvider
Active / Active Architecture
Active / Stand-by Architecture
Add a User
Add an Empty Group
Add ControllerServices
Add Ranger policies
Add Ranger policies
Add the user or group to a pre-defined access policy
Add the user to predefined Ranger access policies
Add User to a Group
Adding a new schema
Adding a Parameter to a Parameter Context
Adding and Configuring Record Reader and Writer Controller Services
Adding Commands
Adding Components to the Canvas
Adding Controller Services for Dataflows
Adding Controller Services for Reporting Tasks
Adding Functionality to Apache NiFi
Adding Snowflake CA certificates to NiFi truststore
Adding Snowflake CA certificates to NiFi truststore
Additional Actions
Additional Certificate Commands
Additional Help
Additional Testing Capabilities
Advanced Documentation
Aggregation for Analytics
Alias Properties
Align Horizontally
Align Vertically
Allow Bundles in a Bucket to be Overwritten
Allow Insecure Cryptographic Modes
Analytics Framework
Analytics Properties
Analyzing data with Apache HBase in CDP Public Cloud
Analyzing data with Apache Kafka in CDP Public Cloud
Analyzing data with Apache Kudu in CDP Public Cloud
Analyzing your data with HBase
Analyzing your data with Kafka
Analyzing your data with Kudu
Anatomy of a Process Group
Anatomy of a Processor
Anatomy of a Remote Process Group
Apache Flink Overview
Apache Kafka
Apache Kafka Overview
Apache Knox
Apache NiFi
Apache NiFi Admin Guide
Apache NiFi Expression Language Guide
Apache NiFi Expression Language Overview
Apache NiFi Overview
Apache NiFi Record Path Reference
Apache NiFi RecordPath Overview
Apache NiFi Registry Admin Guide
Apache NiFi Registry REST API Reference
Apache NiFi REST API Reference
Apache NiFi REST API Reference
Apache Patch Information in Cloudera DataFlow for Data Hub 7.2.9
Appendix - Schema example
Argon2
Arrays
Assign resource roles
Assign the EnvironmentUser role
Assigning a Parameter Context to a Process Group
Assigning administrator level permissions
Assigning selective permissions to user
Authentication
Authorization
Authorization
Authorization example
Authorization workflow
Authorizer Configuration
Authorizer Configuration
Authorizers.xml Setup
Authorizers.xml Setup
Authorizing Flow Management Cluster Access in CDP Public Cloud
Autoloading Custom Processors
Available Configuration Options
Back-Referencing
Backup
Backup & Recovery
Backwards Compatibility
base64Decode
base64Encode
Basic Cluster Setup
Basics
Bcrypt
Bcrypt, Scrypt, PBKDF2, Argon2
Before you begin
Before you begin
Bending Connections
Bi-directional Replication Flows
Bidirectional replication example of two active clusters
Boolean Logic
Bootstrap Properties
Bootstrap Properties
Bootstrap.conf
Broker garbage log collection and log rotation
Broker log management
Broker migration
Broker Tuning
Brokers
Browser Support
Browser Support
Bucket Policies
Bucket Policies
Build the data flow
Build the data flow
Build the data flow
Build the data flow
Build the data flow
Build the data flow
Build the data flow
Build the data flow
Building a DataFlow
Building Cloudera Manager charts with Kafka metrics
Building your dataflow
Building your dataflow
Bundle Coordinates
Bundle Id
Bundle Persistence
Bundle Persistence Providers
CDF for Data Hub
Change Version
Changing Component Versions
Changing Configuration and Context Menu Options
Channel encryption
Checking producer activity
Checking schema registration
Child Operator
Choosing the number of partitions for a topic
Clear Activity and Shutdown Existing NiFi
Client and broker compatibility across Kafka versions
Client authentication using delegation tokens
Client examples
Client examples
Client/Server
Cluster Common Properties
Cluster Migration Architectures
Cluster Node Identities
Cluster Node Properties
Cluster sizing
Clustering Configuration
coalesce
Cohesion and Reusability
Command and Control of the DataFlow
Command Line Tools
Command Line Tools
Comments Tab
Commit Local Changes
Common Processor Patterns
Communication within the Cluster
Compatibility Policies
Component Alignment
Component Lifecycle
Component Linking
Component Notification
Component Support in Cloudera DataFlow for Data Hub 7.2.9
Component types and metrics for alert policies
Component Versions
ComponentLog
Composite Implementations
concat
Configuration Best Practices
Configuration examples
Configuration Files
Configuration Properties Reference for Properties not Available in Cloudera Manager
Configure clients on a producer or consumer level
Configure clients on an application level
Configure JMX ephemeral ports
Configure Kafka brokers
Configure Kafka brokers
Configure Kafka brokers
Configure Kafka brokers
Configure Kafka clients
Configure Kafka clients
Configure Kafka clients
Configure Kafka clients
Configure Kafka MirrorMaker
Configure Ranger policies for site-to-site communication
Configure Site-to-Site client NiFi instance
Configure Site-to-Site Server NiFi Instance
Configure SRM for Failover and Failback
Configure srm-control for secured environments using Cloudera Manager
Configure srm-control for secured environments using environment variables
Configure srm-control for unsecured environments using Cloudera Manager
Configure srm-control for unsecured environments using environment variables
Configure the Controller Service
Configure the controller services
Configure the HBase client service
Configure the processor for merging records
Configure the processor for merging records
Configure the processor for merging records
Configure the processor for your data source
Configure the processor for your data source
Configure the processor for your data source
Configure the processor for your data source
Configure the processor for your data source
Configure the processor for your data source
Configure the processor for your data source
Configure the processor for your data source
Configure the processor for your data target
Configure the processor for your data target
Configure the processor for your data target
Configure the processor for your data target
Configure the processor for your data target
Configure the processor for your data target
Configure the processor for your data target
Configure the processor for your data target
Configure the resource-based Ranger service used for authorization
Configure the service account
Configure your source processor
Configure your truststores
Configure Zookeeper TLS/SSL support for Kafka
Configuring a Process Group
Configuring a Processor
Configuring Apache Kafka
Configuring automatic group offset synchronization
Configuring clusters and replications
Configuring Kafka ZooKeeper chroot
Configuring LDAP authentication
Configuring log levels for command line tools
Configuring multiple listeners
Configuring properties not exposed in Cloudera Manager
Configuring replication specific REST servers
Configuring SMM for monitoring Kafka cluster replications
Configuring srm-control
Configuring State Providers
Configuring Streams Replication Manager
Configuring the Atlas hook in Kafka
Configuring the driver role target clusters
Configuring the service role target cluster
Configuring Users & Access Policies
Configuring your Controller Services
Configuring your source processor
Configuring your target processor
Configuring your target processor
Confirming your dataflow success
Confirming your dataflow success
Connect
Connect NiFi to the Registry
Connect workers
Connecting Components
Connecting Kafka Clients to CDP Public Cloud Clusters
Connecting Kafka clients to Data Hub provisioned clusters
Connecting Kafka clients to Data Hub provisioned clusters
Connecting to a NiFi Registry
Connecting to Kafka host
Connectors
Consider the User
Consuming data from Kafka topic
Consuming data from Kafka topics using stored schemas
Contact Us
contains
Content Repository
Content Viewers
Controller Services
Controller Services
Controller Services
Core Properties
Create a Bucket
Create a Bucket
Create a Bucket Policy
Create a custom access policy
Create a Custom Access Policy
Create a New Group with Selected Users
Create Atlas entity type definitions
Create consumer group policy
Create controller services for your data flow
Create controller services for your data flow
Create controller services for your data flow
Create controller services for your data flow
Create controller services for your data flow
Create IDBroker mapping
Create IDBroker mapping
Create IDBroker mapping
Create IDBroker mapping
Create IDBroker mapping
Create Ranger policies for Machine User account
Create Solr target collection
Create the HBase target table
Create the Hive target table
Create the Kudu target table
Create topic policy
Create your cluster
Create your cluster
Create your cluster
Create your streaming clusters
Creating a Kafka topic
Creating a notifier
Creating a Template
Creating an alert policy
Creating checkpoints and savepoints in Flink
Creating Controller Services for your dataflow
Creating Kafka topic
Creating Machine User
Creating TLS truststore
Creating Users and Groups
Creating your first Flow Management cluster
Creating your First Flow Management Cluster in CDP Public Cloud
Creating your first Streaming Analytics cluster
Creating your First Streaming Analytics Cluster in CDP Public Cloud
Creating your first Streams Messaging cluster
Creating your First Streams Messaging Cluster in CDP Public Cloud
Cross Data Center Replication
Cross data center replication example of multiple clusters
Custom Processor UIs
Data Buffering
Data Egress
Data Hub cluster definitions
Data Ingress
Data model version of serialized Flow snapshots
Data Provenance
Data querying with SQL Client
Data Types
Database Properties
DatabaseFlowPersistenceProvider
Date Manipulation
Decommission Nodes
Define your CDP Private Cloud Base dataflow
Define your CDP Public Cloud dataflow
Defining Schema Registry access policies
Delegation token based authentication
Delete a Bucket
Delete a Bucket Policy
Delete a Flow
Delete a User
Delete Multiple Buckets
Delete Multiple Users
Delete Nodes
Deleting a Kafka topic
Deleting a notifier
Deleting a schema
Deleting an alert policy
Deployment scenarios
Deployment scenarios
Deprecating a Component
Descendant Operator
Details of an Event
Details Tab
Developing a ControllerService
Developing a Reporting Task
Developing Apache Kafka Applications
Disabling an alert policy
Disconnect
Disconnect Nodes
Disk management
Disk Removal
Disk Replacement
Documenting a Component
Documenting Capability and Keywords
Documenting FlowFile Attribute Interaction
Documenting Properties
Documenting Related Components
Documenting Relationships
Download Bundle
Downloading and Installing NiFi Registry
Downloading the Snowflake JDBC driver jar file
Driver inter-node coordination
Edit a Bucket Name
Edit a User Name
Email Notification Service
Embedded ZooKeeper
Embedded ZooKeeper Server
Embedded ZooKeeper with TLS
Enable authorization in Kafka with Ranger
Enable high availability
Enable or disable authentication with delegation tokens
Enabling an alert policy
Enabling end-to-end latency monitoring
Enabling interceptors
Enabling/Disabling a Component
Enabling/Disabling Controller Services
Encode/Decode Functions
Encrypt-Config Tool
Encrypt-Config Tool
Encrypted Content Repository
Encrypted File System Content Repository Properties
Encrypted FlowFile Repository
Encrypted Passwords in Configuration Files
Encrypted Passwords in Configuration Files
Encrypted Passwords in Flows
Encrypted Provenance Considerations
Encrypted Provenance Repository
Encrypted Write Ahead FlowFile Repository Properties
Encrypted Write Ahead Provenance Repository Properties
Encryption Configuration
End to end latency overview
End to end latency use case
endsWith
Enqueue FlowFiles
Enrich/Modify Content
Error Handling
Escaping Expression Language
Essential metrics to monitor
Evaluating Multiple Attributes
Event Hooks
Event-driven applications with Flink
Evolve your schema
Evolving a schema
Example - Secure NiFi Registry with Proxied-Entity
Example - Secure NiFi Registry without Proxied-Entity
Example Dataflow
Examples
Examples of Interacting with Schema Registry
Exceptions within a callback: IOException, RuntimeException
Exceptions within the Processor
Expanding an Event
Expected Behavior
Expected Behavior
Experimental Warning
Exporting a Template
Exposing Processor Properties
Exposing Processor's Relationships
Expression Language Editor
Expression Language Hierarchy
Expression Language in the Application
Extension Directories
Fan-in and Fan-out Replication Flows
fieldName
File descriptor limits
File Manager
File System Content Repository Properties
FileAccessPolicyProvider
FileAuthorizer
FileBasedKeyProvider
FileBasedKeyProvider
FileSystemBundlePersistenceProvider
FileSystemFlowPersistenceProvider
Filesystems
FileUserGroupProvider
Filter Functions
Filters
Filters
Find Parents
Finding list of brokers
Finding Schema Registry endpoint
Fixed Issues in Cloudera DataFlow for Data Hub 7.2.9
Fixed Issues in Flow Management
Fixed Issues in Streaming Analytics
Fixed Issues in Streams Messaging
Flink Streaming Applications
Flow Analyzer
Flow Election
Flow Management cluster definitions
Flow Management cluster layout
Flow Persistence
Flow Persistence Providers
FlowFile
FlowFile Repository
For Linux/Unix/Mac OS X users
format
Function Usage
Functions
Functions
Gather configuration information
General Design Considerations
General Tab
Getting Started with Apache NiFi Registry
Getting Started with Streams Messaging Clusters on CDP Public Cloud
GitFlowPersistenceProvider
Give users access to your cluster
Give users access to your cluster
Give users access to your cluster
Governance
Grant Special Privileges to a User
Granting Machine User access to environment
Granularity of metrics for end-to-end latency
Group Window
Groups and fetching
H2
H2 Settings
Handling disk failures
Handling large messages
Handling state in Flink
hash
High Level Overview of Key NiFi Features
Highly Available Kafka Architectures
Historical Statistics of a Component
How does it work?
How does it work?
How does it work?
How to contribute to Apache NiFi
How to install and start NiFi
How to install and start NiFi Registry
How to Set up Failover and Failback
HTTP Notification Service
I Started NiFi Registry. Now What?
Identity Mapping Properties
Identity Mapping Properties
Import a Versioned Flow
Import a Versioned Flow
Importing a Template
Importing Confluent Schema Registry schemas into Cloudera Schema Registry
Improve Performance in Schema Registry
Individual Port Transmission
Ingesting data into Amazon S3
Ingesting Data into Amazon S3 Buckets
Ingesting Data into Apache HBase in CDP Cloud
Ingesting Data into Apache HBase in CDP Public Cloud
Ingesting Data into Apache Hive in CDP Public Cloud
Ingesting Data into Apache Hive in CDP Public Cloud
Ingesting data into Apache Kafka
Ingesting Data into Apache Kafka in CDP Public Cloud
Ingesting Data into Apache Kudu in CDP Public Cloud
Ingesting Data into Apache Kudu in CDP Public Cloud
Ingesting data into Apache Solr
Ingesting Data into Apache Solr in CDP Public Cloud
Ingesting Data into Azure Data Lake Storage
Ingesting data into Azure Data Lake Storage
Ingesting Data into CDP Public Cloud
Ingesting Data into Google Cloud Storage
Ingesting data into Google Cloud Storage
Initial Admin Identity (New NiFi Instance)
Initial Admin Identity (New NiFi Registry Instance)
Install
Install the new NiFi Version
Installing as a Service
Installing Custom Processors
Instantiate TestRunner
Instantiating a Template
Integrate Kafka and Schema Registry
Integrate Kafka and Schema Registry using NiFi Processors
Integrating with Kafka
Integrating with NiFi
Integrating with Schema Registry
Inter-broker security
Interacting with a ControllerService
Interactive Usage
Introducing streams messaging cluster on CDP Public Cloud
Introduction
Introduction
Introduction to alert policies in Streams Messaging Manager
Introduction to monitoring Kafka cluster replications in SMM
Introduction to Streams Messaging Manager
isBlank
isEmpty
ISR management
Java client
Java Cryptography Extension (JCE) Limited Strength Jurisdiction Policies
JBOD
JBOD Disk migration
JBOD setup
Job monitoring with Flink Dashboard
Job monitoring with Flink Dashboard
Job monitoring with Flink Dashboard
JVM and garbage collection
Kafka Architecture
Kafka brokers and Zookeeper
Kafka clients and ZooKeeper
Kafka consumers
Kafka FAQ
Kafka Introduction
Kafka producers
Kafka public APIs
Kafka security hardening with Zookeeper ACLs
Kafka Streams
kafka-*-perf-test
kafka-configs
kafka-console-consumer
kafka-console-producer
kafka-consumer-groups
kafka-delegation-tokens
kafka-log-dirs
kafka-reassign-partitions
kafka-topics
Kafka-ZooKeeper performance tuning
Kerberizing Embedded ZooKeeper Server
Kerberizing NiFi's ZooKeeper Client
Kerberos
Kerberos
Kerberos authentication
Kerberos authentication using a keytab
Kerberos authentication using the ticket cache
Kerberos Properties
Kerberos Properties
Kerberos Service
Kerberos Service
Key Derivation Functions
Key Features
Key Rotation
Key Rotation
Keywords
Known Issues In Cloudera DataFlow for Data Hub 7.2.9
Known Issues in Flow Management
Known Issues in Streaming Analytics
Known Issues in Streams Messaging
LDAP authentication
LdapUserGroupProvider
Leader positions and in-sync replicas
Legacy Authorized Users (NiFi Instance Upgrade)
Lightweight Directory Access Protocol (LDAP)
Lightweight Directory Access Protocol (LDAP)
Log cleaner
Logging In
Logging In
LoggingEventHookProvider
Logs and log segments
Main Use Cases
Make a Bucket Publicly Visible
Manage Buckets
Manage Bundles
Manage Flows
Manage Groups
Manage individual delegation tokens
Manage Users & Groups
Management basics
Managing Alert Policies
Managing alert policies and notifiers in SMM
Managing Apache Kafka
Managing Kafka Topics
Managing Local Changes
Managing Nodes
Managing Templates
Managing topics across multiple Kafka clusters
Maps
matchesRegex
Mathematical Operations and Numeric Manipulation
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites to create streams messaging cluster
Metadata Database
Metadata Database
Metadata governance with Atlas
Metadata governance with Atlas
Metadata governance with Atlas
Metrics
Migrate brokers by modifying broker IDs in meta.properties
Migrating a Flow with Sensitive Properties
Migrating Between Source and Destination ZooKeepers
Migrating Consumer Groups Between Clusters
Mocking External Resources
Modifying a Kafka topic
Monitor end-to-end latency
Monitoring
Monitoring checkpoint latency for cluster replication
Monitoring End to End Latency
Monitoring end to end latency for Kafka topic
Monitoring Kafka activity in Streams Messaging Manager
Monitoring Kafka brokers
Monitoring Kafka cluster replications by quick ranges
Monitoring Kafka Cluster Replications using Streams Messaging Manager
Monitoring Kafka Clusters
Monitoring Kafka clusters
Monitoring Kafka consumers
Monitoring Kafka producers
Monitoring Kafka topics
Monitoring of DataFlow
Monitoring replication latency for cluster replication
Monitoring replication throughput and latency by values
Monitoring Replication with Streams Messaging Manager
Monitoring status of the clusters to be replicated
Monitoring throughput for cluster replication
Monitoring topics to be replicated
Monitoring your data flow
Monitoring your data flow
Monitoring your data flow
Monitoring your data flow
Monitoring your data flow
Moving data from CDP Private Cloud Base to Public Cloud with NiFi site-to-site
Moving Data in and Out of Snowflake
Moving data out of Snowflake
Moving Data using NiFi Site-to-Site
Multi-Tenant Authorization
MySQL
Naming Conventions
Navigating within a DataFlow
Nested Versioned Flows
Network and I/O threads
Networking parameters
New topic and consumer group discovery
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
NiFi
NiFi
NiFi
NiFi Architecture
NiFi Archives (NARs)
NiFi CLI
NiFi CLI Node Commands
NiFi Components
NiFi Developer Guide Introduction
NiFi Legacy
NiFi Legacy KDF
NiFi patches
NiFi Registry
NiFi Registry
NiFi Registry patches
NiFi Registry User Interface
NiFi Toolkit Administrative Tools
NiFi User Interface
nifi-cert.pem
nifi-key.key
Node Manager
None
not
Notes
Notes
Notification Services
Notifiers
Notify
Obtain HBase connection details
Obtain Hive connection details
Offload Nodes
Offsets Subcommand
Older Existing NiFi Version
On-premise to Cloud and Kafka Version Upgrade
OpenId Connect
OpenSSL PKCS#5 v1.5 EVP_BytesToKey
OpenSSL PKCS#5 v1.5 EVP_BytesToKey
Operating system requirements
Operation Modes
Other Group Level Actions
Other Management Features
Output
Overview
Overview
padLeft
padRight
PAM authentication
Parameter Contexts
Parameters
Parameters and Expression Language
Parameters in Versioned Flows
Partitions
PBKDF2
Penalization vs. Yielding
Per-Instance ClassLoading
Performance considerations
Performance Expectations and Characteristics of NiFi
Performant .NET producer
Performing the Work
Persistence Providers
Persistence Providers
Persistent Provenance Repository Properties
Planning for Streams Replication Manager
Planning your Flow Management deployment
Planning your Streaming Analytics deployment
Planning your Streams Messaging deployment
Port Configuration
Postgres
Potential Issues
Potential Issues
Potential Issues
Potential issues with wildcard certificates
Pre-defined Access Policies for Schema Registry
Predefined Ranger Access Policies for Apache NiFi
Predefined Ranger Access Policies for Apache NiFi Registry
Predicates
Prepare your clusters
Prepare your environment
Prerequisites for Running in a Secure Environment
Preserve Custom Processors
Preserve Modified NARs
Principal name mapping
ProcessContext
Processor API
Processor Behavior Annotations
Processor Initialization
Processor Locations
Processor Validation
ProcessorInitializationContext
ProcessSession
Produce data to Kafka topic
Producing data in Avro format
Producing data to Kafka topic
Properties Tab
Property/Argument Handling
PropertyDescriptor
PropertyValue
Protocol between consumer and broker
Provenance Events
Provenance Events
Provenance Repository
Providers Properties
Proxy Configuration
Proxy Configuration
Pushing data into Snowflake
Pushing data to and moving data from Snowflake using Apache NiFi
Querying a schema
Queue Interaction
Quotas
Rack awareness
Ranger
Reassigning replicas between log directories
Reassignment examples
Rebalancing partitions
Recommendations for client development
Recommended Antivirus Exclusions
Recommended Antivirus Exclusions
Recommended deployment architecture
Reconfiguring the Kafka consumer
Reconfiguring the Kafka producer
Record management
Record order and assignment
Records
Referencing Custom Properties via nifi.properties
Referencing Custom Properties via nifi.properties
Referencing Parameters
Relationship
Release Notes
Remote Process Group Transmission
Remote Topics
Remove
Remove a User from a Group
Removing a Template
replace
replaceRegex
Replaying a FlowFile
Replicate data between Data Hub clusters with cloud SRM
Replicating Data
Replicating data from PvC Base to Data Hub with cloud SRM
Replicating data from PvC Base to Data Hub with on-prem SRM
Replication Flows Overview
Reporting Processor Activity
Reporting Tasks
Reporting Tasks
Responding to Changes in Configuration
Restore
Restrict access to Kafka metadata in Zookeeper
Restricted
Restricted Components in Versioned Flows
Restricted Controller Service Created in Process Group
Restricted Controller Service Created in Root Process Group
Retries
Retrieve keytab file
Retrieving log directory replica assignment information
Reverse Proxy Configurations
Revert Local Changes
RocksDB FlowFile Repository
Rotate the master key/secret
Route Based on Attributes
Route Based on Content (One-to-Many)
Route Based on Content (One-to-One)
Route Streams Based on Content (One-to-Many)
Run the Processor
S2S
S3BundlePersistenceProvider
Salt and IV Encoding
SAML
Save Changes to a Versioned Flow
Scheduling Tab
Schema Differences & Limitations
Schema Entities
Schema Registry
Schema Registry Authorization through Ranger Access Policies
Schema Registry Component Architecture
Schema Registry Concepts
Schema Registry Overview
Schema Registry Overview
Schema Registry Use Cases
Scope
ScriptEventHookProvider
Scrypt
Search Components in DataFlow
Searching
Searching by topic name
Searching for Events
Searching Kafka cluster replications by source
Securing Apache Kafka
Securing Schema Registry
Securing Streams Messaging Manager
Securing Streams Messaging Manager
Securing Streams Replication Manager
Securing ZooKeeper with Kerberos
Securing ZooKeeper with TLS
Security Configuration
Security Configuration
Security Configuration
Security examples
Security examples
Security for Flow Management Clusters and Users in CDP Public Cloud
Security overview
Security Properties
Security Properties
Sensitive Property Key Migration
Sensitive Property Key Migration
Session Rollback
Set Property Values
Set Ranger policies
Set up AWS for your ingest data flow
Set up MirrorMaker in Cloudera Manager
Set up the HortonworksSchemaRegistry Controller Service
Set up your network configuration
Set workload password
Setting up authorization policies
Setting user limits for Kafka
Setting workload password
Settings
Settings Tab
Settings to avoid data loss
Shared Event Hook Properties
ShellUserGroupProvider
Show Local Changes
Signing with Externally-signed CA Certificates
Simple .NET consumer
Simple .NET producer
Simple Java consumer
Simple Java producer
Site to Site and Reverse Proxy Examples
Site to Site Properties
Site to Site protocol sequence
Site to Site Routing Properties for Reverse Proxies
Site-to-Site
Sizing estimation based on network and disk message throughput
Sophisticated windowing in Flink
Sorting & Filtering Buckets
Sorting & Filtering Flows
Sorting & Filtering Users/Groups
Sorting and Filtering Components
Special Privilege Policies
Special Privileges
Split Content (One-to-Many)
SRM Command Line Tools
SRM security example for a cluster environment managed by a single Cloudera Manager instance
SRM security example for a cluster environment managed by multiple Cloudera Manager instances
srm-control
srm-control Options Reference
Standalone
Standalone Functions
StandardManagedAuthorizer
StandardManagedAuthorizer
Start New NiFi
Start the data flow
Start the data flow
Start the data flow
Start the data flow
Start the data flow
Start Version Control
Start Version Control on a Process Group
Start your data flow
Start your data flow
Start your data flow
Starting a Component
Starting NiFi Registry
startsWith
State Management
State Management
State Manager
StateManager
StaticKeyProvider
StaticKeyProvider
Status
Status History Repository
Stop Version Control
Stopping a Component
Storing and Retrieving State
Streaming Analytics cluster layout
Streaming Analytics Data Hub cluster definitons
Streaming Analytics deployment scenarios
Streaming use cases with Flink
Streams Messaging
Streams Messaging cluster layout
Streams Messaging Manager
Streams Messaging Manager Overview
Streams Replication Manager
Streams Replication Manager Architecture
Streams Replication Manager Driver
Streams Replication Manager Overview
Streams Replication Manager Reference
Streams Replication Manager requirements
Streams Replication Manager Service
String Manipulation
Structure of a NiFi Expression
Structure of a RecordPath
Subjectless Functions
Subscribing to a topic
substring
substringAfter
substringAfterLast
substringBefore
substringBeforeLast
Summary Page
Supplying a contribution
Supported NiFi Controller Services
Supported NiFi Extensions
Supported NiFi Processors
Supported NiFi Reporting Tasks
Supporting API
Swap Management
Switching from other Flow Persistence Provider
System Level Broker Tuning
System Properties
System Properties
System Properties
System Requirements
System Requirements
Task architecture and load-balancing
Technologies
Templates
Terminating a Component's tasks
Terminology
Terminology
Terminology
Terminology Used in This Guide
Testing
The core concepts of NiFi
TLS Generation Toolkit
TLS Toolkit
TLS/SSL client authentication
toBytes
toDate
toLowerCase
Tool usage
Topics
Topics and Groups Subcommand
toString
toUpperCase
trim
Troubleshooting
Troubleshooting Kerberos Configuration
Tuning Apache Kafka Performance
Type Coercion
UI Extensions
Understand the NiFi Record Based Processors and Controller Services
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understanding Replication Flows
Understanding the kafka-run-class Bash Script
Understanding Version Dependencies
Unit Tests
Unlock Kafka metadata in Zookeeper
Unsupported Browsers
Unsupported Browsers
Unsupported command line tools
Unsupported Features in Cloudera DataFlow for Data Hub 7.2.9
Unsupported Flow Management features
Unsupported Streaming Analytics features
Unsupported Streams Messaging features
Update Attributes Based on Content
Update the Configuration Files for Your New NiFi Installation
Updating a notifier
Updating an alert policy
Upgrade Recommendations
Upgrading NiFi
Upload Bundle
URL Aliasing
Usage
Usage
Usage
Usage
Usage
Usage
Usage
Usage
Use Case 1: Registering and Querying a Schema for a Kafka Topic
Use Case 2: Reading/Deserializing and Writing/Serializing Data from and to a Kafka Topic
Use Case 3: Dataflow Management with Schema-based Routing
Use Case Architectures
Use cases
Use cases for Streams Replication Manager in CDP Public Cloud
Use Kerberos authentication
Use rsync to copy files from one broker to another
Use Schema Registry
User Authentication
User Authentication
User Authorization
User Window
UserGroupProvider
Using An Existing Intermediate Certificate Authority (CA)
Using Apache NiFi
Using Apache NiFi Registry
Using Custom Properties with Expression Language
Using Record-Enabled Processors
Using Schema Registry
Using SRM in CDP Public Cloud overview
Using Streams Replication Manager
Using the Apache NiFi Toolkit
Using watermark in Flink
uuid5
Validate Output
Validating Processor Properties
ValidationContext
Validator
Variables
Variables in Versioned Flows
Verify data flow operation
Verify data flow operation
Verify data flow operation
Verify data flow operation
Verify data flow operation
Verify that you can write data to Kudu
Verify your data flow
Verify your data flow
Verifying the setup
Version States
Versioning a DataFlow
View a Flow
Viewing data lineage in Apache Atlas
Viewing FlowFile Lineage
Viewing Kafka cluster replication details
Viewing Policies on Users
Viewing the UI in Variably Sized Browsers
Viewing the UI in Variably Sized Browsers
Virtual memory handling
Volatile Content Repository Properties
Volatile FlowFile Repository
Volatile Provenance Repository Properties
Web Properties
Web Properties
What is Apache Flink?
What is Apache NiFi?
What is it?
What is it?
What is it?
What to do next
What's New in Cloudera DataFlow for Data Hub 7.2.9
What's New in Flow Management
What's New in Streaming Analytics
What's New in Streams Messaging
When Processors are Triggered
Where To Go For More Information
Where to Start?
Who is This Guide For?
Why Cluster?
Wildcard Certificates
Write Ahead FlowFile Repository
Write Ahead Provenance Repository
Write Ahead Provenance Repository Properties
Writing and Reading Content Claims
Writing and Reading Event Records
Writing and Reading FlowFiles
Zero-Leader Clustering
ZooKeeper Access Control
ZooKeeper Migration Steps
ZooKeeper Migrator
ZooKeeper Migrator
ZooKeeper Properties
zookeeper-security-migration
«
Filter topics
1.4. URI scheme
▼
1. Overview
1.1. Version information
1.2. Contact information
1.3. License information
1.4. URI scheme
1.5. Tags
▶︎
2. Security
2.1. Authorization
2.2. BasicAuth
▶︎
3. Resources
▶︎
3.1. Access
3.1.1. Get access status
3.1.2. Performs a logout for other providers that have been issued a JWT.
3.1.3. Redirect/callback URI for processing the result of the OpenId Connect login sequence.
3.1.4. Retrieves a JWT following a successful login sequence using the configured OpenId Connect provider.
3.1.5. Performs a logout in the OpenId Provider.
3.1.6. Initiates a request to authenticate through the configured OpenId Connect provider.
3.1.7. Create token trying all providers
3.1.8. Create token using identity provider
3.1.9. Test identity provider
3.1.10. Get identity provider usage
3.1.11. Create token using kerberos
3.1.12. Create token using basic auth
▶︎
3.2. Bucket Bundles
3.2.1. Get extension bundles by bucket
3.2.2. Create extension bundle version
▶︎
3.3. Bucket Flows
3.3.1. Create flow
3.3.2. Get bucket flows
3.3.3. Get bucket flow
3.3.4. Update bucket flow
3.3.5. Delete bucket flow
3.3.6. Get bucket flow diff
3.3.7. Create flow version
3.3.8. Get bucket flow versions
3.3.9. Get latest bucket flow version content
3.3.10. Get latest bucket flow version metadata
3.3.11. Get bucket flow version
▶︎
3.4. Buckets
3.4.1. Create bucket
3.4.2. Get all buckets
3.4.3. Get bucket fields
3.4.4. Get bucket
3.4.5. Update bucket
3.4.6. Delete bucket
▶︎
3.5. Bundles
3.5.1. Get all bundles
3.5.2. Get all bundle versions
3.5.3. Get bundle
3.5.4. Delete bundle
3.5.5. Get bundle versions
3.5.6. Get bundle version
3.5.7. Delete bundle version
3.5.8. Get bundle version content
3.5.9. Get bundle version extensions
3.5.10. Get bundle version extension
3.5.11. Get bundle version extension docs
3.5.12. Get bundle version extension docs details
▶︎
3.6. Config
3.6.1. Get configration
▶︎
3.7. Extension Repository
3.7.1. Get extension repo buckets
3.7.2. Get extension repo groups
3.7.3. Get extension repo artifacts
3.7.4. Get extension repo versions
3.7.5. Get extension repo version
3.7.6. Get extension repo version content
3.7.7. Get extension repo extensions
3.7.8. Get extension repo extension
3.7.9. Get extension repo extension docs
3.7.10. Get extension repo extension details
3.7.11. Get extension repo version checksum
3.7.12. Get global extension repo version checksum
▶︎
3.8. Extensions
3.8.1. Get all extensions
3.8.2. Get extensions providing service API
3.8.3. Get extension tags
▶︎
3.9. Flows
3.9.1. Get flow fields
3.9.2. Get flow
3.9.3. Get flow versions
3.9.4. Get latest flow version
3.9.5. Get latest flow version metadata
3.9.6. Get flow version
▶︎
3.10. Items
3.10.1. Get all items
3.10.2. Get item fields
3.10.3. Get bucket items
▶︎
3.11. Policies
3.11.1. Create access policy
3.11.2. Get all access policies
3.11.3. Get available resources
3.11.4. Get access policy for resource
3.11.5. Get access policy
3.11.6. Update access policy
3.11.7. Delete access policy
▶︎
3.12. Tenants
3.12.1. Create user group
3.12.2. Get user groups
3.12.3. Get user group
3.12.4. Update user group
3.12.5. Delete user group
3.12.6. Create user
3.12.7. Get all users
3.12.8. Get user
3.12.9. Update user
3.12.10. Delete user
▶︎
4. Definitions
4.1. AccessPolicy
4.2. AccessPolicySummary
4.3. AllowableValue
4.4. Attribute
4.5. BatchSize
4.6. Bucket
4.7. BucketItem
4.8. BuildInfo
4.9. Bundle
4.10. BundleInfo
4.11. BundleVersion
4.12. BundleVersionDependency
4.13. BundleVersionMetadata
4.14. ComponentDifference
4.15. ComponentDifferenceGroup
4.16. ConnectableComponent
4.17. ControllerServiceAPI
4.18. ControllerServiceDefinition
4.19. CurrentUser
4.20. DeprecationNotice
4.21. DynamicProperty
4.22. DynamicRelationship
4.23. Extension
4.24. ExtensionBundle
4.25. ExtensionFilterParams
4.26. ExtensionMetadata
4.27. ExtensionMetadataContainer
4.28. ExtensionRepoArtifact
4.29. ExtensionRepoBucket
4.30. ExtensionRepoGroup
4.31. ExtensionRepoVersion
4.32. ExtensionRepoVersionSummary
4.33. ExternalControllerServiceReference
4.34. Fields
4.35. JaxbLink
4.36. Permissions
4.37. Position
4.38. Property
4.39. ProvidedServiceAPI
4.40. RegistryConfiguration
4.41. Relationship
4.42. Resource
4.43. ResourcePermissions
4.44. Restricted
4.45. Restriction
4.46. RevisionInfo
4.47. Stateful
4.48. SystemResourceConsideration
4.49. TagCount
4.50. Tenant
4.51. User
4.52. UserGroup
4.53. VersionedConnection
4.54. VersionedControllerService
4.55. VersionedFlow
4.56. VersionedFlowCoordinates
4.57. VersionedFlowDifference
4.58. VersionedFlowSnapshot
4.59. VersionedFlowSnapshotMetadata
4.60. VersionedFunnel
4.61. VersionedLabel
4.62. VersionedParameter
4.63. VersionedParameterContext
4.64. VersionedPort
4.65. VersionedProcessGroup
4.66. VersionedProcessor
4.67. VersionedPropertyDescriptor
4.68. VersionedRemoteGroupPort
4.69. VersionedRemoteProcessGroup
»
Apache NiFi Registry REST API Reference
1.4. URI scheme
BasePath
: /nifi-registry-api
Schemes
: HTTP, HTTPS
Parent topic:
1. Overview
This site uses cookies and related technologies, as described in our
privacy policy
, for purposes that may include site operation, analytics, enhanced user experience, or advertising. You may choose to consent to our use of these technologies, or
manage your own preferences.
Accept all
7.3.1
7.2
7.2.18
7.2.17
7.2.16
7.2.15
7.2.14
7.2.12
7.2.11
7.2.10
7.2.9
7.2.8
7.2.7
7.2.6
7.2.2
7.2.1
7.2.0
7.1.0
7.0.2