Homepage/Cloudera DataFlow for Data Hub7.2.15

(Public Cloud)

Search Documentation

▶︎Cloudera
1. Reference Architectures
▼Cloudera Public Cloud
▶︎Cloudera Private Cloud
▶︎Cloudera Manager
1. Cloudera Manager
▶︎Applications
▶︎Legacy
▶︎

Getting Started
Patterns
Preview Features
Data Catalog
Data Engineering
DataFlow
Data Hub
Data Warehouse
Data Warehouse Runtime
Cloudera AI
Management Console
Operational Database
Replication Manager
Cloudera Manager
CDF for Data Hub
Runtime

«

Filter topics

CDF for Data Hub
▶︎Release Notes
▶︎Flow Management
▶︎Streams Messaging
▼Streaming Analytics

.NET client
A records and round robin DNS
Active / Active Architecture
Active / Stand-by Architecture
Add Ranger policies
Add Ranger policies
Add the user or group to a pre-defined access policy
Add the user to predefined Ranger access policies
Adding a new schema
Adding and configuring record reader and writer controller services
Adding catalogs as Data Provider
Adding clusters to SRM's configuration
Adding Custom Catalogs
Adding data formats
Adding Hive as Catalog
Adding Java to the Functions language option
Adding Kafka as Data Provider
Adding Kudu as Catalog
Adding new connectors
Adding Schema Registry as Catalog
Adding self-healing goals to Cruise Control in Cloudera Manager
Adding Snowflake CA certificates to NiFi truststore
Adding Snowflake CA certificates to NiFi truststore
Adjusting logging configuration in Advanced Settings
ADLS Sink
ADLS Sink properties reference
Aggregation for Analytics
Amazon S3 Sink
Amazon S3 Sink properties reference
Analyzing data with Apache Flink
Analyzing your data with HBase
Analyzing your data with Kafka
Analyzing your data with Kudu
Apache Flink
Apache Kafka
Apache Kafka overview
Apache NiFi Developer Guide
Apache NiFi Developer Guide
Apache NiFi Expression Language Guide
Apache NiFi Expression Language Guide
Apache NiFi RecordPath Guide
Apache NiFi RecordPath Guide
Apache NiFi Registry REST API
Apache NiFi Registry REST API
Apache NiFi Registry System Administrator Guide
Apache NiFi Registry System Administrator Guide
Apache NiFi REST API
Apache NiFi REST API Reference
Apache NiFi System Administrator Guide
Apache NiFi System Administrator Guide
Appendix - Schema example
Application development
Assign resource roles
Assign the EnvironmentUser role
Assigning administrator level permissions
Assigning Kafka keys in streaming queries
Assigning resource roles
Assigning selective permissions to user
Atlas entities in Flink metadata collection
Authentication
Authentication
Authentication using OAuth2 with Kerberos
Authorization
Authorization
Authorization example
Authorization model
Authorization workflow
Authorizing Flow Management cluster access in CDP Public Cloud
Basics
Before you begin
Before you begin
Behavioral Changes in Cloudera DataFlow for Data Hub 7.2.15
Behavioral Changes in Streaming Analytics
Bidirectional replication example of two active clusters
Bidirectional Replication Flows
Blackhole connector
Broker garbage log collection and log rotation
Broker log management
Broker migration
Broker Tuning
Brokers
Build the data flow
Build the data flow
Build the data flow
Build the data flow
Build the data flow
Build the data flow
Build the data flow
Build the data flow
Build the data flow
Building Cloudera Manager charts with Kafka metrics
Building your dataflow
Building your dataflow
CDC connectors
CDF for Data Hub
Channel encryption
Checking prerequisites
Checking producer activity
Checking schema registration
Choosing the number of partitions for a topic
Client and broker compatibility across Kafka versions
Client authentication using delegation tokens
Client examples
Client examples
client.dns.lookup property options for client
ClouderaRegistryKafkaDeserializationSchema
ClouderaRegistryKafkaSerializationSchema
Cluster discovery using DNS records
Cluster discovery using load balancers
Cluster discovery with multiple Apache Kafka clusters
Cluster Migration Architectures
Cluster sizing
CNAME records configuration
Command Line Tools
Compatibility policies
Component Support in Cloudera DataFlow for Data Hub 7.2.15
Component types and metrics for alert policies
Components Supported by Partners
Compose Tab
Concept of tables in SSB
Configuration example
Configuration example for writing data to HDFS
Configuration example for writing data to Ozone FS
Configuration examples
Configuration Properties Reference for Properties not Available in Cloudera Manager
Configure clients on a producer or consumer level
Configure clients on an application level
Configure data directories for clusters with custom disk configurations
Configure each object store processor
Configure JMX ephemeral ports
Configure Kafka brokers
Configure Kafka brokers
Configure Kafka brokers
Configure Kafka brokers
Configure Kafka clients
Configure Kafka clients
Configure Kafka clients
Configure Kafka clients
Configure Kafka MirrorMaker
Configure Ranger policies for site-to-site communication
Configure SRM for Failover and Failback
Configure the Controller Service
Configure the controller services
Configure the HBase client service
Configure the processor for merging records
Configure the processor for merging records
Configure the processor for merging records
Configure the processor for your data source
Configure the processor for your data source
Configure the processor for your data source
Configure the processor for your data source
Configure the processor for your data source
Configure the processor for your data source
Configure the processor for your data source
Configure the processor for your data source
Configure the processor for your data target
Configure the processor for your data target
Configure the processor for your data target
Configure the processor for your data target
Configure the processor for your data target
Configure the processor for your data target
Configure the processor for your data target
Configure the processor for your data target
Configure the resource-based Ranger service used for authorization
Configure the service account
Configure your source processor
Configure your truststores
Configure Zookeeper TLS/SSL support for Kafka
Configuring a Nexus repository allow list
Configuring Apache Kafka
Configuring automatic group offset synchronization
Configuring Basic Authentication for Remote Querying
Configuring Basic Authentication for the SRM Service
Configuring connector form
Configuring connector JAAS configuration and Kerberos principal overrides
Configuring Cruise Control
Configuring Flink application resources
Configuring Flink applications
Configuring Flow Management clusters to hot load custom NARs
Configuring flow.snapshot
Configuring Kafka brokers
Configuring Kafka clients
Configuring Kafka tables
Configuring Kafka ZooKeeper chroot
Configuring Kerberos authentication
Configuring Kerberos properties
Configuring LDAP authentication
Configuring log levels for command line tools
Configuring Materialized View database information
Configuring Metrics Reporter in Cruise Control
Configuring multiple listeners
Configuring properties for non-Kerberos authentication mechanisms
Configuring properties not exposed in Cloudera Manager
Configuring Ranger policies for SSB
Configuring Remote Querying
Configuring replication specific REST servers
Configuring replications
Configuring Retention Time for Materialized Views
Configuring RocksDB state backend
Configuring rolling restart checks
Configuring SMM for monitoring Kafka cluster replications
Configuring SPNEGO authentication and trusted proxies
Configuring SQL job settings
Configuring SRM Driver heartbeat emission
Configuring SRM Driver retry behaviour
Configuring srm-control
Configuring Streams Replication Manager
Configuring the advertised information of the SRM Service role
Configuring the Atlas hook in Kafka
Configuring the client configuration used for rolling restart checks
Configuring the driver role target clusters
Configuring the Kafka Connect Role
Configuring the Schema Registry client
Configuring the service role target cluster
Configuring the SRM client's secure storage
Configuring TLS/SSL client authentication
Configuring TLS/SSL encryption
Configuring TLS/SSL properties
Configuring YARN queue for SQL jobs
Configuring your Controller Services
Configuring your source processor
Configuring your target processor
Configuring your target processor
Confirming your dataflow success
Confirming your dataflow success
Connect workers
Connecting Kafka clients to CDP Public Cloud clusters
Connecting Kafka clients to Data Hub provisioned clusters
Connecting Kafka clients to Data Hub provisioned clusters
Connecting to Kafka host
Connecting to NiFi Registry with NiFi Toolkit CLI
Connecting to the Kafka cluster using load balancer
Connection to the cluster with configured DNS aliases
Connector support in SSB
Connector support in SSB
Connectors
Connectors
Connectors in SSB
Console Page
Consuming data from Kafka topic
Consuming data from Kafka topics using stored schemas
Core Features of Flink
Create a custom access policy
Create a custom access policy
Create Atlas entity type definitions
Create consumer group policy
Create controller services for your data flow
Create controller services for your data flow
Create controller services for your data flow
Create controller services for your data flow
Create controller services for your data flow
Create IDBroker mapping
Create IDBroker mapping
Create IDBroker mapping
Create IDBroker mapping
Create IDBroker mapping
Create Ranger policies for Machine User account
Create Solr target collection
Create the HBase target table
Create the Hive target table
Create the Kudu target table
Create topic policy
Create your cluster
Create your cluster
Create your cluster
Create your streaming clusters
Creating a connector using Kafka Connect in SMM
Creating a Kafka topic
Creating a notifier
Creating an alert policy
Creating and configuring the HBaseSinkFunction
Creating Controller Services for your dataflow
Creating Flink tables using Templates
Creating IDBroker mapping
Creating Input Transforms
Creating Kafka tables
Creating Kafka tables in SSB
Creating Kafka tables using Console wizard
Creating Kafka tables using Templates
Creating Kafka topic
Creating Machine User
Creating Materialized Views
Creating Streaming Analytics cluster
Creating tables with Flink SQL in SSB
Creating TLS truststore
Creating User Defined Functions
Creating Webhook tables
Creating your first Flow Management cluster
Creating your first Flow Management cluster in CDP Public Cloud
Creating your first Streaming Analytics cluster
Creating your first Streaming Analytics Cluster in CDP Public Cloud
Creating your first Streams Messaging cluster
Creating your First Streams Messaging cluster in CDP Public Cloud
Cross Data Center Replication
Cross data center replication example of multiple clusters
Cruise Control
Cruise Control overview
Cruise Control REST API endpoints
Data Hub cluster definitions
Data Providers Page
Data Types
Dataflow development best practices
Datagen connector
DataStream connectors
Debezium MySQL Source
Debezium Oracle Source
Debezium PostgreSQL Source
Debezium SQL Server Source
Default view of Kafka Connect in the SMM UI
Define your CDP Private Cloud Base dataflow
Define your CDP Public Cloud dataflow
Defining and adding clusters for replication
Defining co-located Kafka clusters using a service dependency
Defining co-located Kafka clusters using Kafka credentials
Defining external Kafka clusters
Defining Schema Registry access policies
Delegation token based authentication
Deleting a connector using Kafka Connect in SMM
Deleting a Kafka topic
Deleting a notifier
Deleting a schema
Deleting an alert policy
Deploying a dataflow
Deployment scenarios
Deployment scenarios
Deserialization tab
Developing a dataflow
Developing Apache Kafka applications
Developing JavaScript functions
Disabling an alert policy
Disk management
Disk Removal
Disk Replacement
Downloading and viewing predefined dataflows
Downloading the Snowflake JDBC driver jar file
Driver inter-node coordination
Dynamic SQL Hints
Enable authorization in Kafka with Ranger
Enable high availability
Enable Kerberos authentication
Enable or disable authentication with delegation tokens
Enable security for Cruise Control
Enabling an alert policy
Enabling Basic Authentication for the SRM Service
Enabling checkpoints for Flink applications
Enabling end-to-end latency monitoring
Enabling Flink DEBUG logging
Enabling interceptors
Enabling Kerberos for the SRM service
Enabling Remote Querying
Enabling savepoints for Flink applications
Enabling TLS/SSL for the SRM service
End to end latency overview
End to end latency use case
Enriching streaming data with join
Essential metrics to monitor
Event Time tab
Evolve your schema
Evolving a schema
Example: joining Kafka and Kudu tables
Examples of interacting with Schema Registry
Exchanging data with external systems
Executing SQL jobs in production mode
Exporting a flow from NiFi Registry
Exporting or importing data flows with NiFi Toolkit CLI
Exporting schemas
Faker connector
Fan-in and Fan-out Replication Flows
File descriptor limits
Filesystem connector
Filesystems
Finding list of brokers
Finding Schema Registry endpoint
Fixed CVEs in Cloudera DataFlow for Data Hub 7.2.15
Fixed CVEs in Flow Management
Fixed Issues in Cloudera DataFlow for Data Hub 7.2.15
Fixed Issues in Flow Management
Fixed Issues in Streaming Analytics
Fixed Issues in Streams Messaging
Flink application example
Flink application structure
Flink Dashboard
Flink DDL
Flink DML
Flink metadata collection using Atlas
Flink Project Template
Flink Queries
Flink SQL Overview
Flow Management
Flow Management cluster definitions
Flow Management cluster layout
Flow Management overview
Functions Tab
Gather configuration information
Getting started with Apache NiFi
Getting started with Apache NiFi
Getting started with Apache NiFi Registry
Getting started with Apache NiFi Registry
Getting started with Streams Messaging clusters in CDP Public Cloud
Give users access to your cluster
Give users access to your cluster
Give users access to your cluster
Governance
Governance
Grant permission for the ATLAS_HOOK topic
Granting Machine User access to environment
Granularity of metrics for end-to-end latency
Groups and fetching
Handling disk failures
Handling large messages
HBase sink with Flink
HDFS Sink
HDFS Sink properties reference
Highly Available Kafka Architectures
History Tab
Hot loading custom NARs
How to Set up Failover and Failback
HTTP SInk
HTTP Sink properties reference
HTTP Source
HTTP Source properties reference
ID ranges in Schema Registry
Importing a new flow into NiFi Registry
Importing and exporting schemas
Importing Confluent Schema Registry schemas into Cloudera Schema Registry
Importing connector configuration
Importing Kafka entities into Atlas
Importing schemas
Improve performance in Schema Registry
Ingesting data into Amazon S3
Ingesting data into Amazon S3 Buckets
Ingesting Data into Apache HBase in CDP Cloud
Ingesting Data into Apache HBase in CDP Public Cloud
Ingesting Data into Apache Hive in CDP Public Cloud
Ingesting Data into Apache Hive in CDP Public Cloud
Ingesting data into Apache Kafka
Ingesting Data into Apache Kafka in CDP Public Cloud
Ingesting data into Apache Kudu in CDP Public Cloud
Ingesting Data into Apache Kudu in CDP Public Cloud
Ingesting data into Apache Solr
Ingesting data into Apache Solr in CDP Public Cloud
Ingesting data into Azure Data Lake Storage
Ingesting data into Azure Data Lake Storage
Ingesting data into CDP Object Stores with RAZ authorization
Ingesting data into CDP Public Cloud
Ingesting data into cloud object stores with RAZ authorizations
Ingesting data into Google Cloud Storage
Ingesting data into Google Cloud Storage
Installing Connectors
Integrating Kafka and Schema Registry
Integrating with Atlas
Integrating with Flink and SSB
Integrating with NiFi
Integrating with Schema Registry
Inter-broker security
Introducing streams messaging cluster on CDP Public Cloud
Introduction to alert policies in Streams Messaging Manager
Introduction to Kafka Connect
Introduction to Materialized Views
Introduction to monitoring Kafka cluster replications in SMM
Introduction to SQL Stream Builder
Introduction to Streams Messaging Manager
ISR management
Java client
JBOD
JBOD Disk migration
JBOD setup
JDBC connector
JDBC Sink
JDBC Sink properties reference
JDBC Source
JDBC Source properties reference
JMS Source
JMS Source properties reference
Job lifecycle
Job Lifecycle
Job monitoring with Flink Dashboard
Job monitoring with Flink Dashboard
Job monitoring with Flink Dashboard
Joining streaming and bounded tables
JVM and garbage collection
JWT algorithms
Kafka Architecture
Kafka brokers and Zookeeper
Kafka clients and ZooKeeper
Kafka cluster load balancing using Cruise Control
Kafka Connect
Kafka Connect connector configuration security
Kafka Connect Connector Reference
Kafka Connect log files
Kafka Connect Overview
Kafka Connect REST API security
Kafka Connect Secrets Storage
Kafka Connect Setup
Kafka Connect tasks
Kafka Connect to Kafka broker security
Kafka Connect worker assignment
Kafka connectors
Kafka consumers
Kafka credentials property reference
Kafka FAQ
Kafka Introduction
Kafka Metrics Reporter
Kafka producers
Kafka public APIs
Kafka security hardening with Zookeeper ACLs
Kafka Streams
Kafka with Flink
kafka-*-perf-test
kafka-configs
kafka-console-consumer
kafka-console-producer
kafka-consumer-groups
kafka-delegation-tokens
kafka-log-dirs
kafka-reassign-partitions
kafka-topics
Kafka-ZooKeeper performance tuning
Kerberos authentication using a keytab
Kerberos authentication using the ticket cache
Key Features
Key features of SSB
Known Issues In Cloudera DataFlow for Data Hub 7.2.15
Known Issues in Flow Management
Known Issues in Streaming Analytics
Known Issues in Streams Messaging
Kudu Sink
Kudu Sink properties reference
Kudu with Flink
LDAP authentication
Leader positions and in-sync replicas
Log cleaner
Log4j vulnerabilities
Logs and log segments
Main Use Cases
Manage individual delegation tokens
Management basics
Managing alert policies
Managing alert policies and notifiers in SMM
Managing Apache Kafka
Managing connectors and data formats
Managing Cruise Control
Managing Kafka topics
Managing registered Data Providers
Managing secrets using the REST API
Managing session for SQL jobs
Managing teams in Streaming SQL Console
Managing time in SSB
Managing topics across multiple Kafka clusters
Managing, Deploying and Monitoring Connectors
Materialized Views Page
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites
Meet the prerequisites to create streams messaging cluster
Metadata governance with Atlas
Metadata governance with Atlas
Metadata governance with Atlas
Metrics
Migrate brokers by modifying broker IDs in meta.properties
Migrating Consumer Groups Between Clusters
Migrating Flink jobs
Migrating Flink jobs without state
Migrating stateful Flink jobs
Modifying a connector using Kafka Connect in SMM
Modifying a Kafka topic
Monitor end-to-end latency
Monitoring
Monitoring
Monitoring checkpoint latency for cluster replication
Monitoring cluster profile using Kafka Connect in SMM
Monitoring connector profile using Kafka Connect in SMM
Monitoring connector settings using Kafka Connect in SMM
Monitoring connectors using Kafka Connect in SMM
Monitoring end to end latency for Kafka topic
Monitoring end to end latency for Kafka topic
Monitoring end-to-end latency
Monitoring Kafka activity in Streams Messaging Manager
Monitoring Kafka brokers
Monitoring Kafka cluster replications by quick ranges
Monitoring Kafka cluster replications using Streams Messaging Manager
Monitoring Kafka clusters
Monitoring Kafka clusters
Monitoring Kafka Connect using Streams Messaging Manager
Monitoring Kafka consumers
Monitoring Kafka producers
Monitoring Kafka topics
Monitoring lineage information
Monitoring replication latency for cluster replication
Monitoring replication throughput and latency by values
Monitoring Replication with Streams Messaging Manager
Monitoring SQL Stream jobs
Monitoring status of the clusters to be replicated
Monitoring throughput for cluster replication
Monitoring topics to be replicated
Monitoring your data flow
Monitoring your data flow
Monitoring your data flow
Monitoring your data flow
Monitoring your data flow
Monitoring your data flow
Moving data from CDP Private Cloud Base to Public Cloud with NiFi site-to-site
Moving data in and out of Snowflake
Moving data out of Snowflake
Moving data using NiFi site-to-site
MQTT Source
MQTT Source properties reference
Network and I/O threads
Networking parameters
New topic and consumer group discovery
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
Next steps
Notifiers
OAuth2 authentication
Obtain HBase connection details
Obtain Hive connection details
Offsets Subcommand
On-premise to Cloud and Kafka Version Upgrade
Operating system requirements
Operating your Flow Management cluster
Other supported statements
Overview
PAM authentication
Partitions
Performance & Scalability
Performance considerations
Performant .NET producer
Planning for Streams Replication Manager
Planning your Flow Management deployment
Planning your Streaming Analytics deployment
Planning your Streams Messaging deployment
Pre-defined access policies for Schema Registry
Predefined Ranger Access Policies for Apache NiFi
Predefined Ranger Access Policies for Apache NiFi Registry
Prepare your clusters
Prepare your environment
Prepare your environment
Principal name mapping
Produce data to Kafka topic
Producing data in Avro format
Producing data to Kafka topic
Properties tab
Protocol between consumer and broker
Public key and secret storage
Pushing data into Snowflake
Pushing data to and moving data from Snowflake using Apache NiFi
Querying a schema
Querying data with SQL Stream Builder
Quotas
Rack awareness
Ranger
Ranger integration
Re-encrypting secrets
Reassigning replicas between log directories
Reassignment examples
Rebalancing partitions
Rebalancing with Cruise Control
Recommendations for client development
Recommended deployment architecture
Reconfiguring the Kafka consumer
Reconfiguring the Kafka producer
Record management
Record order and assignment
Records
Registering Data Providers in SSB
Release Notes
Remote Querying
Remote Topics
Replicate data between Data Hub clusters with cloud SRM
Replicating Data
Replicating data from PvC Base to Data Hub with cloud SRM
Replicating data from PvC Base to Data Hub with on-prem SRM
Replication Flows Overview
Restricting access to Kafka metadata in Zookeeper
Retries
Retrieve and upload keytab file
Retrieving keytab file
Retrieving log directory replica assignment information
Rolling restart checks
Rotate the master key/secret
Running a Flink job
Running a simple Flink application
Running a simple SQL job
Running SQL Stream jobs
Running your Flink application
Running your Flink application
Running your Flink application
S3 Sink
S3 Sink properties reference
Sampling data for a running job
Scaling down a NiFi cluster
Scaling down Kafka brokers
Scaling down Kafka Connect
Scaling Kafka brokers
Scaling Kafka Connect
Scaling Streams Messaging clusters
Scaling up a NiFi cluster
Scaling up Kafka brokers
Scaling up Kafka Connect
Scaling up or down a NiFi cluster
Scaling your Flow Management cluster
Schema entities
Schema Registry
Schema Registry authentication through OAuth2 JWT tokens
Schema Registry authorization through Ranger access policies
Schema Registry component architecture
Schema Registry concepts
Schema Registry overview
Schema Registry overview
Schema Registry server configuration
Schema Registry use cases
Schema Registry with Flink
Schema tab
Searching by topic name
Searching Kafka cluster replications by source
Securing Apache Kafka
Securing Cruise Control
Securing Kafka Connect
Securing Schema Registry
Securing Streams Messaging Manager
Securing Streams Messaging Manager
Securing Streams Replication Manager
Security examples
Security examples
Security for Flow Management Clusters and Users in CDP Public Cloud
Security overview
Set permissions in Ranger
Set Ranger policies
Set up AWS for your ingest data flow
Set up MirrorMaker in Cloudera Manager
Set up the HortonworksSchemaRegistry controller service
Set up your network configuration
Set workload password
Setting a Schema Registry ID range
Setting capacity estimations and goals
Setting parallelism and max parallelism
Setting the secure storage password as an environment variable
Setting up authorization policies
Setting user limits for Kafka
Setting workload password
Setting workload password
Settings to avoid data loss
Setup for SASL with Kerberos
Setup for TLS/SSL encryption
SFTP Source
SFTP Source properties reference
Simple .NET consumer
Simple .Net consumer using Schema Registry
Simple .NET producer
Simple .Net producer using Schema Registry
Simple Java consumer
Simple Java producer
Sizing estimation based on network and disk message throughput
Source, operator and sink in DataStream API
SQL Examples
SQL Jobs Tab
SQL Stream Builder
SQL Stream Builder architecture
SRM Command Line Tools
SRM security example
SRM Service data traffic reference
srm-control
srm-control Options Reference
Start the data flow
Start the data flow
Start the data flow
Start the data flow
Start the data flow
Start the data flow
Start your data flow
Start your data flow
Start your data flow
Stateless NiFi Sink properties reference
Stateless NiFi Source and Sink
Stateless NiFi Source properties reference
Stopping, restarting and editing SQL jobs
Streaming Analytics
Streaming Analytics cluster layout
Streaming Analytics Data Hub cluster definitons
Streaming Analytics deployment scenarios
Streaming Analytics in Cloudera
Streaming Analytics overview
Streams Messaging
Streams Messaging cluster layout
Streams Messaging Manager
Streams Messaging Manager integration
Streams Messaging Manager overview
Streams Replication Manager
Streams Replication Manager Architecture
Streams Replication Manager Driver
Streams Replication Manager overview
Streams Replication Manager reference
Streams Replication Manager requirements
Streams Replication Manager Service
Subscribing to a topic
Supported NiFi Controller Services
Supported NiFi Extensions
Supported NiFi Processors
Supported NiFi Reporting Tasks
Syslog TCP Source
Syslog TCP Source properties reference
Syslog UDP Source
Syslog UDP Source properties reference
System Level Broker Tuning
Tables Tab
Task architecture and load-balancing
Terms and concepts
Testing and validating Flink applications
The downscale operation fails with decommission failed
TLS/SSL client authentication
Tool usage
Topics
Topics and Groups Subcommand
Transformations tab
Troubleshooting
Tuning Apache Kafka performance
Tutorial: developing and deploying a JDBC Source dataflow
Understand the NiFi record based processors and controller services
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understand the use case
Understanding co-located and external clusters
Understanding Replication Flows
Understanding SRM properties, their configuration and hierarchy
Understanding the kafka-run-class Bash Script
Unlocking access to Kafka metadata in Zookeeper
Unsupported command line tools
Unsupported Features in Cloudera DataFlow for Data Hub 7.2.15
Unsupported Flow Management features
Unsupported Streaming Analytics features
Unsupported Streams Messaging features
Updating a notifier
Updating an alert policy
Updating Flink job dependencies
Uploading and unlocking your keytab in SSB
Use case 1: Registering and querying a Schema for a Kafka topic
Use case 2: Reading/deserializing and writing/serializing data from and to a Kafka topic
Use case 3: Dataflow management with schema-based routing
Use Case Architectures
Use cases
Use cases for Streams Replication Manager in CDP Public Cloud
Use Kerberos authentication
Use rsync to copy files from one broker to another
Use Schema Registry
User Authorization
Using Apache NiFi
Using Apache NiFi
Using Apache NiFi Registry
Using Apache NiFi Registry
Using Apache NiFi Toolkit
Using Apache NiFi Toolkit
Using Dynamic Materialized View Endpoints
Using Flink CLI
Using Kafka Connect
Using Materialized Views in SQL Stream Builder
Using record-enabled processors
Using Schema Registry
Using SQL Stream Builder REST API
Using SQL Stream Builder with Cloudera Data Visualization
Using SRM in CDP Public Cloud overview
Using Streams Replication Manager
Using System Functions
Using the Streaming SQL Console
Validating connector configuration
Verify data flow operation
Verify data flow operation
Verify data flow operation
Verify data flow operation
Verify data flow operation
Verify data flow operation
Verify that you can write data to Kudu
Verify your data flow
Verify your data flow
Verifying the setup
Viewing data lineage in Apache Atlas
Viewing Kafka cluster replication details
Virtual memory handling
What is Apache Flink?
What is Apache NiFi
What is Apache NiFi Registry
What to do next
What's New in Cloudera DataFlow for Data Hub 7.2.15
What's New in Flow Management
What's New in Streaming Analytics
What's New in Streams Messaging
Working with flows in Registry using NiFi Toolkit CLI
Writing data in a Kerberos and TLS/SSL enabled cluster
Writing data in an unsecured cluster
Writing Kafka data to Ozone with Kafka Connect
zookeeper-security-migration

«

Filter topics

Job lifecycle

Running a simple Flink application
▶︎Application development
▶︎DataStream connectors
▼Job lifecycle
▶︎Monitoring
▶︎Governance
1. Flink metadata collection using Atlas
2. Atlas entities in Flink metadata collection
▶︎Migrating Flink jobs

»

Job lifecycle

Running a Flink job
After developing your application, you can submit your Flink job in YARN per-job or session mode. To submit the Flink job, you need to run the Flink client in the command line including security parameters and other configurations with the run command.
Using Flink CLI
You can use the Flink command line interface to operate, configure and maintain your Flink applications.
Enabling savepoints for Flink applications
Beside checkpointing, you are also able to create a savepoint of your executed Flink jobs. Savepoints are not automatically created, so you need to trigger them in case of upgrade or maintenance. You can also resume your applications from savepoint.

© 2019, 2021–2022 by Cloudera, Inc. All rights reserved.

We want your opinion

How can we improve this page?

What kind of feedback do you have?

This site uses cookies and related technologies, as described in our privacy policy, for purposes that may include site operation, analytics, enhanced user experience, or advertising. You may choose to consent to our use of these technologies, or

7.3.1
7.2
7.1.0
7.0.2