Homepage
/
Stream Processing
2.0.1
(CDH • latest)
Search Documentation
▶︎
Cloudera
Reference Architectures
▶︎
Cloudera Public Cloud
Getting Started
Patterns
Preview Features
Data Catalog
Data Engineering
DataFlow
Data Hub
Data Warehouse
Data Warehouse Runtime
Cloudera AI
Management Console
Operational Database
Replication Manager
DataFlow for Data Hub
Runtime
▶︎
Cloudera Private Cloud
Data Services
Getting Started
Cloudera Manager
Management Console
Replication Manager
Data Catalog
Data Engineering
Data Warehouse
Data Warehouse Runtime
Cloudera AI
Base
Getting Started
Runtime
Upgrade
Storage
Flow Management
Streaming Analytics
Flow Management Operator
Streaming Analytics Operator
Streams Messaging Operator
▶︎
Cloudera Manager
Cloudera Manager
▶︎
Applications
Cloudera Streaming Community Edition
Data Science Workbench
Data Visualization
Edge Management
Observability SaaS
Observability on premises
Workload XM On-Prem
▼
Legacy
Cloudera Enterprise
Flow Management
Stream Processing
HDP
HDF
Streams Messaging Manager
Streams Replication Manager
▶︎
Cloudera Enterprise
Flow Management
Stream Processing
HDP
HDF
Streams Messaging Manager
Streams Replication Manager
«
Filter topics
Cloudera Stream Processing
▶︎
Release Notes
What's new in Cloudera Stream Processing
Component support
Known issues
Download locations
▶︎
Concepts
▶︎
Cloudera Stream Processing Overview
CSP Architecture
▶︎
Streams Messaging Manager Overview
Streams Messaging Manager Overview
Kafka Overview
▶︎
Schema Registry Overview
▶︎
Schema Registry Overview
Examples of Interacting with Schema Registry
▶︎
Schema Registry Use Cases
Use Case 1: Registering and Querying a Schema for a Kafka Topic
Use Case 2: Reading/Deserializing and Writing/Serializing Data from and to a Kafka Topic
Use Case 3: Dataflow Management with Schema-based Routing
Schema Registry Component Architecture
▶︎
Schema Registry Concepts
Schema Entities
Compatibility Policies
▶︎
Streams Replication Manager Overview
Overview
Key Features
Main Use Cases
▶︎
Use Case Architectures
▶︎
Highly Available Kafka Architectures
Active / Stand-by Architecture
Active / Active Architecture
Cross Data Center Replication
▶︎
Cluster Migration Architectures
On-premise to Cloud and Kafka Version Upgrade
Aggregation for Analytics
▶︎
Understanding Replication Flows
Replication Flows Overview
Remote Topics
Bi-directional Replication Flows
Fan-in and Fan-out Replication Flows
▼
Installation & Upgrade
▶︎
Support Matrix
Review System Requirements
▼
Deployment
▼
CSP Deployment Overview
▶︎
Prepare your environment
Install the JDK
▶︎
Installing Databases
Installing MySQL
Configuring Metadata Stores in MySQL
Install Postgres
Configure Postgres to Allow Remote Connections
Configure Metadata Stores in Postgres
▶︎
Prepare your cluster
Install or upgrade Cloudera Manager
Install CDH
Configure Apache Kafka for SMM
▶︎
Secure your cluster
Enable Kerberos on your cluster
Install Sentry
Enable Auto-TLS for your cluster
Set up a secure Apache Kafka cluster
Install the Parcel and CSD files
Add and Configure Schema Registry
▶︎
Add and Configure SRM
Add and Configure SRM
Enable Kerberos on SRM
Enable TLS/SSL for SRM
▼
Add and Configure SMM
Configuring Cloudera Manager and Service Monitor
Obtain the Kafka service name
Add and configure the SMM Node
Enable Kerberos on SMM
▶︎
Enable TLS for SMM
TLS/SSL settings for Streams Messaging Manager
Enabling Sentry Authorization for SMM
Enable partition level metric collection
Configuring SMM to Monitor SRM Replications
▶︎
How To
▶︎
Schema Registry
▶︎
Integrating with Schema Registry
▶︎
Integrating with NiFi
Understanding NiFi Record Based Processing
Setting up the HortonworksSchemaRegistry Controller Service
Adding and Configuring Record Reader and Writer Controller Services
Using Record-Enabled Processors
▶︎
Integrating with Kafka
Integrating Kafka and Schema Registry Using NiFi Processors
Integrating Kafka and Schema Registry
▶︎
Using Schema Registry
Adding a new schema
Querying a schema
Evolving a schema
Deleting a schema
▶︎
Streams Replication Manager
▶︎
Configuring SRM
Configuring Clusters and Replications
Configuring the Driver Role Target Clusters
Configuring the Service Role Target Cluster
Configuring Properties Not Exposed in Cloudera Manager
New Topic and Consumer Group Discovery
▶︎
Configuration Examples
Bidirectional Replication of Active Clusters
Cross Data Center Replication of Multiple Clusters
Configuration Properties Reference
Configuration Properties Reference for Properties Available in Cloudera Manager
▶︎
Using SRM
▶︎
SRM Command Line Tools
▶︎
srm-control
Topics and Groups Subcommand
Offsets Subcommand
Options reference
Monitoring Replication with Streams Messaging Manager
Replicating Data
▶︎
How to Set up Failover and Failback
Configure SRM for Failover and Failback
Migrating Consumer Groups Between Clusters
▶︎
Streams Messaging Manager
▶︎
Monitoring Kafka Clusters
Monitoring Clusters
Monitoring Producers
Monitoring Topics
Monitoring Brokers
Monitoring Consumers
▶︎
Managing Alert Policies
Alert Policies Overview
Component Types and Metrics for Alert Policies
Notifiers
▶︎
Managing Alert Policies and Notifiers
Creating a Notifier
Updating a Notifier
Deleting a Notifier
Creating an Alert Policy
Updating an Alert Policy
Enabling an Alert Policy
Disabling an Alert Policy
Deleting an Alert Policy
▶︎
Managing Topics
Creating a Kafka Topic
Modify a Kafka Topic
Deleting a Kafka Topic
▶︎
Monitoring End to End Latency
End to End Latency Overview
Granularity of Metrics
Enabling Interceptors
Monitoring End-to-end Latency
End to End Latency Use Cases
▶︎
Monitoring Kafka Cluster Replications
Monitoring Cluster Replications Overview
▶︎
Viewing Replication Details
Searching Cluster Replications by Source
Monitoring Cluster Replications by Quick Ranges
Monitoring Status of the Clusters to be Replicated
▶︎
Monitoring Topics to be Replicated
Searching by Topic Name
Monitoring Throughput for Cluster Replication
Monitoring Replication Latency for Cluster Replication
Monitoring Checkpoint Latency for Cluster Replication
Monitoring Throughput and Latency by Values
▶︎
Apache Kafka
Setting up Kafka
Integrating with Kafka
Administering Kafka
Kafka Performance Tuning
Using Kafka Streams
▶︎
Reference
Kafka Reference
Kafka Public APIs
SMM REST API Reference
SRM REST API Reference
Active / Active Architecture
Active / Stand-by Architecture
Add and Configure Schema Registry
Add and Configure SMM
Add and Configure SRM
Add and Configure SRM
Add and configure the SMM Node
Adding a new schema
Adding and Configuring Record Reader and Writer Controller Services
Aggregation for Analytics
Alert Policies Overview
Apache Kafka
Bi-directional Replication Flows
Bidirectional Replication of Active Clusters
Cloudera Stream Processing
Cloudera Stream Processing Overview
Cluster Migration Architectures
Compatibility Policies
Component support
Component Types and Metrics for Alert Policies
Configuration Examples
Configuration Properties Reference
Configuration Properties Reference for Properties Available in Cloudera Manager
Configure Apache Kafka for SMM
Configure Metadata Stores in Postgres
Configure Postgres to Allow Remote Connections
Configure SRM for Failover and Failback
Configuring Cloudera Manager and Service Monitor
Configuring Clusters and Replications
Configuring Metadata Stores in MySQL
Configuring Properties Not Exposed in Cloudera Manager
Configuring SMM to Monitor SRM Replications
Configuring SRM
Configuring the Driver Role Target Clusters
Configuring the Service Role Target Cluster
Creating a Kafka Topic
Creating a Notifier
Creating an Alert Policy
Cross Data Center Replication
Cross Data Center Replication of Multiple Clusters
CSP Architecture
CSP Deployment Overview
Deleting a Kafka Topic
Deleting a Notifier
Deleting a schema
Deleting an Alert Policy
Deployment
Disabling an Alert Policy
Download locations
Enable Auto-TLS for your cluster
Enable Kerberos on SMM
Enable Kerberos on SRM
Enable Kerberos on your cluster
Enable partition level metric collection
Enable TLS for SMM
Enable TLS/SSL for SRM
Enabling an Alert Policy
Enabling Interceptors
Enabling Sentry Authorization for SMM
End to End Latency Overview
End to End Latency Use Cases
Evolving a schema
Examples of Interacting with Schema Registry
Fan-in and Fan-out Replication Flows
Granularity of Metrics
Highly Available Kafka Architectures
How to Set up Failover and Failback
Install CDH
Install or upgrade Cloudera Manager
Install Postgres
Install Sentry
Install the JDK
Install the Parcel and CSD files
Installing Databases
Installing MySQL
Integrating Kafka and Schema Registry
Integrating Kafka and Schema Registry Using NiFi Processors
Integrating with Kafka
Integrating with NiFi
Integrating with Schema Registry
Key Features
Known issues
Main Use Cases
Managing Alert Policies
Managing Alert Policies and Notifiers
Managing Topics
Migrating Consumer Groups Between Clusters
Modify a Kafka Topic
Monitoring Brokers
Monitoring Checkpoint Latency for Cluster Replication
Monitoring Cluster Replications by Quick Ranges
Monitoring Cluster Replications Overview
Monitoring Clusters
Monitoring Consumers
Monitoring End to End Latency
Monitoring End-to-end Latency
Monitoring Kafka Cluster Replications
Monitoring Kafka Clusters
Monitoring Producers
Monitoring Replication Latency for Cluster Replication
Monitoring Replication with Streams Messaging Manager
Monitoring Status of the Clusters to be Replicated
Monitoring Throughput and Latency by Values
Monitoring Throughput for Cluster Replication
Monitoring Topics
Monitoring Topics to be Replicated
New Topic and Consumer Group Discovery
Notifiers
Obtain the Kafka service name
Offsets Subcommand
On-premise to Cloud and Kafka Version Upgrade
Options reference
Overview
Prepare your cluster
Prepare your environment
Querying a schema
Release Notes
Remote Topics
Replicating Data
Replication Flows Overview
Review System Requirements
Schema Entities
Schema Registry
Schema Registry Component Architecture
Schema Registry Concepts
Schema Registry Overview
Schema Registry Overview
Schema Registry Use Cases
Searching by Topic Name
Searching Cluster Replications by Source
Secure your cluster
Set up a secure Apache Kafka cluster
Setting up the HortonworksSchemaRegistry Controller Service
SRM Command Line Tools
srm-control
Streams Messaging Manager
Streams Messaging Manager Overview
Streams Messaging Manager Overview
Streams Replication Manager
Streams Replication Manager Overview
Support Matrix
TLS/SSL settings for Streams Messaging Manager
Topics and Groups Subcommand
Understanding NiFi Record Based Processing
Understanding Replication Flows
Updating a Notifier
Updating an Alert Policy
Use Case 1: Registering and Querying a Schema for a Kafka Topic
Use Case 2: Reading/Deserializing and Writing/Serializing Data from and to a Kafka Topic
Use Case 3: Dataflow Management with Schema-based Routing
Use Case Architectures
Using Record-Enabled Processors
Using Schema Registry
Using SRM
Viewing Replication Details
What's new in Cloudera Stream Processing
«
Filter topics
Add and Configure SMM
▼
CSP Deployment Overview
▶︎
Prepare your environment
Install the JDK
▶︎
Installing Databases
Installing MySQL
Configuring Metadata Stores in MySQL
Install Postgres
Configure Postgres to Allow Remote Connections
Configure Metadata Stores in Postgres
▶︎
Prepare your cluster
Install or upgrade Cloudera Manager
Install CDH
Configure Apache Kafka for SMM
▶︎
Secure your cluster
Enable Kerberos on your cluster
Install Sentry
Enable Auto-TLS for your cluster
Set up a secure Apache Kafka cluster
Install the Parcel and CSD files
Add and Configure Schema Registry
▶︎
Add and Configure SRM
Add and Configure SRM
Enable Kerberos on SRM
Enable TLS/SSL for SRM
▼
Add and Configure SMM
Configuring Cloudera Manager and Service Monitor
Obtain the Kafka service name
Add and configure the SMM Node
Enable Kerberos on SMM
▶︎
Enable TLS for SMM
TLS/SSL settings for Streams Messaging Manager
Enabling Sentry Authorization for SMM
Enable partition level metric collection
Configuring SMM to Monitor SRM Replications
»
Deployment
Add and Configure SMM
Configuring Cloudera Manager and Service Monitor
SMM requires high levels of Cloudera Manager and Service Monitor Service memory usage, and is important to configure them appropriately. For details see
Cloudera Manager tuning documentation
and
Service Monitor tuning documentation
.
Obtain the Kafka service name
You will need the Kafka service name when you are configuring SMM. It is helpful to obtain it in advance.
Add and configure the SMM Node
There are a number of required configurations you must make before you complete the SMM installation.
Enable Kerberos on SMM
Enable TLS for SMM
Learn how to enable TLS/SSL encryption for Streams Messaging Manager (SMM). You can enable the settings in Cloudera Manager according to the cluster configuration.
Enabling Sentry Authorization for SMM
You can use Sentry to configure and manage authorization for SMM.
Enable partition level metric collection
You must enable SMM to configure partition level metrics about Kafka topics.
Configuring SMM to Monitor SRM Replications
If you plan to use SMM to monitor Kafka cluster replication, ensure that you have configured SMM to communicate with SRM.
Parent topic:
CSP Deployment Overview
This site uses cookies and related technologies, as described in our
privacy policy
, for purposes that may include site operation, analytics, enhanced user experience, or advertising. You may choose to consent to our use of these technologies, or
manage your own preferences.
Accept all
2.0
2.0.1
2.0.0
1.0.0
