Overview

Get familiar with Streams Replication Manager and its components.

Streams Replication Manager is an enterprise-grade replication solution that enables fault tolerant, scalable and robust cross-cluster Kafka topic replication. Streams Replication Manager provides the ability to dynamically change configurations and keeps the topic properties in sync across clusters at high performance. Streams Replication Manager also delivers custom extensions that facilitate installation, management and monitoring making Streams Replication Manager a complete replication solution that is built for mission critical workloads. Streams Replication Manager consists of two main components. The Stream Replication Engine and the Stream Replication Management Services.

Figure 1. Streams Replication Manager Overview

Stream Replication Engine

The Stream Replication Engine is a next generation multi-cluster and cross-datacenter replication engine for Kafka clusters.

Stream Replication Management Services

Stream Replication Management Services are services powered by open source Cloudera extensions which utilize the capabilities of the Stream Replication Engine. These services provide:

Easy installation
Lifecycle management
Management and monitoring of replication flows across clusters

The Stream Replication Management Services includes the following custom extensions:

Cloudera Streams Replication Manager Driver: The Cloudera Streams Replication Manager Driver is a small wrapper around the Stream Replication Engine that adds the extensions provided by Cloudera. It provides the ability to spin up Streams Replication Manager clusters and has a metrics reporter. The driver is managed by Cloudera Manager and is represented by the Streams Replication Manager Driver role.
Cloudera Streams Replication Manager Client: The Cloudera Streams Replication Manager Client provides users with command line tools that enable replication management for topics and consumer groups. The command line tool associated with the Cloudera Streams Replication Manager Client is called srm-control.
Cloudera Streams Replication Manager Service: The Cloudera Streams Replication Manager Service consist of a REST API and a Kafka Streams application to aggregate and expose cluster, topic and consumer group metrics. The service is managed by Cloudera Manager and is represented by the Streams Replication Manager Service role.