Streams Replication Manager Service

The Streams Replication Manager Service is responsible for processing the metrics produced by the Streams Replication Manager Drivers. Additionally, it provides a queryable REST API that you can use to monitor and track replications.

Streams Replication Manager Driver roles produce raw metrics into the target Kafka clusters. Each replication has a separate raw metric topic. Streams Replication Manager Service roles run a Kafka Streams application internally, which aggregates the raw metrics. The Streams application ensures that the metrics processing work is load balanced between the Streams Replication Manager Service instances of the same cluster, and that the members of the cluster can coordinate when serving REST API queries.

As a result of this architecture, the Streams Replication Manager Service can only report on replication flows targeting the Kafka cluster which is targeted by the Streams Replication Manager Service.

Multi cluster targeting

The Streams Replication Manager Service is capable of targeting multiple Kafka clusters. In this case, a Streams Replication Manager Service instance spins up a Kafka Streams application for each target Kafka cluster. This way, the Streams Replication Manager Service is able to report on all replication flows targeting any of the configured target clusters.

Remote Querying

A Streams Replication Manager Service cluster can be configured to target and gather metrics from multiple clusters (multi cluster targeting). However, a setup like this can result in heavily loaded Service roles, which might not be suitable for your deployment. To circumvent this issue, the Streams Replication Manager Service is capable of connecting to and communicating with other, remote Streams Replication Manager Services to fetch their metrics. This is called Remote Querying. Using Remote Querying makes it possible to designate a Streams Replication Manager Service cluster in your deployment to act as a monitoring gateway. The designated Streams Replication Manager Service cluster then can be used to monitor all clusters and replications in your deployment. For more information, see Remote Querying.

Using the Streams Replication Manager Service REST API

The Streams Replication Manager Service offers a Swagger UI for exploring and querying the REST API. For a user-friendly UI solution, Streams Messaging Manager can be integrated with Streams Replication Manager. When Streams Messaging Manager is configured to connect to Streams Replication Manager, the Replications page becomes available on the Streams Messaging Manager UI. This page displays the information available on the Streams Replication Manager Service REST API. For more information on how you can integrate the two services, see Integrating Streams Replication Manager with Streams Messaging Manager.