Enabling Remote Querying

Remote Querying can be enabled for a cluster of SRM Service roles (SRM Service cluster). When this feature is enabled, the configured SRM Service cluster acts as a monitoring gateway and fetches metrics from other, remote SRM Service clusters. This enables you to monitor the replications of your whole deployment using a single SRM Service cluster.

You enable Remote Querying for the SRM Service clusters that you want to designate as monitoring gateways. Once enabled, the designated SRM Service cluster discovers and communicates with other, remote SRM Service clusters and fetches the metrics collected by the remote SRM Service clusters. After configuration is complete, you can use the designated SRM Service cluster to monitor all replications in your deployment.

Configuration is done in Cloudera Manager by providing a list of external Kafka cluster aliases in the Streams Replication Manager Service Remote Target Clusters property. These clusters must be defined in Cloudera Manager using Kafka credentials.

  • If any of the SRM installations are located behind a proxy, complete Configuring the advertised information of the SRM Service role
  • Define all external Kafka clusters that you want to monitor.
    • External clusters are defined by creating and configuring Kafka credentials. Kafka credentials are created and configured in Cloudera Manager on the Administration > External Accounts > Kafka Credentials page. For more information, see Defining external Kafka clusters.
    • Ensure that the Name property of each Kafka credential you create for the external Kafka clusters matches the alias set for that Kafka cluster in the configuration of that Kafka cluster’s co-located SRM instance. The alias can be found in the Streams Replication Manager Co-located Kafka Cluster Alias property.
    • Note down the Name property of each Kafka credential you create for the external Kafka clusters. You need to provide these during configuration.
  • Ensure that network connectivity and DNS resolution are established between the clusters in your deployment. Additionally, the SRM Service’s HTTPS port (default 6671) must have both inbound and outbound traffic enabled.
  1. In Cloudera Manager, select the Streams Replication Manager service.
  2. Go to Configuration.
  3. Find and configure the Streams Replication Manager Service Remote Target Clusters property.
    Add a comma separated list of the external Kafka cluster aliases that you want to monitor. For example:
    remote_1, remote_2
  4. Click Save Changes.
  5. Restart the Streams Replication Manager service.

Remote Querying is enabled and configured for the selected SRM Service cluster. The SRM Service cluster discovers and then communicates with other, remote SRM Service clusters and fetches metrics available on the remote SRM Service clusters.

  • If any of the remote SRM Services have Basic Authentication enabled, complete Configuring Basic Authentication for Remote Querying.
  • Access the Replications page on the SMM UI. Remote replications will be visible in the UI.
  • Query metrics using the SRM REST API. For example:
    1. Go to Streams Replication Manager > Web UI > SRM Service Swagger UI.
    2. Find and open the /v2/replications endpoint.
    3. Click Try it out then click Execute.

      The response includes all discovered replications, replicated topics, and various other metrics. This includes replications that target remote clusters.