Configuring Basic Authentication for SRM Services that have Remote Querying enabled

If a Streams Replication Manager (SRM) Service has Remote Querying enabled, it will connect to other, remote SRM Services to fetch the remote cluster replication metrics. If the remote SRM Service has Basic Authentication (BA) enabled, the SRM Service that has Remote Querying enabled must be provided with valid credentials that it can use to connect to the remote SRM Service.

Remote Querying is a feature of the SRM Service that enables you to configure an SRM Service in your deployment to connect to other, remote SRM Services and fetch the remote cluster replication metrics gathered by the remote Service. This allows you to monitor the replications of a whole deployment that has multiple installations of SRM through a single SRM and Streams Messaging Manager service. For more information on Remote Querying, see Remote Querying.

If the remote SRM Services are configured to use BA, the SRM Services that have Remote Querying enabled (monitoring gateways) require valid credentials that can be used to access the remote SRM Services. As a result, the credentials required to access the remote SRM Services must be provided to the monitoring gateways.

Configuration is done in Cloudera Manager by creating Streams Replication Manager Service Credentials and then configuring the monitoring gateways to use these credentials for Remote Querying.

  • Basic authentication is set up and configured for the remote SRM Services. For more information on how to enable Basic authentication see, Enabling Basic Authentication for the SRM Service.
  • Remote querying is enabled and configured. For more information see, Enabling Remote Querying
  • Ensure that you have access to the BA usernames and passwords used by the remote SRM Services. You will need to provide these during configuration. These usernames and passwords were configured using Basic Authentication Credentials when you enabled BA for the remote SRM Services.
  1. Access the Cloudera Manager instance managing the SRM Service that has Remote Querying enabled (monitoring gateway).
  2. Create Streams Replication Manager Credentials:
    A Streams Replication Manager Credential represents a remote SRM Service that has Basic Authentication enabled. The credential stores the information necessary to establish a connection with the remote SRM Service. Create a credential for each BA enabled remote SRM Service.
    1. Go to Administration > External Accounts > Streams Replication Manager Credentials.
    2. Click Add Streams Replication Manager Service Credentials.
    3. Configure the following properties:
      • Name

        Add a unique and easily identifiable name. Note down the name you configure, you will need to provide it in a later step.

      • Remote Streams Replication Manager Service target Kafka cluster alias

        This is the alias of the Kafka cluster that the remote SRM Service (which this credential represents) is targeting. The alias that you add here must be one of the aliases that you added to the Streams Replication Manager Service Remote Target Clusters property when you enabled Remote Querying.

      • Username and Password

        Add the username and password pair that can be used to access the remote SRM Service that this credential represents. Both the username and password that you add here must be valid. That is, in the Cloudera Manager instance managing the remote SRM Service, a Basic Authentication credential must exist with this username and password combination. Additionally, the remote SRM Service must be configured to use these credentials.

      To better understand how Streams Replication Manager Service Credentials should be configured, review the following two cluster example:

      In this specific example, a single Streams Replication Manager Service Credential must be created in Cloudera Manager A. This credential represents SRM Service B and will be used by SRM Service A to access SRM Service B. The credential would be configured as follows:
      Name: srm_b
      Remote Streams Replication Manager Service target Kafka cluster alias: kafka_b
      Username: admin
      Password: password
      
  3. Configure this SRM Service (monitoring gateway) to use the credentials when querying the remote SRM Service:
    1. Go to Clusters and select the Streams Replication Manager Service.
    2. Go to Configuration.
    3. Find and Configure the External Accounts For Remote Querying SRM Services property.
    4. Add the names of all Streams Replication Manager Service Credentials you created in Step 1. For example:
      remote_1
      remote_2
      remote_3
  4. Click Save Changes.
  5. Restart Streams Replication Manager.
The credentials required to access the remote SRM Services are set up and configured. When this SRM Service establishes a connection with the configured remote SRM Services, it uses the configured credentials.
  • Access the Replications page on the SMM UI. Remote replications will be visible in the UI.
  • Query metrics using the SRM REST API. For example:
    1. Go to Streams Replication Manager > Web UI > SRM Service Swagger UI.
    2. Find and open the /v2/replications endpoint.
    3. Click Try it out then click Execute.

      The response includes all discovered replications, replicated topics, and various other metrics. This includes replications that target remote clusters.