Replicating data from CDP PvC Base cluster to Data Hub cluster with SRM running in CDP PvC Base cluster

You can set up and configure an instance of SRM running in a CDP PvC Base cluster to replicate data between the CDP PvC Base cluster and a Data Hub cluster. In addition, you can use SMM to monitor the replication process. Review the following example to learn how this can be set up.

Consider the following replication scenario:

In this scenario, data is replicated from a CDP PvC Base cluster that has Kafka, SRM, and SMM deployed on it. This is a secure cluster that has TLS/SSL encryption and Kerberos authentication enabled. In addition, it uses Ranger for authorization.

Data is being replicated from this cluster by SRM deployed in this cluster to a Data Hub cluster.

The Data Hub cluster is provisioned with the one of the default Streams Messaging cluster definitions.

This example scenario does not go into detail on how to set up the clusters and assumes the following:

  • A Data Hub cluster provisioned with the Streams Messaging Light Duty or Heavy Duty cluster definition is available.

    For more information, see Creating your first Streams Messaging cluster in the CDF for Data Hub library. Alternatively, you can also review the cloud provider specific cluster creation instructions available in the Cloudera Data Hub library.

  • A CDP PvC Base cluster with Kafka, SRM, and SMM is available. This cluster is TLS/SSL and Kerberos enabled. In addition, it uses Ranger for authorization.

    For more information, see the CDP Private Cloud Base Installation Guide.

  • Network connectivity and DNS resolution are established between the clusters.
  1. Create a machine user for SRM in Management Console:
    A machine user is required so that SRM has credentials that it can use to connect to the Kafka service in the Data Hub cluster.
    1. Navigate to Management Console > User Management.
    2. Click Actions > Create Machine User.
    3. Enter an unique name for the user and click Create.
      For example: srm
      After the user is created, you are presented with a page that displays the user details.
    4. Click Set Workload Password.
    5. Type a password in the Password and Confirm Password fields. Leave the Environment field blank.
    6. Click Set Workload Password.
      A message appears on successful password creation. .
  2. Grant the machine user access to your environment:
    You must grant the machine user access to your environment for SRM to connect to the Kafka service with this user.
    1. Navigate to Management Console > Environments, and select the environment where your Kafka cluster is located.
    2. Click Actions > Manage Access.
      Use the search box to find and select the machine user you want to use.
      A list of Resource Roles appears.
    3. Select the EnvironmentUser role and click Update Roles.
    4. Go back to the Environment Details page and click Actions > Synchronize Users to FreeIPA.
    5. On the Synchronize Users page, click Synchronize Users.
      Synchronizing users ensures that the role assignment is in effect for the environment.
  3. Add Ranger permissions for the SRM user in the Data Hub cluster:
    You must to grant the necessary privileges to the user so that the user can access Kafka resources. This is configured through Ranger policies.
    1. Navigate to Management Console > Environments, and select the environment where your Kafka cluster is located.
    2. Click the Ranger link on the Environment Details page.
    3. Select the resource-based service corresponding to the Kafka resource in the Data Hub cluster.
    4. Add the Workload User Name of the user you created for SRM to the following Ranger policies:
      • All - consumergroup
      • All - topic
      • All - transactionalid
      • All - cluster
      • All - delegationtoken
  4. Ensure that Ranger permissions exist for the streamsrepmgr user in the CDP PvC Base cluster:
    1. Access the Cloudera Manager instance of your CDP PvC Base cluster.
    2. Go to Ranger > Ranger Admin Web UI.
    3. Log in to the Ranger Console (Ranger Admin Web UI).
    4. Ensure that the streamsrepmgr user is added to all required policies.
      If the user is missing, add it. The required policies are as follows:
      • All - consumergroup
      • All - topic
      • All - transactionalid
      • All - cluster
      • All - delegationtoken
  5. Create a truststore on the CDP PvC Base cluster:
    A truststore is required so that the SRM instance running in the CDP PvC Base cluster can trust the secure Data Hub cluster. To do this, you extract the FreeIPA certificate from the CDP environment, create a truststore that includes the certificate, and copy the truststore to all hosts on the CDP PvC Base cluster.
    1. Navigate to Management Console > Environments, and select the environment where your Kafka cluster is located.
    2. Go to the Summary tab.
    3. Scroll down to the FreeIPA section.
    4. Click Actions > Get FreeIPA Certificate.
      The FreeIPA certificate file, [***ENVIRONMENT NAME***]-env.crt, is downloaded to your computer.
    5. Run the following command to create the truststore:
      keytool \
        -importcert \
        -storetype JKS \
        -noprompt \
        -keystore datahub-truststore.jks \
        -storepass [***PASSWORD***] \
        -alias freeipa-ca \
        -file [***PATH TO FREEIPA CERTIFICATE***]
    6. Copy the datahub-truststore.jks file to a common location on all the hosts in your CDP PvC Base cluster.
      Cloudera recommends that you use the following location: /opt/cloudera/security/datahub-truststore.jks.
    7. Set the correct file permissions.
      Use 751 for the directory and 444 for the truststore file.
  6. Configure the SRM properties in the CDP PvC Base cluster:
    1. Access the Cloudera Manager instance of your CDP PvC Base cluster.
    2. Go to Streams Replication Manager > Configuration and configure the following properties:
      • Streams Replication Manager Cluster alias: datahub, cdppvc
      • Streams Replication Manager Driver Target Cluster: datahub, cdppvc
      • Streams Replication Manager Service Target Cluster: datahub
      • Streams Replication Manager's Replication Configs:
        #Bootstrap servers:
        cdppvc.bootstrap.servers=[***MY-CDP-PVC-CLUSTER-HOST-1.COM:9093***],[***MY-CDP-PVC-CLUSTER-HOST-2:9093***]
        datahub.bootstrap.servers=[**MY-DATAHUB-CLUSTER-HOST-1.COM:9093***],[***MY-DATAHUB-CLUSTER-HOST-1.COM:9093***]
        
        #Replications:  
        cdppvc->datahub.enabled=true
        
        #Security properties for the Datahub cluster:
        datahub.security.protocol=SASL_SSL
        datahub.sasl.mechanism=PLAIN
        datahub.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="[***WORKLOAD USER NAME***]" password="[***MACHINE USER PASSWORD***]"; 
        datahub.ssl.truststore.location=/opt/cloudera/security/datahub-truststore.jks
        datahub.ssl.truststore.password=[***PASSWORD***] 
        
        #Use the FQDN when specifying the cluster hosts.
        #The terminating semicolon in the [***ALIAS***].sasl.jaas.config property must be included in the configuration.
        #The value of the [***ALIAS***].ssl.truststore.location is the location where you copied the truststore in a previous step.
        #The [***ALIAS***].ssl.truststore.password property must be specified. Otherwise, the configuration might get overriden by the service ssl.truststore.password property.
        
    3. Click Save.
    4. Restart SRM.
    5. Deploy client configuration for SRM.
  7. Start the replication process using the srm-control tool:
    1. SSH as an administrator to any of the SRM hosts in the CDP PvC cluster.
      ssh [***USER***]@[***MY-CDP-PVC-CLUSTER.COM***]
    2. Create a configuration file for the srm-control tool.

      The srm-control tool behaves as a Kafka client and requires configuration that is similar to any Kafka client. The configuration file is specified with the --config option when you run the tool. The configuration file must include cluster alias definitions, as well as properties related to connection information and security. Cluster aliases are defined a single time, connection and security properties are defined separately for each alias (cluster). In this example the file is named srm.properties.

      #Define aliases:
      clusters=datahub, cdppvc
      
      #Bootstrap servers:
      datahub.bootstrap.servers=[***MY-DATAHUB-CLUSTER-HOST-1.COM:9093***],[***MY-DATAHUB-CLUSTER-HOST-1.COM:9093***]
      cdppvc.bootstrap.servers=[***MY-CDP-PVC-CLUSTER-HOST-1.COM:9093***],[***MY-CDP-PVC-CLUSTER-HOST-2:9093***]
      
      #DataHub cluster's security properties:
      datahub.security.protocol=SASL_SSL
      datahub.sasl.mechanism=PLAIN
      datahub.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="[***WORKLOAD USER NAME***]" password="[***MACHINE USER PASSWORD***]"; 
      datahub.ssl.truststore.location=/opt/cloudera/security/datahub-truststore.jks
      datahub.ssl.truststore.password=[***PASSWORD***]
      
      #CDP PvC Base cluster's security properties:
      cdppvc.security.protocol=SASL_SSL
      cdppvc.sasl.mechanism=GSSAPI
      cdppvc.sasl.kerberos.service.name=kafka
      cdppvc.sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="[***PATH TO KEYTAB FILE***]" storeKey=true useTicketCache=false principal="[***MY KERBEROS PRINCIPAL****]";
      cdppvc.ssl.truststore.location=[***CDP PVC BASE GLOBAL TRUSTSTORE LOCATION***]
      cdppvc.ssl.truststore.password=[***CDP PVC BASE GLOBAL TRUSTSTORE PASSWORD***]
      
      #Use the FQDN when specifying the cluster hosts.
      #The terminating semicolon in the [***ALIAS***].sasl.jaas.config properties must be included in the configuration.
      #The value of the datahub.ssl.truststore.location property is the location where you copied the truststore in a previous step.
      
    3. Use the srm-control tool with the topics subcommand to add topics to the allow list:
      srm-control --config ./srm.properties topics --source cdppvc --target datahub --add [***TOPIC NAME***]
    4. Use the srm-control tool with the groups subcommand to add groups to the allow list:
      srm-control --config ./srm.properties groups --source cdppvc --target datahub --add ".*"
  8. Monitor replications.
    Access the SMM UI in the CDP PvC Base cluster and go to the Cluster Replications page. The replications you set up will be visible on this page.