Defining co-located Kafka clusters using Kafka credentials
Before you can start replicating data with SRM, you must define the co-located Kafka clusters that take part in the replication process. This can be done using Cloudera Manager by creating Kafka credentials.
The following list of steps walk you through how you can define the co-located Kafka cluster using Kafka credentials.
-
Ensure that you have reviewed Defining and adding clusters for replication and understand that the following list of steps are only one part of the full configuration workflow.
- In Cloudera Manager, go to Administration > > External Accounts.
- Go to the Kafka Credentials tab.
The Kafka Credentials tab enables you to create, configure, and manage Kafka credentials. On this tab you will create a credential for your co-located Kafka cluster.
- Create and configure Kafka credentials for the external clusters:
- Click Add Kafka credentials.
- Configure the Kafka credential:The security configuration of the co-located cluster determines which of the available properties you must set. To see which exact properties are required for your scenario, click one of the following tabs matching the security protocol of the cluster you are defining:
- Name
- Bootstrap servers
- Security protocol
- Encryption only
-
- Name
- Bootstrap servers
- Security protocol
- Truststore Password
- Truststore Path
- Truststore Type
- Encryption and authentication
-
- Name
- Bootstrap servers
- Security protocol
- Truststore Password
- Truststore Path
- Truststore Type
- Key Password
- Keystore Password
- Keystore Path
- Keystore Type
- Name
- Bootstrap servers
- Security protocol
- JAAS Secret [1-3]
- JAAS Template
- Kerberos Service Name
- SASL Mechanism
- Name
- Bootstrap servers
- Security protocol
- Truststore Password
- Truststore Path
- Truststore Type
- JAAS Secret [1-3]
- JAAS Template
- Kerberos Service Name
- SASL Mechanism
- Click Add.If credential creation is successful, a new entry corresponding to the Kafka credential you specified appears on the page.
The following tabs collect examples of different Kafka credentials as well as some notes regarding configuration. Review these examples to better understand how to correctly configure a Kafka credential.-
name=uswest Bootstrap servers=uswest-node1.cluster.com:9092,uswest-node2.cluster.com:9092,uswest-node3.cluster.com:9092 Security protocol=PLAINTEXT
- Encryption only
-
Name=uswest Bootstrap servers=uswest-node1.cluster.com:9093,uswest-node2.cluster.com:9093,uswest-node3.cluster.com:9093 Security protocol=SSL Truststore Password=password Truststore Path=/opt/srm/us-west-truststore.jks Truststore Type=JKS
- Encryption and authentication
-
Name=uswest Bootstrap servers=uswest-node1.cluster.com:9093,uswest-node2.cluster.com:9093,uswest-node3.cluster.com:9093 Security protocol=SSL Truststore Password=password Truststore Path=/opt/srm/us-west-truststore.jks Truststore Type=JKS Keystore Password=password Keystore Path=/opt/srm/us-west-keystore.jks Keystore Type=JKS Key Password=password
- Kerberos
-
Name=uswest Bootstrap servers=uswest-node1.cluster.com:9092,uswest-node2.cluster.com:9092,uswest-node3.cluster.com:9092 Security protocol=SASL_PLAINTEXT JAAS Template=com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/opt/srm/streamsrepmgr.keytab" principal="streamsrepmgr@REALM.COM"; Kerberos Service Name=kafka SASL Mechanism=GSSAPI
- PLAIN
-
Name=uswest Bootstrap servers=uswest-node1.cluster.com:9092,uswest-node2.cluster.com:9092,uswest-node3.cluster.com:9092 Security protocol=SASL_PLAINTEXT JAAS Secret 1=password JAAS Template=org.apache.kafka.common.security.plain.PlainLoginModule required username="username" password="##JAAS_SECRET_1##"; SASL Mechanism=PLAIN
- Kerberos
-
Name=uswest Bootstrap servers=uswest-node1.cluster.com:9093,uswest-node2.cluster.com:9093,uswest-node3.cluster.com:9093 Security protocol=SASL_SSL Truststore Password=password Truststore Path=/opt/srm/us-west-truststore.jks Truststore Type=JKS JAAS Template=com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/opt/srm/streamsrepmgr.keytab" principal="streamsrepmgr@REALM.COM"; Kerberos Service Name=kafka SASL Mechanism=GSSAPI
- PLAIN
-
Name=uswest Bootstrap servers=uswest-node1.cluster.com:9093,uswest-node2.cluster.com:9093,uswest-node3.cluster.com:9093 Security protocol=SASL_SSL Truststore Password=password Truststore Path=/opt/srm/us-west-truststore.jks Truststore Type=JKS JAAS Secret 1=password JAAS Template=org.apache.kafka.common.security.plain.PlainLoginModule required username="username" password="##JAAS_SECRET_1##"; SASL Mechanism=PLAIN