Enable SSL for Kafka Clients
Kafka allows clients to connect over SSL. By default SSL is disabled, but it can be enabled as needed.
Before you begin, be sure to generate the key, SSL certificate, keystore, and truststore that will be used by Kafka.
Configuring the Kafka Broker
The Kafka Broker supports listening on multiple ports and IP addresses. To enable
this feature, specify one or more comma-separated values in the
listeners
property in server.properties
.
Both PLAINTEXT and SSL ports are required if SSL is not enabled for inter-broker communication (see the following subsection for information about enabling inter-broker communication):
listeners=PLAINTEXT://host.name:port,SSL://host.name:port
The following SSL configuration settings are needed on the broker side:
ssl.keystore.location = /var/private/ssl/kafka.server.keystore.jks ssl.keystore.password = test1234 ssl.key.password = test1234 ssl.truststore.location = /var/private/ssl/kafka.server.truststore.jks ssl.truststore.password = test1234
The following optional settings are available:
Property | Description | Value(s) |
---|---|---|
ssl.client.auth |
Specify whether client authentication is required, requested, or not required.
Note: If you set
| none |
ssl.cipher.suites | Specify one or more cipher suites: named combinations of authentication, encryption, MAC and key exchange algorithms used to negotiate the security settings for a network connection using the TLS or SSL network protocol. | |
ssl.enabled.protocols | Specify the SSL protocols that you will accept from clients. Note: SSL is deprecated; its use in production is not recommended. | TLSv1.2,TLSv1.1,TLSv1 |
ssl.keystore.type | Specify the SSL keystore type. | JKS |
ssl.truststore.type | Specify the SSL truststore type. | JKS |
Enabling SSL for Inter-Broker Communication
To enable SSL for inter-broker communication, add the following setting to the broker properties file (default is PLAINTEXT):
security.inter.broker.protocol = SSL
Enabling Additional Cipher Suites
To enable any cipher suites other than the defaults that come with JVM (see Java Cryptography documentation), you will need to install JCE Unlimited Strength Policy files (download link).
Validating the Configuration
After you start the broker, you should see the following information in the
server.log
file:
with addresses: PLAINTEXT -> EndPoint(192.168.64.1,9092,PLAINTEXT),SSL -> EndPoint(192.168.64.1,9093,SSL)
To make sure that the server keystore and truststore are set up properly, run the following command:
openssl s_client -debug -connect localhost:9093 -tls1
(Note: TLSv1 should be listed under ssl.enabled.protocols
)
In the openssl
output you should see the
server certificate; for example:
Server certificate -----BEGIN CERTIFICATE----- MIID+DCCAuACCQCx2Rz1tXx3NTANBgkqhkiG9w0BAQsFADB6MQswCQYDVQQGEwJV UzELMAkGA1UECAwCQ0ExFDASBgNVBAcMC1NhbnRhIENsYXJhMQwwCgYDVQQKDANv cmcxDDAKBgNVBAsMA29yZzEOMAwGA1UEAwwFa2FmYWsxHDAaBgkqhkiG9w0BCQEW DXRlc3RAdGVzdC5jb20wHhcNMTUwNzMwMDQyOTMwWhcNMTYwNzI5MDQyOTMwWjBt MQswCQYDVQQGEwJVUzELMAkGA1UECBMCQ0ExFDASBgNVBAcTC1NhbnRhIENsYXJh MQwwCgYDVQQKEwNvcmcxDDAKBgNVBAsTA29yZzEfMB0GA1UEAxMWU3JpaGFyc2hh IENoaW50YWxhcGFuaTCCAbcwggEsBgcqhkjOOAQBMIIBHwKBgQD9f1OBHXUSKVLf Spwu7OTn9hG3UjzvRADDHj+AtlEmaUVdQCJR+1k9jVj6v8X1ujD2y5tVbNeBO4Ad NG/yZmC3a5lQpaSfn+gEexAiwk+7qdf+t8Yb+DtX58aophUPBPuD9tPFHsMCNVQT WhaRMvZ1864rYdcq7/IiAxmd0UgBxwIVAJdgUI8VIwvMspK5gqLrhAvwWBz1AoGB APfhoIXWmz3ey7yrXDa4V7l5lK+7+jrqgvlXTAs9B4JnUVlXjrrUWU/mcQcQgYC0 SRZxI+hMKBYTt88JMozIpuE8FnqLVHyNKOCjrh4rs6Z1kW6jfwv6ITVi8ftiegEk O8yk8b6oUZCJqIPf4VrlnwaSi2ZegHtVJWQBTDv+z0kqA4GEAAKBgB+Pdz0306bq TpUAdb2FERMPLFsx06H0x+TULivcp7HbS5yrkV9bXZmv/FD98x76QxXrOq1WpQhY YDeGDjH+XQkJ6ZxBVBZNJDIpCnfQpfzXAvryQ+cm8oXUsKidtHf4pLMYViXX6BWX Oc2hX4rG+lC8/NXW+1zVvCr9To9fngzjMA0GCSqGSIb3DQEBCwUAA4IBAQBfyVse RJ+ugiNlWg5trZscqH0tlocbnek4UuV/xis2eAu9l4EFOM5kRt5GmkGZRcM/zHF8 BRJwXbf0fytMQKSPFk8R4/NGDOlzoK+F7uXeJ0S2u/T29xk0u2i4tjvleq6OCphE i9vdjM0E0Whf9SHRhOXirOYFX3cL775XwKdzKKRkk+AszFR+mRu90rdoaepQtgGh 9Kfwr4+6AU/dPtdGuomtBQqMxCzlrLd8EYhVVQ97wHIZ3sPvlM5PIhOJ/YHSBJIC 75eo/4acDxZ+j3sR5kcFulzYwFLgDYBaKH/w3mYCgTALeB1zUkX53NVizIvhUd69 XJO4lDSDtGOlfort -----END CERTIFICATE----- subject=/C=US/ST=CA/L=Santa Clara/O=org/OU=org/CN=JBrown issuer=/C=US/ST=CA/L=Santa Clara/O=org/OU=org/CN=kafak/emailAddress=test@test.com
If the certificate does not display, or if there are any other error messages, then your keystore is not set up properly.
Configuring Kafka Producer and Kafka Consumer
SSL is supported for new Kafka Producers and Consumer processes; the older API is not supported. Configuration settings for SSL are the same for producers and consumers.
If client authentication is not needed in the broker, then the following is a minimal configuration example:
security.protocol = SSL ssl.truststore.location = /var/private/ssl/kafka.client.truststore.jks ssl.truststore.password = test1234
If client authentication is required, first create a keystore (described earlier in this chapter). Next, specify the following settings:
ssl.keystore.location = /var/private/ssl/kafka.client.keystore.jks ssl.keystore.password = test1234 ssl.key.password = test1234
One or more of the following optional settings might also be needed, depending on your requirements and the broker configuration:
Property | Description | Value(s) |
---|---|---|
ssl.provider
| The name of the security provider used for SSL connections. Default value is the default security provider of the JVM. | |
ssl.cipher.suites | Specify one or more cipher suites: named combinations of authentication, encryption, MAC and key exchange algorithms used to negotiate the security settings for a network connection using the TLS or SSL network protocol. | |
ssl.enabled.protocols | List at least one of the protocols configured on the broker side. | TLSv1.2,TLSv1.1,TLSv1
|
ssl.keystore.type | Specify the SSL keystore type. | JKS |
ssl.truststore.type | Specify the SSL truststore type. | JKS |
The following two examples launch console-producer and console-consumer processes:
kafka-console-producer.sh --broker-list localhost:9093 --topic test --producer.config client-ssl.properties kafka-console-consumer.sh --bootstrap-server localhost:9093 --topic test --new-consumer --consumer.config client-ssl.properties