Kafka Connect Secrets Storage

Learn about the Kafka Connect Secrets Storage feature which can be used to hide and securely store sensitive data found in connector configurations.

In most production environments, the Kafka Connect connectors that you deploy connect to and move data either from or into secured services. Because of this, connectors that you deploy might require various credentials to access services. Credentials like passwords, access keys, or any other sensitive information must be provided to the connectors in their configuration. By default, the connector configurations can be easily read, their contents are unencrypted. Additionally, if you are using Streams Messaging Manager (SMM) to manage and deploy connectors, all sensitive information might be visible to anyone who has access to SMM with appropriate authorization rights.

Kafka Connect Secrets Storage is a feature within Kafka Connect that makes it possible for you to hide any sensitive values found in connector configurations and to store these values in a secure manner. For example, assume that you want to deploy a connector that connects to a secured service that requires a username and password for access. By default the password could be easily read by anyone with access to the connector configuration as the configuration is stored in plaintext. This poses a security risk. To circumvent this, you can mark the password as a secret. Once you mark the password as a secret, the Kafka Connect Secret Storage feature ensures that the password is stored and handled in a secure manner by doing the following:

The property is replaced by a reference in the configuration. The reference resolves to the actual password at runtime. That is, the value is no longer readable if the configuration is accessed.
The actual value of the password is encrypted and stored in an internal Kafka topic used by the feature.
The value gets hidden (locked) on the SMM UI.