Cloudera Docs

Channel encryption (TLS) in Cloudera Surveyor for Apache Kafka

TLS encryption in Cloudera Surveyor covers four distinct traffic types: internal, client, LDAP, and inter-instance cache. Configure each type independently using dedicated Helm chart properties, and manage the required certificates manually or with cert-manager.

You can protect sensitive data and secure all traffic between components by configuring TLS for Cloudera Surveyor. You can configure TLS for the following types of traffic.

  • TLS for internal (backend) traffic – Secures traffic between the external access point (LoadBalancer or Ingress) and Cloudera Surveyor. Internal TLS ensures that even internal traffic within the Kubernetes cluster remains encrypted, protecting against potential threats inside the cluster network.

  • TLS for client traffic – Secures connections from end users (clients) to the Cloudera Surveyor UI. Configured on the Kubernetes Service that provides external access (LoadBalancer or Ingress). Ensures that all data exchanged with clients connecting from the outside world are encrypted.

  • TLS for LDAP traffic – Secures connections between Cloudera Surveyor and your LDAP authentication provider. Typically, you set this up when you configure LDAP authentication.

  • TLS for inter-instance cache traffic – Secures cache replication traffic between Cloudera Surveyor Pods. Cloudera Surveyor runs multiple Pods that synchronously replicate cached Kafka cluster snapshots; encrypting this channel prevents interception of snapshot data within the Kubernetes cluster network.

Cloudera recommends that you enable TLS for all of the above listed categories. This ensures that all traffic handled by Cloudera Surveyor is secure.

Configuring TLS channel encryption

Configure TLS with TLS-related properties in your custom values file. TLS for internal, client, LDAP, and inter-instance cache traffic is configured separately.

TLS for internal traffic

TLS for internal traffic is configured with the tlsConfigs.* properties. For example:

tlsConfigs:
  enabled: true
  secretRef: "[***TLS SECRET NAME***]"

  • tlsConfigs.enabled – Enables or disables TLS

  • tlsConfigs.secretRef – The name of the Kubernetes Secret containing the TLS certificate and private key. This Secret must be created in advance or managed by cert-manager.

TLS for client traffic

TLS for client traffic is configured on the Service (Ingress or LoadBalancer) that provides access to Cloudera Surveyor.

For Ingress, you configure ingress.tls.* properties. For example:

#...
ingress:
  enabled: true
  className: "nginx"
  rules:
    host: my-domain.example
  tls:
    enabled: true
    issuer: "[***ISSUER NAME***]"
    secretRef: "[***INGRESS TLS CERT SECRET***]"

TLS for Ingress is typically configured when you set up external access with Ingress. For a step-by-step guide see Configuring external access with Ingress.

For LoadBalancers, the actual load balancer is provisioned and managed by your cloud or infrastructure provider. As a result, TLS settings and certificate management may vary depending on the platform. Refer to vendor-specific documentation for detailed guidance on configuring TLS.

TLS for LDAP traffic

TLS for LDAP traffic is configured using globalTruststore.* properties. These properties specify a Secret that contains a truststore with the TLS certificate of the root Certificate Authority (CA) of the LDAP certificate chain as well as a password to access the truststore. You typically configure these properties when you set up LDAP. For a step-by-step guide, see Configuring LDAP authentication

TLS for inter-instance cache traffic

TLS for inter-instance cache traffic is configured with the surveyorConfig.surveyor.clustering.tls.* properties. For example:

surveyorConfig:
  surveyor:
    clustering:
      tls:
        enabled: true
        keystore:
          secretRef:
            name: [***CLUSTERING CERT SECRET***]
            key: keystore.p12
          password:
            secretRef:
              name: [***CLUSTERING PASSWORD SECRET***]
              key: password
        truststore:
          secretRef:
            name: [***CLUSTERING CERT SECRET***]
            key: truststore.p12
          password:
            secretRef:
              name: [***CLUSTERING PASSWORD SECRET***]
              key: password

  • clustering.tls.enabled – Enables or disables TLS for inter-instance cache traffic.

  • clustering.tls.keystore.* – Specifies the Secrets and keys that contain the keystore file and its password. The keystore holds the Pod's private key and certificate, used to authenticate with other Cloudera Surveyor Pods.

  • clustering.tls.truststore.* – Specifies the Secrets and keys that contain the truststore file and its password. The truststore holds the CA certificate used to verify the certificates presented by other Cloudera Surveyor Pods. Required when using self-signed certificates.

Managing certificates

Learn about managing TLS certificates in Cloudera Surveyor. You can manage certificates manually or use cert-manager to automate certificate management. Cloudera recommends automatic certificate management.

Cloudera Surveyor stores its TLS certificates and keystores in Secrets. The Secrets are specified by the following properties.

  • tlsConfigs.secretRef – Specifies the name of the Secret containing the TLS certificate used for internal TLS traffic.

  • ingress.tls.secretRef – Specifies the name of the Secret containing the TLS certificate used by the Ingress to secure incoming client communications.

  • clustering.tls.keystore.secretRef and clustering.tls.truststore.secretRef – Specify the Secrets containing the keystore and truststore used for inter-instance cache TLS.

Cloudera recommends that you use cert-manager to manage the Secrets and the certificates that they store. Alternatively, you can choose to manage them manually.

Automatic certificate management with cert-manager

cert-manager is a popular Kubernetes add-on for automating the management and issuance of TLS certificates. In order to manage the certificates used by Cloudera Surveyor with cert-manager, you need the following.

  • A cert-manager instance in your Kubernetes cluster.

  • An Issuer deployed for cert-manager.

The management of the Secrets that Cloudera Surveyor uses to store certificates differs.

TLS for internal (backend) traffic – tlsConfigs.secretRef

The Certificate resource for this Secret must be created manually. When creating a Certificate resource, set spec.secretName to the same value as tlsConfigs.secretRef. This way cert-manager saves the certificate and private key to the Secret that Cloudera Surveyor uses.

For example, a Certificate resource:

#...
kind: Certificate
spec:
  secretName: [***SURVEYOR TLS CERT SECRET***]

The corresponding Surveyor values:

tlsConfigs:
  enabled: true
  secretRef: "[***SURVEYOR TLS CERT SECRET***]"
TLS for client traffic – ingress.tls.secretRef

The Certificate resource for this Secret is created automatically when the cert-manager.io/issuer: [***ISSUER NAME***] annotation is set in the ingress.extraAnnotations property. Specifically, the Ingress requests a certificate from cert-manager using the Issuer name from the annotation. This triggers the creation of the Certificate resource and saves the certificate and private key to the Secret defined in ingress.tls.secretRef.

ingress:
  enabled: true
  tls:
    enabled: true
    secretRef: "[***INGRESS TLS CERT SECRET***]"
  extraAnnotations:
    cert-manager.io/issuer: "[***ISSUER NAME***]"
TLS for inter-instance cache traffic – clustering.tls.keystore.secretRef and clustering.tls.truststore.secretRef

The Certificate resource for these Secrets must be created manually. Set spec.secretName to a name of your choice and enable spec.keystores.pkcs12.create. cert-manager stores the keystore and truststore in the Secret under the keystore.p12 and truststore.p12 keys respectively.

Additionally, create a Secret containing the keystore and truststore password and reference it in spec.keystores.pkcs12.passwordSecretRef so that cert-manager uses it when generating the files.

For example, a Certificate resource:

#...
kind: Certificate
spec:
  secretName: [***CLUSTERING CERT SECRET***]
  keystores:
    pkcs12:
      create: true
      passwordSecretRef:
        name: [***CLUSTERING PASSWORD SECRET***]
        key: password

In your Cloudera Surveyor values file, point both properties at the Secret specified in spec.secretName, using keystore.p12 and truststore.p12 as the respective keys. Reference the same password Secret in clustering.tls.keystore.password.secretRef and clustering.tls.truststore.password.secretRef.

The corresponding Surveyor values:

surveyorConfig:
  surveyor:
    clustering:
      tls:
        enabled: true
        keystore:
          secretRef:
            name: [***CLUSTERING CERT SECRET***]
            key: keystore.p12
          password:
            secretRef:
              name: [***CLUSTERING PASSWORD SECRET***]
              key: password
        truststore:
          secretRef:
            name: [***CLUSTERING CERT SECRET***]
            key: truststore.p12
          password:
            secretRef:
              name: [***CLUSTERING PASSWORD SECRET***]
              key: password

Manual certificate management

When managing certificates manually, you must create the Secrets that contain the certificates, private keys, and keystores manually. Ensure that you create and update the appropriate Secrets specified in tlsConfigs.secretRef, ingress.tls.secretRef, clustering.tls.keystore.secretRef, and clustering.tls.truststore.secretRef.