Cloudera Docs

Kafka Connect property configuration in Cloudera Manager for Prometheus

Learn about the Kafka Connect properties that you need to configure in Cloudera Manager before you start using the Prometheus metric store.

In order to set up Prometheus as the SMM metrics store, you need to configure the metrics reporter of Kafka Connect. Configuration is done in Cloudera Manager by setting Kafka Connect properties. Configuration differs depending on whether you want to enable security for the Kafka Connect metrics reporter. All of the following properties are found in Cloudera Manager > Kafka service > Configuration. Completing the following makes Prometheus-compatible metrics available on the /prometheus-metrics API path on each Kafka Connect worker host.

Do the following to configure an unsecured metrics-scraping endpoint:
  1. Add metrics.jetty.server.prometheus.metrics.enable=true to the Kafka Connect Advanced Configuration Snippet (Safety Valve) for connect-distributed.properties advanced configuration snippet.
  2. Look up or configure the value of Jetty Metrics Port.

    This is the port where Kafka Connect exposes its metrics when TLS/SSL is not enabled. This is the port that you need to add to prometheus.yml. Only configure this property if you want to change the port.

Do the following to configure a secure metrics scraping endpoint:
  1. Add metrics.jetty.server.prometheus.metrics.enable=true to the Kafka Connect Advanced Configuration Snippet (Safety Valve) for connect-distributed.properties advanced configuration snippet.
  2. Enable TLS/SSL, Basic Authentication (BA), or both for the metrics reporter by configuring the following properties:
    • Enable TLS/SSL for Kafka Connect
    • Enable Basic Authentication for Metrics Reporter
    • Jetty Metrics User Name
    • Jetty Metrics User Password
    Note the following about these properties:
    • The Enable TLS/SSL for Kafka Connect property is not specific to the metrics reporter. It enables TLS/SSL for Kafka Connect roles including their metrics reporter.
    • You can enable both TLS/SSL and BA on their own, however, Cloudera recommends that you enable both.
    • Jetty Metrics User Name and Jetty Metrics User Password set the username and password that you need to add to prometheus.yml if you enable BA.
  3. Look up or configure the value of Secure Jetty Metrics Port or Jetty Metrics Port

    These are the ports where Kafka Connect exposes its metrics. Secure Jetty Metrics Port is only used if TLS/SSL is enabled, otherwise Kafka Connect uses Jetty Metrics Port. You need to add the port being used to prometheus.yml. Only configure this property if you want to change the port.