Integrating with Apache KafkaPDF version

Configuring Kafka for integration with EFM

Configuring Apache Kafka to work seamlessly with Edge Flow Manager (EFM) involves creating Kafka topics that receive the heartbeats and acknowledgments from MiNiFi agents.

Ensure that you are using Apache Kafka version 2.5 or above. For more information about Kafka and stream processinging setup, see the Stream Processing documentation.

  1. Create a Kafka topic for receiving heartbeats.
    To receive heartbeats from MiNiFi agents, you need to create a Kafka topic dedicated to this purpose. For example, you can create a Kafka topic named heartbeat.
  2. Create a Kafka topic for receiving acknowledgements.
    Similar to heartbeats, acknowledgments from MiNiFi agents require a separate Kafka topic. For example, you can create a Kafka topic called ack.
  3. Optional: If you wish to enhance the security of your Kafka deployment by enabling two-way TLS or authentication using SASL_SSL, configure the specific Kafka server properties accordingly.
    For more information about enabling TLS, see the Confluent Security Tutorial.