Integration with Apache Kafka

You can integrate Edge Flow Manager (EFM) with Apache Kafka and forward agent heartbeats to defined Kafka topics. Learn how to perform the integration with Apache Kafka.

To integrate EFM with Kafka, you need to configure Kafka and EFM properties.

EFM supports the forwarding of agent heartbeats and acknowledges messages exchanged on the C2 protocol between the EFM server and MiNiFi agents. You can also enable two-way TLS or authentication using SASL_SSL for secure communication between EFM and Kafka brokers.

The integration is not on critical path so, if there is any communication issue between the server and the Kafka broker(s), it results in error logs but not in core functionality degradation. When connection between EFM and Kafka is restored, the accumulated messages are forwarded. You can fine tune the buffering related settings.