Integration with Apache Kafka

You can integrate Edge Flow Manager with Apache Kafka and forward agent heartbeats to defined Kafka topics. Learn how to perform the integration with Apache Kafka.

To integrate Edge Flow Manager with Kafka, you need to configure Kafka and Edge Flow Manager properties.

Edge Flow Manager supports the forwarding of agent heartbeats and acknowledges messages exchanged on the C2 protocol between the Edge Flow Manager server and MiNiFi agents. You can also enable two-way TLS or authentication using SASL_SSL for secure communication between Edge Flow Manager and Kafka brokers.

The integration is not on critical path so, if there is any communication issue between the server and the Kafka broker(s), it results in error logs but not in core functionality degradation. When connection between Edge Flow Manager and Kafka is restored, the accumulated messages are forwarded. You can fine tune the buffering related settings.