Configuring the EFM properties
To integrate EFM with Kafka, you need to configure the EFM properties in the
efm.properties
file.
You have created Kafka topics to receive heartbeats and acknowledgements from MiNiFi agents.
-
Set the following property to true:
efm.heartbeat.kafka.publishEnabled=true
If you do not set it to true, none of the configurations are considered.
-
Set the following property to provide comma separated addresses for brokers if
Kafka runs in a cluster:
efm.heartbeat.kafka.brokerAddress=localhost:9093
-
Set the ID that shows up as client on Kafka side logs.
efm.heartbeat.kafka.clientId=efm
-
Provide topic names where the heartbeat and acknowledgement messages are sent
to respectively.
efm.heartbeat.kafka.heartbeatTopicName=heartbeat efm.heartbeat.kafka.ackTopicName=ack
The topic names must match the topics previously created on Kafka side. -
Optional. Set the following Kafka parameters:
efm.heartbeat.kafka.retryBackoff=1000 efm.heartbeat.kafka.reconnectBackoff=1000 efm.heartbeat.kafka.reconnectBackoffMax=60000 efm.heartbeat.kafka.requestTimeout=30000 efm.heartbeat.kafka.bufferMemory=33554432 efm.heartbeat.kafka.batchSize=16384 efm.heartbeat.kafka.compressionType=gzip efm.heartbeat.kafka.deliveryTimeout=120000 efm.heartbeat.kafka.maxRequestSize=1048576 efm.heartbeat.kafka.acks=1
Fore more details on Kafka parameters, see https://kafka.apache.org/documentation/.
-
Optional. Set the following properties if two-way TLS is enabled:
efm.heartbeat.kafka.ssl.enabled=true efm.heartbeat.kafka.ssl.keystoreLocation= efm.heartbeat.kafka.ssl.keystorePassword= efm.heartbeat.kafka.ssl.keyPassword= efm.heartbeat.kafka.ssl.truststoreLocation= efm.heartbeat.kafka.ssl.truststorePassword= efm.heartbeat.kafka.ssl.securityProtocol=SSL efm.heartbeat.kafka.ssl.enabledProtocols=
-
Optional. Set the following properties if SASL_SSL authentication is
enabled:
efm.heartbeat.kafka.ssl.enabled=true efm.heartbeat.kafka.ssl.truststoreLocation= efm.heartbeat.kafka.ssl.truststorePassword= efm.heartbeat.kafka.ssl.securityProtocol=SASL_SSL efm.heartbeat.kafka.ssl.saslSslUsername=client efm.heartbeat.kafka.ssl.saslSslPassword=client-secret