Configuring Edge Flow Manager properties

To integrate Edge Flow Manager with Kafka, you need to configure the Edge Flow Manager properties in the efm.properties file.

You have created Kafka topics to receive heartbeats and acknowledgements from MiNiFi agents.

  1. Set the following property to true:
    efm.heartbeat.kafka.publishEnabled=true

    If you do not set it to true, none of the configurations are considered.

  2. Set the following property to provide comma separated addresses for brokers if Kafka runs in a cluster:
    efm.heartbeat.kafka.brokerAddress=localhost:9093
  3. Set the ID that shows up as client on Kafka side logs.
    efm.heartbeat.kafka.clientId=efm
  4. Provide topic names where the heartbeat and acknowledgement messages are sent to respectively.
    efm.heartbeat.kafka.heartbeatTopicName=heartbeat
    efm.heartbeat.kafka.ackTopicName=ack
    The topic names must match the topics previously created on Kafka side.
  5. Optional. Set the following Kafka parameters:
    efm.heartbeat.kafka.retryBackoff=1000
    efm.heartbeat.kafka.reconnectBackoff=1000
    efm.heartbeat.kafka.reconnectBackoffMax=60000
    efm.heartbeat.kafka.requestTimeout=30000
    efm.heartbeat.kafka.bufferMemory=33554432
    efm.heartbeat.kafka.batchSize=16384
    efm.heartbeat.kafka.compressionType=gzip
    efm.heartbeat.kafka.deliveryTimeout=120000
    efm.heartbeat.kafka.maxRequestSize=1048576
    efm.heartbeat.kafka.acks=1

    Fore more details on Kafka parameters, see https://kafka.apache.org/documentation/.

  6. Optional. Set the following properties if two-way TLS is enabled:
    efm.heartbeat.kafka.ssl.enabled=true
    efm.heartbeat.kafka.ssl.keystoreLocation=
    efm.heartbeat.kafka.ssl.keystorePassword=
    efm.heartbeat.kafka.ssl.keyPassword=
    efm.heartbeat.kafka.ssl.truststoreLocation=
    efm.heartbeat.kafka.ssl.truststorePassword=
    efm.heartbeat.kafka.ssl.securityProtocol=SSL
    efm.heartbeat.kafka.ssl.enabledProtocols=
  7. Optional. Set the following properties if SASL_SSL authentication is enabled:
    efm.heartbeat.kafka.ssl.enabled=true
    efm.heartbeat.kafka.ssl.truststoreLocation=
    efm.heartbeat.kafka.ssl.truststorePassword=
    efm.heartbeat.kafka.ssl.securityProtocol=SASL_SSL
    efm.heartbeat.kafka.ssl.saslSslUsername=client
    efm.heartbeat.kafka.ssl.saslSslPassword=client-secret