Connection to the cluster with configured DNS aliases

If you only want to use a single hostname for the whole cluster, configure a DNS alias.

This A record setup is a convenient solution for having a single hostname for the whole cluster. You only need to provide a hostname that serves as an alias for the brokers:

kafka-topics --list --bootstrap-server primary.kafka.cluster.hostname:9092

When connecting your clients to the brokers, the DNS alias is specified and then translated to the actual hostname.

However, a port number must also be specified, and the port you specify is used for all brokers identified by the DNS alias. As a result of this, you must ensure that all brokers identified by a specific DNS alias use the same port number. Otherwise, clients fail to connect even if DNS resolution is successful.

In case of a broker failure, you can change the records in the DNS server to point to other brokers. The next time a client tries to connect, it resolves to a different cluster’s broker. This way you do not hardcode broker FQDNs to the bootstrap server list.

If the client has discovered a cluster and is actively using it, and the cluster suddenly stops, the client tries to connect to the bootstrap servers it got from the DNS server before the bootstrap phase. So it does not automatically ask the DNS server for new bootstrap servers and does not fail over to the other cluster, even if the DNS records have already been changed.