Client access to your cluster is set up in Cloudera Streams Messaging - Kubernetes Operator by configuring listeners in your
Kafka resource. Listeners can be used to expose your brokers, allowing clients to access
them.
Each listener is configured as an array in your Kafka resource. For
example:
You can configure any number of listeners as long as their names and ports are unique. Their
configuration is also highly customizable. For an exhaustive list of accepted properties, see
the GenericKafkaListener as well as other listener schema references in the
Strimzi API reference.
Listener categories and types🔗
There are two categories of listeners, internal and external. Internal listeners are used
to expose Kafka to clients that are internal to the Kubernetes cluster. External listeners
provide a way to expose Kafka to the outside world.
Listeners are further categorized by their type. The different listener
types expose Kafka with different connection mechanisms. The types of listeners available
are as follows.
Internal listener types
internal
An internal type listener uses a
Kubernetes headless Service that gives each broker pod a stable hostname. These
hostnames are set as advertised listeners for Kafka. In addition, a ClusterIp
Kubernetes Service is set up that acts as the Kafka bootstrap. The initial
connection is done using the bootstrap, subsequent connections are opened using
the hostnames given to the pods by the headless Kubernetes Service.
cluster-ip
With a cluster-ip type listener,
individual ClusterIP type Kubernetes services are set up for each broker. The
hostnames of the ClusterIP services are configured as the advertised listeners for
Kafka. In addition, another ClusterIP is provisioned that acts as the Kafka
bootstrap. The initial connection is done using the bootstrap, subsequent
connections are opened using the ClusterIP Services corresponding to each broker.
All Kafka resources that you create in Cloudera Streams Messaging - Kubernetes Operator most
likely contain an internal listener by default. This means that you can test your
cluster and connect your client as soon as the cluster is up and running. To connect a
client, direct it to the address of the bootstrap service that was set up by the
listener. From there Kubernetes and the Strimzi Cluster Operator handle everything
else ensuring that connection requests are sent to the appropriate brokers.
External listener types
nodeport
A nodeport type listener sets up NodePort type Kubernetes
Services to provide external access to Kafka.
route
A route type listener uses OpenShift routes and the default
HAProxy router to provide external access to Kafka.
loadbalancer
A loadbalancer type listener sets up LoadBalancer type
Kubernetes Services and cloud provider or infrastructure managed load balancers to
provide external access to Kafka.
ingress
An ingress type listener uses Kubernetes Ingress and the
Ingress-NGINX controller to provide external access to Kafka.
Which of the available external listener types you choose will depend on your
requirements and infrastructure. Each external listener type is further documented in
their dedicated section. See these sections for more information on how they work as
well as instructions on how to set them up.
Default listener ports🔗
When configuring listeners for Kafka clients, you can use ports from 9092 and above. There
are two default listeners configured in each cluster that can not be configured and accessed
by external clients. These are as follows.
Control plane listener (9090) - This port is used for controller communication
Replication listener (9091) - This port is used by replica fetchers for replicating
topic partitions
