Load balancer

Load balancers automatically and efficiently distribute network traffic between multiple backend servers. A load balancer setup can be used in CSM Operator to expose your Kafka brokers to the outside world.

There are many load balancer implementations available and all cloud providers provide their own solutions. Different implementations handle load balancing on different levels of the network, most commonly you have layer 4 (transport) and layer 7 (application) load balancing. CSM Operator uses layer 4 load balancing. This is because common load balancer implementations do not support the Kafka protocol.

In CSM Operator, you set up external cluster access using load balancers by adding a loadbalancer type listener to your Kafka resource (listeners.type:loadbalancer).

Once configuration is done, CSM Operator sets up multiple load balancers as well as multiple LoadBalancer type Kubernetes Services. This means that you will have the following:

A load balancer and a corresponding LoadBalancer Service that serves as an external bootstrap. This is used by clients for the initial connection and to receive metadata (advertised listeners) from the Kafka cluster.
A unique load balancer and a LoadBalancer Service for each Kafka Broker.

CSM Operator creates the LoadBalancer type Services first. Following the creation of the Services, the load balancers are automatically created. Typically your infrastructure provider assigns the load balancer a hostname and IP address. These are automatically added to the status section of the Kafka resource. CSM Operator collects both hostname and IP address and uses them to configure the advertised listeners of your Kafka brokers.

CSM Operator uses hostnames instead of IP addresses by default. This is because load balancer IP addresses might change, the hostnames, however, are fixed and remain the same as long as the load balancer is running. By default, CSM Operator uses the IP address if there is no hostname assigned to the load balancer. In case you want to use IP addresses, you can do so by manually configuring them during setup.

Once the listener is configured, you can connect your clients running outside of the Kubernetes network by directing them to the bootstrap load balancer. The load balancers, Kubernetes, and Kafka handle everything else and ensure that client requests are routed to the correct brokers.

Configuring load balancer listeners

Complete the following steps to set up and configure a loadbalancer listener in CSM Operator. The following steps also include an example on how to connect a Kafka console client to the cluster.

These steps demonstrate basic listener configuration with typical customizations. In addition to the configuration shown here, you can further customize your listener and enable and disable TLS encryption using the tls property, specify a client authentication mechanism with the authentication property, as well as add various additional configurations using the configuration property. For a comprehensive list of available properties, see the GenericKafkaListener schema reference in the Strimzi API reference.

Configure your Kafka resource.
Add a new external listener that has its type set to loadbalancer.
Optionally, you can further customize the listener. For example, the following configuration snippet shows an example where the advertised hostnames and ports are specified using advertisedHost and advertisedPort properties.
```
#...
kind: Kafka
spec:
  kafka:
    listeners:
      - name: external
        port: 9094
        type: loadbalancer
        tls: true
        authentication:
          type: tls
        configuration:
          brokers:
            - broker: 0
              advertisedHost: my-broker-0.cloudera.com
              advertisedPort: 12340
            - broker: 1
              advertisedHost: my-broker-1.cloudera.com
              advertisedPort: 12341
            - broker: 2
              advertisedHost: my-broker-2.cloudera.com
              advertisedPort: 12342
```
tip
The advertisedHost property also accepts IP addresses. Specify IP addresses instead if DNS resolution does not work for the Kafka clients. Configuring exact hostnames or ports does not change the hostname or port of the load balancer, instead it changes the address advertised by Kafka.
Verify that LoadBalancer type services as well as load balancers are running
```
kubectl get services --namespace [***NAMESPACE***]
```
The output will be similar to the following example.
```
NAME                                  TYPE        CLUSTER-IP      	EXTERNAL-IP                           
#...       
my-cluster-kafka-external-bootstrap   LoadBalancer   10.43.18.136    10.65.0.5   
my-cluster-kafka-external-0           LoadBalancer   10.43.1.63      10.65.0.6     
my-cluster-kafka-external-1           LoadBalancer   10.43.46.74     10.65.0.7     
my-cluster-kafka-external-2           LoadBalancer   10.43.113.194   10.65.0.8 
```
Notice that there is a LoadBalancer Service deployed for each Kafka broker. Additionally you have a separate external bootstrap LoadBalancer called [***CLUSTER NAME***]-kafka-external-bootstrap.

Clients connecting to the Kafka cluster should be directed to the external bootstrap. The addresses in the EXTERNAL-IP column are the hostnames or IPs of the load balancers. Having this column populated with values indicates that the load balancers are created.

Optional: Extract the TLS certificate form your broker and import it into a Java truststore file.

Doing the following is only required if you have TLS/SSL encryption enabled for the load balancer listener.

kubectl get secret [***CLUSTER NAME***]-cluster-ca-cert \
  --namespace [***NAMESPACE***] --output jsonpath='{.data.ca\.crt}' \
  | base64 -d > ca.crt

keytool -import -trustcacerts -alias [***ALIAS***] \
  -file ca.crt \
  -keystore truststore.jks \
  -storepass [***PASSWORD***] \
  -noprompt

Optional: Ensure that the resulting truststore is available on the machine where you run your client and that the client has access to the file.

Get the address of the bootstrap load balancer.

kubectl get kafka [***CLUSTER NAME***] \
  --namespace [***NAMESPACE***] \
  --output=jsonpath='{.status.listeners[?(@.name=="[***LISTENER NAME***]")].bootstrapServers}{"\n"}'

Clients that you want to connect to the cluster should be directed to this address.

Configure and run your client.
The following example shows a Kafka console producer. Configuring TLS/SSL related properties is only required if TLS/SSL is enabled for the load balancer listener.
```
kafka-console-producer.sh \
  --bootstrap-server [***BOOTSTRAP LOAD BALANCER HOST***]:9094 \
  -- producer-property security.protocol=SSL \
  --producer-property ssl.truststore.password=[***PASSWORD***] \
  --producer-property ssl.truststore.location=[***TRUSTSTORE LOCATION***] \
  --topic [***TOPIC***]
```
tip
Instead of passing TLS/SSL properties directly using the --producer-property option, you can also create a configuration file containing the properties and pass the file with --producer.config option.