Cloudera Docs

What is CSM Operator?

CSM (Cloudera Streams Messaging) Operator allows you to deploy and manage Streams Messaging components as container applications on Kubernetes. CSM Operator simplifies the process of creating, managing, and troubleshooting Kafka deployments in a Kubernetes environment. CSM Operator includes several services and components such as Strimzi, Kafka, Cruise Control, and others.

Strimzi
Strimzi is an open-source project that provides a way to run an Apache Kafka cluster on Kubernetes. Within CSM Operator, Strimzi is the component that makes it possible to deploy and manage Kafka workloads in a Kubernetes environment using Kubernetes-native tooling and processes.

Strimzi itself is made up of multiple components and includes various operator applications, Custom Resource Definitions (CRD) as well as Docker (container) images.

Operator applications are purpose built Kubernetes applications that act as an extension to Kubernetes. These applications provide an easy way for you to deploy, manage and configure Kafka and related components.

The CRDs created by Strimzi define the APIs to interface with Kafka-related custom resources on Kubernetes as, for example, KafkaCluster, KafkaNodePools, and KafkaTopic. The custom resources are created as instances of these APIs by providing an associated set of configurations to be applied to the resource. CRDs and custom resources are defined as YAML files.

Kafka
Apache Kafka is an open-source, high performance, highly available, and redundant streams messaging platform. It supports millions of messages per second with low latency and high throughput, scaling elastically and transparently without downtime. Kafka addresses a wide range of streaming data initiatives, enabling enterprises to keep up with customer demand, provide better services, and proactively manage risk.
Apache ZooKeeper
Apache ZooKeeper is an open-source centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. Kafka uses Zookeeper for broker coordination as well as to store broker, topic, and partition metadata.
Cruise Control
Cruise Control acts as a load balancing component in large Kafka installations. It provides automatic data balancing of Kafka partitions across Kafka clusters based on user specified parameters (goals) as well as workload data.

Components shipped in CSM Operator are based on open source projects and might contain additional changes or fixes to guarantee that they work in Cloudera supported environments.

Additionally, not all Kafka and Strimzi features are supported by CSM Operator. See the Release Notes for a comprehensive list of unsupported features and components.

Key features and benefits

A Kafka deployment with CSM Operator provides the following key features and benefits.

  • Flexible, agile, and rapid deployment as well as scaling for variable workloads
  • Standardization of deployments on existing Kubernetes infrastructure
  • Operational efficiency with simple upgrades, and swift creation of new clusters.
  • Ability to deploy Kafka and related components on existing, shared Kubernetes infrastructure. No need for dedicated infrastructure.
  • Lightweight dependencies and system requirements for Kafka-centric deployments.