Converting Strimzi custom resources to the v1 API

The Strimzi API conversion tool converts custom resources and CRDs from deprecated API versions to v1. You must complete resource conversion and CRD storage upgrade before upgrading to Cloudera Streams Messaging Operator for Kubernetes 1.7.

The v1alpha1, v1beta1, and v1beta2 API versions for Strimzi Custom Resource Definitions (CRDs) are removed in Cloudera Streams Messaging Operator for Kubernetes 1.7. Only the kafka.strimzi.io/v1 API is supported. Custom resources that still use old API versions stop working after upgrading.

The Strimzi API conversion tool handles most of the migration of custom resources and CRDs to the v1 API automatically, but some changes must be applied manually before or after running the tool. You have two methods when running the tool:

Running the conversion tool locally — download and run the tool from your machine. Use this method for GitOps and file-based workflows, or when you prefer to run the tool from your machine against a cluster you have kubectl access to. In this method you:
1. Convert resources using the convert-file command to convert local YAML files, or the convert-resource command to convert resources straight in a running cluster.
2. Finalize the CRD storage version using the crd-upgrade command.
note
Running the tool locally requires Java to be installed and available in your PATH. If Java is not available on your local machine, run the tool as a Kubernetes Job instead.
Running the conversion tool as a Kubernetes Job — run the tool inside the cluster using the Strimzi Docker image. Use this method when you cannot run the conversion tool from your local machine. This approach does not require you to download the tool to your local machine. In this method you:
1. Convert resources using the convert-resource command as a Kubernetes Job.
2. Finalize the CRD storage version using the crd-upgrade command as a Kubernetes Job by editing and reapplying the same manifest.

Running the conversion tool locally

Converting custom resource YAML files

Convert Strimzi custom resources from local YAML files to the v1 API using the convert-file command. This approach is suitable for GitOps workflows or environments where direct cluster access is restricted.

Java is installed and available in your PATH.
Download and extract the Strimzi API conversion tool from the Cloudera Archive. https://archive.cloudera.com/p/csm-operator/1.7/tools/.
Back up all custom resources.
You have reviewed the manual changes required for your custom resource types. Some manual changes must be applied before running the tool. See Strimzi v1 API configuration changes.
The Cloudera Streams Messaging Operator for Kubernetes version deployed in the cluster is 1.6 or later.

Run convert-file with the --file option to specify the input file.

tip
Each input file can contain multiple resource definitions. The conversion operation converts all Strimzi custom resources in the file while leaving other Kubernetes resources unchanged.
Choose one of the following output options:
- Use --in-place to overwrite the source file.
- Use --output [***OUTPUT FILE***] to write the converted resource to a new file.
- Omit both options to print the converted resource to standard output.
Command examples:
Convert a single file and print output with INFO logging.
```
bin/v1-api-conversion.sh convert-file --file [***INPUT FILE***] --log-level INFO
```
Convert a file and overwrite it in place.
```
bin/v1-api-conversion.sh convert-file --file [***INPUT FILE***] --in-place
```
Convert a file and write the output to a new file.
```
bin/v1-api-conversion.sh convert-file --file [***INPUT FILE***] --output [***OUTPUT FILE***]
```
Convert multiple input files and print the output.
```
bin/v1-api-conversion.sh convert-file --file [***INPUT FILE 1***] --file [***INPUT FILE 2***]
```
Review the log output to ensure that all custom resources converted successfully and address any errors.
Inspect the output YAML and confirm that Strimzi resources use the kafka.strimzi.io/v1 API version.
Apply the converted files to the cluster using kubectl apply or kubectl replace or commit them to your GitOps repository.
Ensure that all conversions complete successfully before proceeding.

Upgrade the CRD storage version. See Upgrading CRD storage version.

Converting custom resources in a Kubernetes cluster

Convert existing Strimzi custom resources directly in a live Kubernetes cluster to the v1 API using the convert-resource command. This command directly updates the resources in the cluster without requiring local YAML files.

Java is installed and available in your PATH.
Download and extract the Strimzi API conversion tool from the Cloudera Archive. https://archive.cloudera.com/p/csm-operator/1.7/tools/.
Back up all custom resources.
You have reviewed the manual changes required for your custom resource types. Some manual changes must be applied before running the tool. See Strimzi v1 API configuration changes.
The user or service account running the tool has list, get, and replace permissions for all Strimzi custom resources in the target namespaces.
The Cloudera Streams Messaging Operator for Kubernetes version deployed in the cluster is 1.6 or later.

Run convert-resource with the appropriate scope options.
Choose from the following scope options:
- --namespace [***NAMESPACE***] to convert resources in a specific namespace.
- --all-namespaces to convert resources in all namespaces.
- Omit both options to convert resources in the current namespace.
Use the following filtering options to limit the scope of conversion:
- --kind [***KIND***] to convert only resources of a specific kind, for example Kafka or KafkaConnect.
- --name [***NAME***] with --kind to convert a single resource by name.
Command examples:
Convert all Strimzi resources in the current namespace with INFO logging.
```
bin/v1-api-conversion.sh convert-resource --log-level INFO
```
Convert all Strimzi resources in all namespaces.
```
bin/v1-api-conversion.sh convert-resource --all-namespaces
```
Convert all Strimzi resources in a specific namespace.
```
bin/v1-api-conversion.sh convert-resource --namespace [***NAMESPACE***]
```
Convert only resources of a specific kind in all namespaces.
```
bin/v1-api-conversion.sh convert-resource --all-namespaces --kind Kafka
```
Convert resources of multiple kinds in all namespaces.
```
bin/v1-api-conversion.sh convert-resource --all-namespaces --kind Kafka --kind KafkaConnect
```
Convert a single resource by name in a specific namespace.
```
bin/v1-api-conversion.sh convert-resource --namespace [***NAMESPACE***] --kind Kafka --name [***CLUSTER NAME***]
```
Review the log output to ensure that all custom resources converted successfully and address any errors.
Ensure that all conversions complete successfully before proceeding.

Upgrade the CRD storage version. See Upgrading CRD storage version.

Upgrading CRD storage version

Upgrade Strimzi Custom Resource Definitions (CRDs) to the v1 storage version using the crd-upgrade command as the final step of the conversion before upgrading to Cloudera Streams Messaging Operator for Kubernetes 1.7.

Java is installed and available in your PATH.
Download and extract the Strimzi API conversion tool from the Cloudera Archive. https://archive.cloudera.com/p/csm-operator/1.7/tools/.
The user or service account running the tool has patch permissions for Custom Resource Definitions.
The Cloudera Streams Messaging Operator for Kubernetes version deployed in the cluster is 1.6 or later.

Run crd-upgrade:
```
bin/v1-api-conversion.sh crd-upgrade
```

Verify that all Strimzi CRDs use the v1 API as the storage version:

kubectl get crd -o name | grep kafka.strimzi.io | while read crd; do
  echo -n "$crd  "
  kubectl get "$crd" -o jsonpath='{.status.storedVersions}{"\n"}'
done

All CRDs must return [v1] as the stored version.

After upgrading the CRDs, use only the properties supported in the v1 API in all custom resources.

Upgrade Cloudera Streams Messaging Operator for Kubernetes

Running the conversion tool as a Kubernetes Job

Run the Strimzi API conversion tool as a Kubernetes Job to convert custom resources in a cluster without running the tool locally.

Running the conversion tool as a Kubernetes Job is an alternative to running it locally. The tool is embedded in the Strimzi Docker image and can be run as a Kubernetes Job.

Use this approach when you cannot run the conversion tool from your local machine, or when you prefer to run the conversion entirely within the cluster. You still need access to apply manifests to the cluster (for example, using kubectl).

You have kubectl access to the cluster and can apply manifests.
All custom resources are backed up.
You have reviewed the manual changes required for your custom resource types. Some manual changes must be applied before running the tool. See Strimzi v1 API configuration changes.
In air-gapped environments, ensure the Strimzi Docker image is available in your own registry before proceeding. The image you must mirror is:
```
container.repository.cloudera.com/cloudera/kafka-operator:1.0.1-csmop-1.7.0-b248
```
The Cloudera Streams Messaging Operator for Kubernetes version deployed in the cluster is 1.6 or later.

Create a manifest that defines the resources required to run the conversion tool in the cluster.

The manifest must include the following:

A ServiceAccount that the Job runs under.
A ClusterRole and ClusterRoleBinding (RBAC permissions) that grant the required access to Strimzi custom resources and CRDs.
A Job resource that runs the conversion tool using those permissions.

If you are running the Job in a specific namespace, ensure that the namespace name is consistent across the ServiceAccount, ClusterRoleBinding, and Job definitions.

The following example combines all required resources into a single YAML file. Replace [***NAMESPACE***] with the namespace where the Job should run.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: strimzi-v1-api-conversion
  namespace: [***NAMESPACE***]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: strimzi-v1-api-conversion
rules:
  - apiGroups:
      - kafka.strimzi.io
    resources:
      - kafkas
      - kafkanodepools
      - kafkaconnects
      - kafkaconnectors
      - kafkabridges
      - kafkamirrormaker2s
      - kafkarebalances
      - kafkatopics
      - kafkausers
    verbs:
      - get
      - list
      - patch
      - update
  - apiGroups:
      - core.strimzi.io
    resources:
      - strimzipodsets
    verbs:
      - get
      - list
      - patch
      - update
  - apiGroups:
      - apiextensions.k8s.io
    resources:
      - customresourcedefinitions
      - customresourcedefinitions/status
    resourceNames:
      - kafkabridges.kafka.strimzi.io
      - kafkaconnectors.kafka.strimzi.io
      - kafkaconnects.kafka.strimzi.io
      - kafkamirrormaker2s.kafka.strimzi.io
      - kafkanodepools.kafka.strimzi.io
      - kafkarebalances.kafka.strimzi.io
      - kafkas.kafka.strimzi.io
      - kafkatopics.kafka.strimzi.io
      - kafkausers.kafka.strimzi.io
      - strimzipodsets.core.strimzi.io
    verbs:
      - get
      - list
      - patch
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - get
      - list
      - create
      - patch
      - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: strimzi-v1-api-conversion
subjects:
  - kind: ServiceAccount
    name: strimzi-v1-api-conversion
    namespace: [***NAMESPACE***]
roleRef:
  kind: ClusterRole
  name: strimzi-v1-api-conversion
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: batch/v1
kind: Job
metadata:
  name: strimzi-v1-api-conversion
  namespace: [***NAMESPACE***]
spec:
  template:
    spec:
      serviceAccountName: strimzi-v1-api-conversion
      containers:
        - name: strimzi-v1-api-conversion
          image: container.repository.cloudera.com/cloudera/kafka-operator:1.0.1-csmop-1.7.0-b248
          command:
            - /opt/v1-api-conversion/bin/v1-api-conversion.sh
            - convert-resource
      restartPolicy: OnFailure

Apply the manifest to the cluster:

kubectl apply --filename strimzi-v1-api-conversion-job.yaml

Check that the Job has completed and review the logs.

kubectl get jobs --namespace [***NAMESPACE***]

kubectl logs job/strimzi-v1-api-conversion --namespace [***NAMESPACE***]

Verify that the converted resources now use the kafka.strimzi.io/v1 API version.

If direct CLI access is unavailable, check the logs or your monitoring dashboard to confirm that the Job completed successfully.
Ensure that all conversions complete successfully before proceeding to upgrade the CRD storage version.

Run crd-upgrade to finalize the CRD storage version upgrade.

To run crd-upgrade as a Job, edit the container command in the manifest and reapply it. Replace convert-resource with crd-upgrade in the command field:

#...
kind: Job
spec:
  template:
    spec:
      containers:
        - name: strimzi-v1-api-conversion
          image: container.repository.cloudera.com/cloudera/kafka-operator:1.0.1-csmop-1.7.0-b248
          command:
            - /opt/v1-api-conversion/bin/v1-api-conversion.sh
            - crd-upgrade

Monitor the Job to completion as in the previous steps.

Strimzi v1 API configuration changes

Strimzi v1 API configuration changes for custom resources, listing automatic updates applied by the conversion tool and the manual changes you must apply independently.

Manual and automatic changes

Custom resources can be updated for the v1 API in two ways:

By using the Strimzi API conversion tool to apply supported automatic changes.
By editing and reapplying resources manually without using the tool.

Even when the conversion tool is used, some changes must still be applied manually because the tool cannot modify or remove unsupported configuration. The reference information below lists which changes the tool applies automatically and which require manual updates.

Manual changes are updates the conversion tool does not apply automatically.

To apply a manual change:

Edit the affected custom resource.
Update or remove the required properties.
Reapply the resource using kubectl apply -f or kubectl replace -f.

Automatic changes are updates the conversion tool applies during conversion. Even when the conversion tool is used, some manual changes are still required.

Kafka

Manual changes required

The conversion tool does not apply the following updates. You must make these changes manually to ensure compatibility:

Ensure the .spec section is present (required).
Replace unsupported authentication type: oauth with type: custom.
In type: custom authentication, remove the secrets property. Mount secrets using the template section instead.
Replace unsupported authorization types keycloak and opa with type: custom.
Remove the .spec.kafka.resources property. Define resource requests and limits in KafkaNodePool resources instead.

Automatic changes supported

The following updates are applied automatically by the conversion tool. If the conversion tool is not used, the same changes can be made manually:

Removes the .spec.zookeeper section. ZooKeeper-based clusters are not supported.
Removes the .spec.jmxTrans section. JMXTrans is not supported.
Removes the following Kafka properties:
- .spec.kafka.replicas (replicas are configured through KafkaNodePool resources)
- .spec.kafka.storage (storage is configured through KafkaNodePool resources)
- .spec.kafka.template.statefulset
Removes the following Cruise Control properties:
- .spec.cruiseControl.tlsSidecar
- .spec.cruiseControl.template.tlsSidecarContainer
- .spec.cruiseControl.brokerCapacity.disk
- .spec.cruiseControl.brokerCapacity.cpuUtilization
Removes the .spec.kafkaExporter.template.service property.
Removes the following Entity Operator properties:
- .spec.entityOperator.tlsSidecar
- .spec.entityOperator.template.tlsSidecarContainer
- .spec.entityOperator.topicOperator.topicMetadataMaxAttempts
- .spec.entityOperator.topicOperator.zookeeperSessionTimeoutSeconds
- .spec.entityOperator.userOperator.zookeeperSessionTimeoutSeconds
Replaces .spec.entityOperator.topicOperator.reconciliationIntervalSeconds with .spec.entityOperator.topicOperator.reconciliationIntervalMs (value multiplied by 1000).
Replaces .spec.entityOperator.userOperator.reconciliationIntervalSeconds with .spec.entityOperator.userOperator.reconciliationIntervalMs (value multiplied by 1000).
Removes the following status properties from YAML files that include the status section:
- .status.registeredNodeIds
- .status.kafkaMetadataState
- .status.listeners[].type

KafkaConnect

Manual changes required

The conversion tool does not apply the following updates. You must make these changes manually to ensure compatibility:

Ensure the .spec section is present (required).
Replace unsupported authentication type: oauth with type: custom.
Remove the .spec.externalConfiguration property. Use the .spec.template section to add custom volumes or environment variables instead.

Automatic changes supported

The following updates are applied automatically by the conversion tool. If the conversion tool is not used, the same changes can be made manually:

Adds the .spec.replicas property if missing, defaulting to 3.
Renames .spec.build.output.additionalKanikoOptions to .spec.build.output.additionalBuildOptions.
Removes the .spec.template.deployment property.
Removes unsupported tracing type: jaeger.
Adds the following required properties, setting them from existing configuration or default values:
- .spec.groupId: set from .spec.config.group.id, or defaults to connect-cluster.
- .spec.configStorageTopic: set from .spec.config.config.storage.topic, or defaults to connect-cluster-configs.
- .spec.offsetStorageTopic: set from .spec.config.offset.storage.topic, or defaults to connect-cluster-offsets.
- .spec.statusStorageTopic: set from .spec.config.status.storage.topic, or defaults to connect-cluster-status.

KafkaNodePool

Manual changes required

Ensure the .spec section is present (required).

Automatic changes supported

Removes the overrides property from .spec.storage. To configure different storage settings for specific nodes, define separate KafkaNodePool resources.

KafkaTopic

Manual changes required

Ensure the .spec section is present (required).

No automatic changes are applied by the conversion tool for KafkaTopic resources.

KafkaUser

Manual changes required

Ensure the .spec section is present (required).

Automatic changes supported

Replaces the operation property in .spec.authorization.acls[] with the operations array.

KafkaRebalance

Manual changes required

Ensure the .spec section is present (required).

No automatic changes are applied by the conversion tool for KafkaRebalance resources.

KafkaConnector

Manual changes required

Ensure the .spec section is present (required).

Automatic changes supported

Removes the .spec.pause property. If pause: true was set, it is converted to state: paused.

StrimziPodSet

StrimziPodSet is an internal resource managed automatically by the Cluster Operator. Conversion is handled by Cloudera Streams Messaging Operator for Kubernetes and does not require user action.