Upgrading an Azure Kubernetes Service cluster for CDW

Azure Kubernetes Service (AKS) cluster requires regular updates to the Kubernetes versions. Using the latest AKS version supported by Cloudera avoids compatibility issues between Cloudera Data Warehouse (CDW) and Azure resources. Using the Azure CLI to upgrade the AKS cluster to AKS 1.22 is not supported in this release 1.5.1-b110 (released Nov 22, 2022), but through the environment activation from CDW process you can automatically get and use AKS 1.22 in your CDP cluster.

Cloudera supports the following AKS versions for upgrade using the Azure CLI to upgrade the AKS cluster.

  • AKS 1.19
  • AKS 1.20
  • AKS 1.21

Do not upgrade your AKS cluster to a version later than AKS 1.21. Doing so can cause the cluster to become unusable and can cause downtime.

AKS 1.22 is provisioned automatically when you activate your environment from CDW using this release 1.5.1-b110 (released Nov 22, 2022). AKS 1.22 will be effective after activating your environment from CDW.

The CDW application uses Kubernetes (K8S) clusters to deploy and manage Hive and Impala in the cloud. Kubernetes versions are updated every 3 months on average. When the version is updated, minor versions are deprecated. While upgrading the Kubernetes version for your AKS cluster, you cannot perform a skip-level upgrade. You must upgrade sequentially to every minor version that is available for the AKS cluster. For example, to upgrade from 1.18 to 1.20, you must first upgrade from 1.18 to 1.19 and then to 1.20.

  • Suspend the CDW Virtual Warehouses.
  • If there are more than three Database Catalogs (including custom and default Database Catalog) in an environment, suspend them before proceeding with the upgrade.

The reason behind suspending Virtual Warehouses and Database Catalogs is to reduce the upgrade time.

  1. Use the Azure CLI and upgrade the AKS cluster to a supported release as described in the Azure Kubernetes Service documentation.
    Exception: Do not upgrade to AKS 1.22. AKS 1.22 is not supported for upgrading pre-existing environments.
  2. Restart the Virtual Warehouses and Database Catalogs after you have upgraded the cluster.