Cloudera Docs

Upgrading an Amazon Kubernetes Service cluster for CDW

To avoid compatibility issues between Cloudera Data Warehouse (CDW) and AWS resources, the version of Kubernetes that supports your CDW clusters should be upgraded if supported by Cloudera.

The CDW application uses Kubernetes (K8S) clusters to deploy and manage Hive and Impala in the cloud. Kubernetes versions are updated every 3 months on average. When the version is updated, minor versions are deprecated.

If you activated your pre-existing AWS environment in version 1.4.2-b118 (released Aug 4, 2022) or later, you can upgrade to EKS 1.22 in this version 1.6.3-b319 (released May 5, 2023). To determine the activation version your pre-existing environment, in the Data Warehouse service, expand Environments. In Environments, search for and locate the environment that you want to view. Click Edit. In Environment Details, you see the CDW version.

New CDW clusters using AWS environments you activate in this release 1.6.3-b319 (released May 5, 2023) of Cloudera Data Warehouse will use Amazon Kubernetes (EKS) version 1.24.

While an environment is being upgraded, all the associated Database Catalogs and Virtual Warehouses remain in a running state, but the system is unstable.

Cloudera recommends that you not use your Database Catalog or Virtual Warehouse while the underlying Kubernetes cluster is being upgraded.

Required role: DWAdmin

  • Obtain the CDW_K8S_UPGRADE entitlement.
  • Amazon EKS, which CDW clusters use, requires 2-3 IP addresses from the subnets you used when you registered the environment in Management Console. Check that these subnets have the available IP addresses; otherwise, the environment upgrade might fail.
  • If any of the subnets or security groups that were added to the environment when it was initially created during environment registration in Management Console have been deleted, the environment upgrade might fail. Check that these subnets or security groups are in place.
  • Check to determine whether the AWS environment you want to upgrade first needs to have its Kubernetes package manager, Helm, migrated from version 2 to version 3:
    1. Follow Steps 1 to 3 below to locate the environment that you want to upgrade.
    2. If the environment tile has a MIGRATE icon in the upper right corner of the tile, the Helm package manager must first be updated for this environment before you can upgrade its Kubernetes version. For details about migrating the environment from Helm 2 to Helm 3, see Helm 2 to Helm 3 migration. When you have completed migrating the environment from Helm 2 to Helm 3, return to this procedure to upgrade the Kubernetes version on the CDW cluster.
  1. In the CDW service, expand the Environments column by clicking More….
  2. In Environments, click search and locate the environment on AWS for upgrading the Kubernetes version.
  3. In the environment tile, click optionsand select Upgrade.
  4. Confirm that you want to upgrade: Click Upgrade to start the upgrade process.