Upgrade Cloudera Data Warehouse on AWS environments for Amazon EKS Kubernetes cluster updates

Amazon Elastic Kubernetes Service (EKS) is updating to Kubernetes version 1.16 and is ending support for version 1.15 on May 2, 2021. To avoid compatibility issues between Cloudera Data Warehouse (CDW) and AWS resources, the version of Kubernetes that supports your CDW clusters must be updated.

The CDW application uses Kubernetes (K8S) clusters to deploy and manage Hive, Impala, and Real-time Event Store Analytics (Druid) in the cloud. Kubernetes versions are updated every 3 months on average. When the version is updated, minor versions are deprecated.

Required role: DWAdmin

  • Amazon EKS, which CDW clusters use, requires 2-3 IP addresses from the subnets you used when you registered the environment in Management Console. If these subnets do not have the available IP addresses, the environment upgrade might fail.
  • If any of the subnets or security groups that were added to the environment when it was initially created during environment registration in Management Console have been deleted, the environment upgrade might fail.
  • Check to determine whether the AWS environment you want to upgrade first needs to have its Kubernetes package manager, Helm, migrated from version 2 to version 3:
    1. Follow Steps 1 to 3 below to locate the environment that you want to upgrade.
    2. If the environment tile has a MIGRATE icon in the upper right corner of the tile, the Helm package manager must first be updated for this environment before you can upgrade its Kubernetes version. For details about migrating the environment from Helm 2 to Helm 3, see Helm 2 to Helm 3 migration. When you have completed migrating the environment from Helm 2 to Helm 3, return to this procedure to upgrade the Kubernetes version on the CDW cluster.
  1. In the CDW service, expand the Environments column by clicking the More… menu on the left side of the page.
  2. In the Environments column, click the search icon and locate the environment on AWS for which you want to upgrade its Kubernetes version.
  3. In the environment tile, click the options menu and select Upgrade, which launches an Upgrade dialog box.
  4. Depending on what version of Kubernetes the environment uses, you will see a message in the Upgrade dialog box that asks you whether you want to upgrade to the appropriate version.

    For example, if your environment currently is using Kubernetes version 1.15, the dialog box asks you, Do you want to upgrade to Kubernetes version 1.17?

  5. Click Upgrade in the dialog box to start the upgrade process.