Upgrade-related known issues

This topic describes the upgrade-related known issues for Cloudera Data Warehouse (CDW) Private Cloud.

DWX-15299: Atlas lineage does not work after upgrading from 1.5.0 to 1.5.1

If you upgrade the Database Catalogs and the Virtual Warehouses after upgrading the platform from 1.5.0 to 1.5.1, then the Kafka producers have idempotence enabled by default. Idempotent producers must have idempotent write permission set on the cluster resource in Ranger. If idempotence is not disabled or if the write permission is not set correctly, then Atlas integration fails with the following error Entity is not found in atlas, please check metastore pod/atlas application logs’.

To resolve this issue, you can either:

Explicitly disable idempotence by setting the value of the atlas.kafka.enable.idempotence property to false in hive-atlas-config for Metastore, Hiveserver2, Hive query-executor and Impala coordinator.
Update your policies in Ranger and ensure that Hive and Impala users have Idempotent Write permission on the cluster resource.

DWX-15176: Hue frontend pods crashloop after upgrading from 1.5.0 to 1.5.1 on ECS

After you upgrade from 1.5.0 to 1.5.1 on Embedded Container Service (ECS), Virtual Warehouses go into an error state as Hue frontend pods crash. This is due to a breaking change in the underlying Kubernetes version.

Upgrade the Virtual Warehouses to upgrade their runtime software by clicking

> Upgrade.

DWX-8525 and DWX-8526: Known limitations for upgrading from 1.4.1. to 1.5.1

Hue does not work after upgrade due to DWX-15176 (Hue frontend pods crashloop) after upgrading from 1.4.1 to 1.5.1.

You must reactivate the CDW environment after upgrading the Private Cloud data services from 1.4.1 to 1.5.1.

important

Review the following caveats before upgrading from 1.4.1 to 1.5.1:

Reactivating the environment deletes the existing Database Catalogs and Virtual Warehouses. However, the metadata stored in the Hive Metastore (HMS) and data that is stored on HDFS or Ozone from the previous version is not deleted from the base cluster. Cloudera recommends that you manually take a backup of the Virtual Warehouse configurations, which you can use to create new Virtual Warehouses after the upgrade.
Apply workaround for the AD limitation (TSB-639) on the base cluster. Cloudera recommends that you do not use deprecated encryption types in production environments. Use stronger encryption types such as AES 256 and AES 128. For more information, see Resolving Kerberos authentication failure.

DWX-15571: tez.history.logging.proto-base-dir and hive.hook.proto.base-directory locations change after the runtime upgrade

The values of following Hue Query Processor properties change after you upgrade the Database Catalog and Virtual Warehouses to the latest version:

tez.history.logging.proto-base-dir
hive.hook.proto.base-directory

CDW changes these locations to match the base cluster configuration for hive.metastore.warehouse.external.dir. You may also see the FileNotFoundException: File does not exist error in the Hue Query Processor logs.

To use the old location and to avoid seeing the "File does not exist" error in the logs, you must update the Hue Query Processor configurations as follows:

Log in to the Data Warehouse service as an DWAdmin.
Go to Database Catalog > > Edit > CONFIGURATIONS > Hue query processor and select hue-query-processor.json from the Configuration files drop-down menu.
Note the paths of the tez.history.logging.proto-base-dir and hive.hook.proto.base-directory properties.
After you upgrade the Database Catalog, go to Database Catalog > > Edit > CONFIGURATIONS > Hue query processor and select hue-query-processor.json from the Configuration files drop-down menu.

Specify the desired path under the dasConf section as shown in the following example:

  "dasConf": {
    "hive.hook.proto.base-directory": "/warehouse/tablespace/external/hive/sys.db/query_data",
    "tez.history.logging.proto-base-dir": "/warehouse/tablespace/external/hive/sys.db"
  },

Click Apply Changes.