Upgrade-related known issues

This topic describes the upgrade-related known issues for Cloudera Data Warehouse (CDW) Private Cloud.

DWX-15299: Atlas lineage does not work after upgrading from 1.5.0 to 1.5.1
If you upgrade the Database Catalogs and the Virtual Warehouses after upgrading the platform from 1.5.0 to 1.5.1, then the Kafka producers have idempotence enabled by default. Idempotent producers must have idempotent write permission set on the cluster resource in Ranger. If idempotence is not disabled or if the write permission is not set correctly, then Atlas integration fails with the following error Entity is not found in atlas, please check metastore pod/atlas application logs’.
To resolve this issue, you can either:
  • Explicitly disable idempotence by setting the value of the atlas.kafka.enable.idempotence property to false in hive-atlas-config for Metastore, Hiveserver2, Hive query-executor and Impala coordinator.
  • Update your policies in Ranger and ensure that Hive and Impala users have Idempotent Write permission on the cluster resource.
DWX-15176: Hue frontend pods crashloop after upgrading from 1.5.0 to 1.5.1 on ECS
After you upgrade from 1.5.0 to 1.5.1 on Embedded Container Service (ECS), Virtual Warehouses go into an error state as Hue frontend pods crash. This is due to a breaking change in the underlying Kubernetes version.
Upgrade the Virtual Warehouses to upgrade their runtime software by clicking > Upgrade.
DWX-8525 and DWX-8526: Known limitations for upgrading from 1.4.1. to 1.5.1
Hue does not work after upgrade due to DWX-15176 (Hue frontend pods crashloop) after upgrading from 1.4.1 to 1.5.1.
You must reactivate the CDW environment after upgrading the Private Cloud data services from 1.4.1 to 1.5.1.
DWX-15571: tez.history.logging.proto-base-dir and hive.hook.proto.base-directory locations change after the runtime upgrade
The values of following Hue Query Processor properties change after you upgrade the Database Catalog and Virtual Warehouses to the latest version:
  • tez.history.logging.proto-base-dir
  • hive.hook.proto.base-directory
CDW changes these locations to match the base cluster configuration for hive.metastore.warehouse.external.dir. You may also see the FileNotFoundException: File does not exist error in the Hue Query Processor logs.
To use the old location and to avoid seeing the "File does not exist" error in the logs, you must update the Hue Query Processor configurations as follows:
  1. Log in to the Data Warehouse service as an DWAdmin.
  2. Go to Database Catalog > > Edit > CONFIGURATIONS > Hue query processor and select hue-query-processor.json from the Configuration files drop-down menu.
  3. Note the paths of the tez.history.logging.proto-base-dir and hive.hook.proto.base-directory properties.
  4. After you upgrade the Database Catalog, go to Database Catalog > > Edit > CONFIGURATIONS > Hue query processor and select hue-query-processor.json from the Configuration files drop-down menu.
  5. Specify the desired path under the dasConf section as shown in the following example:
      "dasConf": {
        "hive.hook.proto.base-directory": "/warehouse/tablespace/external/hive/sys.db/query_data",
        "tez.history.logging.proto-base-dir": "/warehouse/tablespace/external/hive/sys.db"
  6. Click Apply Changes.