Database upgrade known limitations and troubleshooting

Below are the known limitations associated with the database upgrade of Data Lake and Data Hubs and ways to troubleshoot them.

Known limitations and troubleshooting:

Performing the Database Upgrade on Runtime versions 7.2.6 or below
Cloudera has verified PostgreSQL version 11 compatibility for Runtime version 7.2.7 and above. There is no known reason why older Runtimes should not be compatible with PostgreSQL version 10.
Workaround: You can request an entitlement that allows the Database Upgrade to be performed on older Runtime versions on an exceptional basis.
Performing the Database Upgrade on Data Lakes with attached Data Hubs that cannot be stopped
Technically, the Database Upgrade can be performed on a Data Lake without stopping the attached Data Hubs. However, please be aware that during the upgrade, the Hive Metastore database will likely become temporarily unavailable and this can cause serious disruption or in the worst case can result in an inconsistent state for workloads running in Data Hubs or Data Services.
Workaround: If you acknowledge the risk and confirm that all cluster services and third party components relying on the Hive Metastore will be stopped for the time of the Database Upgrade, Cloudera can grant an entitlement that allows performing the upgrade with a running Data Hub cluster on an exceptional basis.
PostgreSQL client binaries will upgraded to version 11 on all clusters hosts
As part of the upgrade process we will try to install the PostgreSQL 11 libraries, pulling them from archive.cloudera.com. If the installation of these libraries does not succeed, a notification message will be sent that installation was attempted, but failed for some reason (network connectivity issues, etc).
Workaround: Follow the process to install the libraries manually, see Installing PostergeSQL 11 packages manually.
note
Failing to install the PostgreSQL 11 client libraries as part of the Database Upgrade process will cause the Data Lake backup and restore operations to stop working correctly.
Upgrading embedded databases
Data Hub clusters using an embedded database will not require the Database Upgrade operation to be performed. The embedded database, including client libraries will be automatically upgraded during an OS upgrade.
note
This capability is currently disabled and will be activated later.
Workaround: If you need to upgrade the embedded databases of your Data Hub clusters, contact Cloudera to enable this capability on an exceptional basis. Once this entitlement has been granted, your embedded databases can be upgraded by performing an OS upgrade.
Exceeding the End of Life deadline
Data Lake and Data Hub clusters that are not upgraded until November 10, 2022 will continue to run on a PostgreSQL version 10 instance of the underlying AWS, Azure or GCP database service. As this instance will be considered End-of-Life (EoL) by the respective Cloud Service provider, they may reserve the right to schedule an automated major version upgrade, resulting in a temporary downtime. In the case of extreme events the Cloud Service Provider may also stop the instance, see Versioning policy- Azure Database for PostgreSQL. In either case, your CDP Public Cloud workloads may be seriously impacted.
Workaround: Cloudera recommends performing the Database Upgrade via the CDP UI, or CLI as soon as possible.
Possibility of custom config reset after Database upgrade on AWS and Azure
During Postgres database upgrade for Data Lakes and Data Hubs there is a possibility that manually changed configs of the database server, that the control plane does not know about, will be reverted to the original configs.
Reason: On Azure the custom config can possibly reset during the database upgrade because Cloudbreak deletes and recreates the database server with the configs that the control plane knows about, so custom configs will be reverted.
On AWS if SSL enforcement is enabled then the database server uses a custom parameter group with the SSL enforcement settings (created by control plane) and if the customer made any custom changes to this custom parameter group then those changed will be reverted, because the database upgrade requires the recreation of the custom parameter group.