Upgrading Azure Single Server to Flexible Server

PostgreSQL version 11 databases used by Data Lakes and Data Hubs need to be upgraded to PostgreSQL 14. During this process, the Azure Single Server database will be upgraded to Azure Flexible Server. If there is a Private DNS Zone added to the environment, a Private Link will be created for the database and the Flexible Server will be private. Otherwise, a public Flexible Server will be created.

note

Single Server to Flexible Server upgrades for Data Lakes and Data Hubs have been temporarily deactivated because several Cloudera on Azure customers have observed and reported Flexible Server capacity limitations in their Azure subscriptions during the upgrade process and, currently, Cloudera is unable to programmatically check whether the required capacity is available in a customer's Azure subscription for a particular region.

As a next step, Cloudera is introducing a pre-upgrade validation that attempts to provision a PostgreSQL Flexible Server same-zone HA deployment before attempting the actual upgrade. This is intended to ensure that in case of capacity issues, the upgrade process can quickly return with a validation error that specifically outlines the problem.

Once Cloudera has implemented this validation, the Single Server to Flexible Server upgrade will be enabled again. If you see a capacity limitation related warning during the Flexible Server upgrade after this, see the steps described in Troubleshooting Flexible Server.

For the latest update on this issue see the corresponding Knowledge Article: Cloudera Customer Advisory 2024-812: Azure Flexible Server upgrade impacted by capacity limitations

This process describes how to upgrade a database used by a Data Lake or Data Hub to Azure Flexible Server. For a general description of Data Lake and Data Hub database upgrades, see Upgrading Data Lake/Data Hub database.

During the upgrade process, a backup is created from the Azure Single Server database, the server is deleted, and a new Azure Flexible Server is created with the same name. This way there is no need to change hostnames. However, the username format has changed between Single Server and Flexible Server. For details, see the Comparison table in the Microsoft Azure documentation.


Feature / Capability	Azure Database for PostgreSQL Single Server	Azure Database for PostgreSQL Flexible Server
Username in connection string	`<user_name>@server_name`. For example, `pgadmusr@mypgServer`.	Just username. For example, `pgadmusr`.

This change is automatically handled by Cloudera, however, as a result, Cloudera Runtime services using the database will be reconfigured during the upgrade process to use the new username format. Any third-party applications using the database would need to reflect this change by updating the username.

Limitations

If you are using a public Single Server and want to upgrade to a private Flexible Server, you need to add a Private DNS Zone to your environment first. For instructions on adding a Private DNS Zone, see Private DNS Zone.
If you would like to upgrade from Azure Single Server to a private Flexible Server, the upgrade to Flexible Server with delegated subnet is not available, as the Flexible Server with delegated subnet option is deprecated. It is recommended to use Flexible Server with Private Link instead.
If you are already using a private Flexible Server with a delegated subnet, it is not yet supported to migrate to a private Flexible Server with Private Link.
If you are performing a Data Lake resize because you want to upgrade to Cloudera Runtime 7.2.18 or later version, you must perform the upgrade of the database first.
If the server-side encryption (SSE) that the Data Lake or Data Hubs are encrypted with is configured with Customer Managed Keys (CMK), the upgrade is currently not available. This capability is expected to be released soon. For details on Customer Managed Keys, see Encrypting VM disks with customer managed keys and Data encryption with a customer-managed key in Azure Database for PostgreSQL - Flexible Server.

Prerequisites

When you are upgrading from Azure Single Server to Azure Flexible Server, new permissions are required. These permissions are being validated during the first phase of the upgrade. For a detailed list of prerequisites for Azure Flexible Server, see Azure prerequisites for Flexible Server.

important
There is a permission which is not required for enabling Flexible Server on a new environment but is specific to upgrading to Flexible Server from Single Server. Check the Azure prerequisites for Flexible Server even if you have enabled Flexible Server on your environment earlier.

If you have missed adding any of the required permissions and the upgrade to Flexible Server fails, you need to fix the permission issues on Azure side and then perform a retry through the CDP Public Cloud Management Console.
If you are upgrading from a public Single Server, make sure that port 5432 is open for outbound traffic. Otherwise the database upgrade might fail with a connection error. Public Azure Single Server access works with Service Endpoints, but these are not available for Flexible Server.
If you are using Cloudera Data Warehouse, stop the virtual warehouses and data catalogs associated with the environment.