Cloudera Data Engineering upgrade version compatibility
Cloudera Data Engineering provides the in-place upgrade option from the eligible versions to the latest version. If you are on a version that is ineligible for in-place upgrade, the in-place upgrade option is disabled and you must use the side-by-side upgrade with the dex-upgrade-utils Docker image.
Checking your upgrade options
In-place upgrades are supported from Cloudera Data Services on premises 1.5.5 SP3 including cumulative hotfixes such as 1.5.5 SP3 CHF3 to 1.5.5 SP4. Services running earlier releases are not eligible for in-place upgrades and must use a side-by-side upgrade with the dex-upgrade-utils Docker image. For all supported upgrade paths, see Cloudera Data Services on premises upgrade documentation.
In-place upgrades
To perform an in-place upgrade, your Cloudera Data Services on premises deployment requires an eligible release. The following table lists all releases eligible for upgrade to the latest release.
| Supported upgrade versions | Target version |
|---|---|
|
Cloudera Data Services on premises 1.5.5 SP3 and later 1.5.5 SP3 hotfixes, for example 1.5.5 SP3 CHF3 |
Cloudera Data Services on premises 1.5.5 SP4 |
Side-by-side upgrade for ineligible releases
If your version does not support in-place upgrades, use the side-by-side upgrade with the dex-upgrade-utils Docker image. With this method, you create a new Cloudera Data Engineering service on the upgraded platform and migrate your jobs, resources, and job run history. For details, see Migrating the Cloudera Data Engineering endpoint and Post-upgrade tasks for the Cloudera Data Engineering endpoint.
Runtime component version behavior after upgrading
To maintain backwards compatibility, the in-place upgrade does not modify the image type. Virtual Clusters that use the older RedHat (deprecated) images continue to use them after the upgrade. To use the new security hardened image after the upgrade, you must create new Virtual Clusters and select the new image. For more information on the supported runtime components, see Compatibility for Cloudera Data Engineering and Runtime components.
Upgrade precheck for version compatibility
As part of the upgrade preparation workflow, a check has been introduced to ensure that the Virtual Clusters (VCs) use a Spark version that is compatible with the Cloudera Base on premises (Cloudera Runtime) version. If the Spark version used for the VCs is incompatible with the Cloudera Runtime version, the upgrade preparation fails. For example, if the Cloudera Runtime version is upgraded from 7.1.9 SP1 to 7.3.1, and any of the VCs use a Spark version lower than 3.5, the upgrade preparation fails.
