Upgrading Spark 2 to Spark 3 for Cloudera Private Cloud 7.3.1

Upgrading Apache Spark from version 2 to version 3 in Cloudera Private Cloud is a process that involves:

Intermediate in-place cluster upgrade tasks, due to different support for Spark 3 versions of connectors in Cloudera versions.
Intermediate Spark application migration tasks, due to minor or maintenance Spark version changes.
Pre-application migration tasks.
Application migration from Spark 2 to Spark 3, due to major Spark version changes.
Post-application migration tasks.
In-place cluster upgrade tasks.

important
Cluster upgrade to version 7.3.1 is only allowed if all Spark 2 applications have been migrated to Spark 3 and tested in the source cluster.
Spark application migration tasks, due to minor or maintenance Spark version changes.

This table below provides links to the upgrade guides based on the version of Cloudera Private Cloud you're using, and the source versions of Spark in your environment.

Each upgrade guide contain all steps needed to upgrade Spark 2 to Spark 3 and perform the upgrade to Cloudera Private Cloud version 7.3.1.


Source cluster version	Source cluster Spark 2 version	Source cluster Spark 3 version	Documentation
7.1.9 SP1	2.4.8	None	Upgrade guide
	2.4.8	3.3.2 (CDS parcel)	Upgrade guide
	None	3.3.2 (CDS parcel)	Upgrade guide
7.1.8	2.4.8	None	Upgrade guide
	2.4.8	None	Upgrade (with connectors) guide
	2.4.8	3.3.0 or 3.3.2 (CDS parcel)	Upgrade guide
	2.4.8	3.3.0 or 3.3.2 (CDS parcel)	Upgrade guide (with connectors)
	None	3.3.0 or 3.3.2 (CDS parcel)	Upgrade guide
7.1.7 SP1	2.4.7	None	Upgrade guide
	2.4.7	None	Upgrade guide (with connectors)
	2.4.7	3.2.3 (CDS parcel)	Upgrade guide
	2.4.7	3.2.3 (CDS parcel)	Upgrade guide (with connectors)
	None	3.2.3 (CDS parcel)	Upgrade guide