Upgrading Apache Spark 2 to Spark 3

Upgrading Apache Spark from version 2 to version 3 on CDP Private Cloud 7.1.9 is a process that involves:

1. Installing and activating the CDS 3.3 parcel and Spark 3 service.
2. Application migration from Spark 2 to Spark 3, due to major Spark version changes.
3. Post-application migration tasks (optional decommissioning of Spark 2 services).
4. Final verification and benchmark testing.

The table below provides links to the upgrade guides based on the source versions of Spark in your environment.

Source cluster version	Source cluster Spark 2 version	Source cluster Spark 3 version	Documentation
7.1.9 SP1	2.4.8	None	Upgrade guide
7.1.9 SP1	2.4.8	3.3.2 (CDS parcel)	Upgrade guide