Upgrading Apache Spark 2.4.8 (with 3.3.2 bundled) on 7.2.17 to Spark 3 on 7.3.1

The following steps will help you upgrading from Apache Spark 2.4.8 (with 3.3.2 bundled) on Cloudera on cloud 7.2.17 to Spark 3.4.1 on 7.3.1.


Source cluster version	Source cluster Spark 2 version	Source cluster Spark 3 version	Target cluster version	Target cluster Spark 3 version	Spark 2 used with connectors¹
7.2.17	2.4.8	3.3.2 (bundled)	7.3.1	3.4.1	yes

¹ Oozie, Solr, Phoenix, Hive Warehouse Connector, Spark Schema Registry

Intermediate in-place cluster upgrade

Upgrade the cluster OS from Centos 7 to RedHat 8.
Upgrade your cluster to 7.2.18 SP2.
1. Identify cluster version details.
2. Identify your upgrade path.
3. Review the prerequisites
4. High-level upgrade steps.
For more information on upgrading your cluster to 7.2.18 SP2, see:
- Upgrading to 7.2.18
Once complete, continue the upgrade with the steps described in Upgrading Apache Spark 2.4.8 (with 3.4.1 bundled) on 7.2.18 SP2 to Spark 3 on 7.3.1.