Upgrading Apache Spark 3.4.1 (bundled) on 7.2.18 SP2 to Spark 3 on 7.3.1
The following steps will help you upgrading from Apache Spark 2.4.8 (with 3.4.1 bundled) on Cloudera Public Cloud 7.2.18 SP2 to Spark 3.4.1 on 7.3.1.
Source cluster version | Source cluster Spark 2 version | Source cluster Spark 3 version | Target cluster version | Target cluster Spark 3 version | Spark 2 used with connectors1 |
---|---|---|---|---|---|
7.2.18 SP2 | none | 3.4.1 (bundled) | 7.3.1 | 3.4.1 | no |
In-place cluster upgrade
Final steps
After the upgrade and application migration are complete:
- Check the status of your Data Lakes, Data Hubs, and clusters.
- Perform benchmark testing on your applications. See Spark Application Migration.
1 Oozie, Solr, Phoenix, Hive Warehouse Connector, Spark Schema Registry