Upgrading Apache Spark
Upgrading Spark 2 to Spark 3 for Cloudera Public Cloud 7.3.1
Upgrading from 7.2.18
Upgrade from Spark 2.4.8
Sidecar migration of Data Hub clusters
Application migration tasks (Spark 2 to 3)
Post-application migration tasks
In-place cluster upgrade
Final steps
Upgrade from Spark 2.4.8 (with 3.4.1 bundled)
Sidecar migration of Data Hub clusters
Application migration tasks (Spark 2 to 3)
Post-application migration tasks
In-place cluster upgrade
Final steps
Upgrade from Spark 3.4.1 (bundled)
In-place cluster upgrade
Final steps
Upgrading from 7.2.17
Upgrade from Spark 2.4.8
Intermediate in-place cluster upgrade
Sidecar migration of Data Hub clusters
Application migration (Spark 2 to 3)
Post-application migration tasks
In-place cluster upgrade
Application migration tasks (Spark 3.x to 3.4.1)
Final steps
Upgrade from Spark 2.4.8 (with connectors)
Intermediate in-place cluster upgrade
Upgrade from Spark 2.4.8 (with 3.3.2 bundled)
Intermediate in-place cluster upgrade
Intermediate application migration tasks
Sidecar migration of Data Hub clusters
Application migration tasks (Spark 2 to 3)
Post-application migration tasks
In-place cluster upgrade
Application migration tasks (Spark 3.x to 3.4.1)
Final steps
Upgrade from Spark 2.4.8 (with 3.3.2 bundled and connectors)
Intermediate in-place cluster upgrade
Upgrade from Spark 3.3.2
In-place cluster upgrade
Application migration tasks (Spark 3.x to 3.4.1)
Final steps
Migrating Spark applications
Java versions
Scala versions
Python versions
Spark commands
Spark connectors
Logging
Third-party libraries
Spark behavior changes
Apache Spark Migration guides
Spark 2 to Spark 3 workload refactoring
Unsupported features
Post-migration checklist
Benchmark testing
Troubleshooting