Apache Spark upgrades
Upgrading Apache Spark
Upgrading Apache Spark 2 to Spark 3
Upgrading from 7.1.9 SP1
Upgrade from Spark 2.4.8
Pre-application migration tasks
Spark application migration (from Spark 2 to Spark 3)
Post-application migration tasks
In-place cluster upgrade
Spark application migration (from Spark 3.x to Spark 3.4.1)
Final steps
Upgrade from Spark 2.4.8 (with CDS 3.3.2)
Application migration tasks from Spark 2 to Spark 3
Post-application migration tasks
In-place cluster upgrade
Spark application migration (from Spark 3.x to Spark 3.4.1)
Final steps
Upgrade from Spark 3.3.2 (CDS)
In-place cluster upgrade
Spark application migration (from Spark 3.x to Spark 3.4.1)
Final steps
Upgrading from 7.1.8
Upgrade from Spark 2.4.8
Pre-application migration tasks
Application migration tasks (Spark 2 to Spark 3)
Post-application migration tasks
In-place cluster upgrade
Application migration tasks (Spark 3.x to Spark 3.4.1)
Final steps
Upgrade from Spark 2.4.8 (with connectors)
Intermediate in-place cluster upgrade
Upgrade from Spark 2.4.8 (with CDS 3.3.x)
Application migration tasks (from Spark 2 to 3)
Post-application migration tasks
In-place cluster upgrade
Application migration tasks (from Spark 3.x to 3.4.1)
Final steps
Upgrade from Spark 2.4.8 (with CDS 3.3.x and connectors)
Intermediate in-place cluster upgrade
Upgrade from Spark 3.3.x (CDS)
In-place cluster upgrade
Application migration tasks (Spark 3.x to Spark 3.4.1)
Final steps
Upgrading from 7.1.7
Upgrade from Spark 2.4.7
Pre-application migration tasks
Application migration tasks (Spark 2 to 3)
Post-application migration tasks
In-place cluster upgrade
Application migration tasks (Spark 3.x to 3.4.1)
Fianl steps
Upgrade from Spark 2.4.7 (with connectors)
Intermediate in-place cluster upgrade
Upgrade from Spark 2.4.7 (with CDS 3.2.3)
Application migration tasks (Spark 2 to 3)
Post-application migration tasks
In-place cluster upgrade
Application migration tasks (Spark 3.x to 3.4.1)
Final steps
Upgrade from Spark 2.4.7 (with CDS 3.2.3 and connectors)
Intermediate in-place cluster upgrade
Upgrade from Spark 3.2.3 (CDS)
In-place cluster upgrade
Application migration tasks (Spark 3.x to 3.4.1)
Final steps
Upgrades with Spark 3.4.1 (bundled)
Upgrade from Spark 2.4.8 (with 3.4.1 bundled)
Application migration tasks (Spark 2 to 3)
Post-application migration tasks
In-place cluster upgrade
Final steps
Upgrade from Spark 3.4.1 (bundled)
In-place cluster upgrade
Final steps
Upgrading Apache Spark 3.3.x or 3.4.x to Spark 3.5.4
Upgrade Spark 3.3.2 (CDS) to 3.5.4
Activate CDS 3.5.4 parcel
In-place cluster upgrade
Spark application migration
Final steps
Upgrading Apache Spark 3.4.1 to 3.5.4
Spark application migration
In-place cluster upgrade
Final steps
Migrating Apache Spark applications
Java versions
Scala versions
Python versions
Spark commands
Spark connectors
Logging
Third-party libraries
Spark behavior changes
Migrating Apache Spark applications
Migrating Apache Spark applications
Migrating Apache Spark applications
Apache Spark Migration guides
Spark 2 to Spark 3 workload refactoring
Unsupported features
Post-migration checklist
Benchmark testing
Troubleshooting