Upgrading from Spark 2 to Spark 3

If you have a Cloudera Data Engineering Data Hub cluster, during the Cloudera Runtime upgrade process to version 7.3.1, you need to remove Spark 2 while creating a new cluster with Spark 3 and migrate your data. Depending on your environment and Spark applications, besides the cluster upgrade tasks, you need to perform Spark application migration tasks and sidecar migration tasks for the Cloudera Data Hub cluster in multiple steps. For more information, see Upgrading Apache Spark.