Cloudera Docs

Upgrading Apache Spark 2.4.8 (with 3.3.2 bundled) on 7.2.17 to Spark 3 on 7.3.1

The following steps will help you upgrading from Apache Spark 2.4.8 (with 3.3.2 bundled) on Cloudera Public Cloud 7.2.17 to Spark 3.4.1 on 7.3.1.

Source cluster version Source cluster Spark 2 version Source cluster Spark 3 version Target cluster version Target cluster Spark 3 version Spark 2 used with connectors1
7.2.17 2.4.8 3.3.2 (bundled) 7.3.1 3.4.1 yes

Intermediate in-place cluster upgrade

  1. Upgrade the cluster OS from Centos 7 to RedHat 8.
  2. Upgrade your cluster to Cloudera Public Cloud 7.2.18 SP2.
    1. Identify cluster version details.
    2. Identify your upgrade path.
    3. Review the prerequisites
    4. High-level upgrade steps.

    For more information on upgrading your cluster to 7.2.18 SP2, see:

1 Oozie, Solr, Phoenix, Hive Warehouse Connector, Spark Schema Registry