Upgrade Spark3 within Oozie
In Cloudera Private Cloud Base, Spark 3 is not included in the CDH parcel but is available as a separate parcel. This allows independent upgrades of Spark 3.
When upgrading the Spark 3 parcel, you need to update the Spark 3 distribution in Oozie's Spark 3 ShareLib since Oozie's Spark 3 ShareLib is created during the Oozie build process.
The setting should specify the name of the folder for the Spark 3 parcel to use,
located at the root directory of parcels, typically
default, it is set as
SPARK3, which is a symbolic link to the currently
activated Spark 3 parcel. You can also specify the folder name of another downloaded and
distributed Spark 3 parcel that is not currently activated. For instance,
SPARK3-220.127.116.11.4.7190.0-174-1.p0.42371348. Cloudera recommends to change the
default setting only if you encounter any problem with the currently activated Spark 3 parcel in
Oozie. While your issue is being investigated, you can temporarily use a different version of
Spark 3 with Oozie by modifying this setting.