This section highlights several new features in HDP 2.3.2.

Spark 1.4.1 is the default Spark version with HDP 2.3.2. With a new HDP 2.3.2 cluster install, Spark 1.4.1 is installed. With the upgrade of an existing HDP cluster to 2.3.2 using Ambari, the Spark component is automatically upgraded to 1.4.1. For clusters not managed with Ambari, Spark can be manually upgraded from 1.2.1 or 1.3.1 to Spark 1.4.1; see the Optional Spark Manual Upgrade Procedure for instructions.

If you have upgraded to Spark 1.4.1 and want to downgrade to Spark follow the Optional Spark Manual Downgrade Procedure. The Spark downgrade is only available as a manual step.