This section highlights the only new feature in HDP 2.2.8.
Spark 1.3.1 is the default Spark version with HDP 2.2.8. With a new HDP 2.2.8 cluster install, Spark 1.3.1 is installed. With the upgrade of an existing HDP cluster to 2.2.8 using Ambari, the Spark component is automatically upgraded from 1.2.1 to 1.3.1. For clusters not managed with Ambari, Spark can be manually upgraded from 1.2.1 to Spark 1.3.1; see the Optional Spark Manual Upgrade Procedure for instructions.
If you have upgraded to Spark 1.3.1 and want to downgrade to Spark 1.2.1, follow the Optional Spark Manual Downgrade Procedure. The downgrade to Spark 1.2.1 is only available as a manual step.
In HDP 2.2.8, Spark 1.3.1 is now available as a certified component.
Spark Core
Spark on YARN
Spark History Server
Spark on YARN on Kerberos-enabled clusters
Spark MLLib
Support for Hive 0.13.1, including the collect_list UDF