HDP-2.4.0 Release Notes
Also available as:
PDF

Spark

HDP 2.4.0 provides Spark 1.6.0 and the following Apache patches:

  • SPARK-11314: Add service API and test service for Yarn Cluster schedulers.

  • SPARK-11315: Add YARN extension service to publish Spark events to YARN timeline service.

  • SPARK-11323: Add History Service Provider to service application histories from YARN timeline server.

  • SPARK-12417: Support to have ORC bloom filters during write code path.

  • SPARK-12898: Consider having dummyCallSite for HiveTableScan.

  • SPARK-12925: Improve HiveInspectors.unwrap for StringObjectInspector.getPrimitiveWritableObject.