What's New in Apache Spark

Learn about the new features of Spark in Cloudera Runtime 7.2.16.

Support for Iceberg

Spark 3 now supports Iceberg.

Updated Spark 3 version of CDP

Spark 3 version of CDP stack was updated to 3.3.

Apache Spark 3 version support

  • Support for virtual clusters powered by Apache Spark 3 is now available.
  • The following functionalities are not currently supported:
    • Deep analysis (visual profiler)
    • HWC - that is, Hive managed ACID tables (Direct Reader & JDBC mode)
    • Phoenix Connector
    • SparkR

See Running Apache Spark 3 applications and Data Engineering clusters.