Fixed Issues in Apache Spark

Review the list of Spark issues that are resolved in Cloudera Runtime 7.1.8.

CDPD-2650: Spark can't write ZSTD and LZ4 compressed Parquet to dynamically partitioned table.
CDPD-3783: Unable to create database in spark.
CDPD-30201: Fixed "Event Timeline" expansion in SparkUI.
clicking "Event Timeline" in SparkUI works as expected.
CDPD-28196: Spark 3 History Server link Resource Manager UI works again.
CDPD-27961: Insert overwrite no longer fails due to partition directory double delete.
CDPD-27528: Apache Spark version has been updated to 2.4.8 version.

Apache patch information

  • SPARK-38992