Review the list of Spark issues that are resolved in Cloudera Runtime
7.3.2, its service packs and cumulative hotfixes.
Cloudera Runtime 7.3.2
Cloudera Runtime 7.3.2 resolves Spark issues and incorporates fixes from
the service packs and cumulative hotfixes from 7.3.1.100 through 7.3.1.706. For a
comprehensive record of all fixes in Cloudera Runtime 7.3.1.x, see Spark Fixed Issues.
- CDPD-90482: Jersey Library update
- 7.3.2.0
- Updated the Jersey suite of libraries to versions 2.0 and above to ensure JDK 17 compatibility.
- CDPD-82979: Added Cloudera-specific configuration for
MapPartitionsRDD default determinacy
- 7.3.2.0
- To avoid failures during error handling, a Cloudera-specific configuration was added to change the default determinacy level for any
MapPartitionsRDD.
- CDPD-80239: Fixed handling of non-deterministic SQL expressions
- 7.3.2.0
- Fixed a bug related to intermittent data corruption and record loss during task failures. Spark now honors the
deterministic=false attribute for non-deterministic data types.
- CDPD-67187: Fixed missing JVM class imports for Spark
- 7.3.2.0
- JVM classes are now imported during session initialization.
- OPSAPS-74346, OPSAPS-74346, OPSAPS-74316: Enhancing Spark security
- 7.3.2.0
- Changed the generation of the
spark.yarn.historyServer.address value to use the HTTPS address when SSL/TLS is enabled. The spark3.network.crypto.enabled new configuration property is now available to enable AES-based encryption.
- OPSAPS-72254: UCL | FIPS Failed to upload Spark example jar to HDFS in cluster mode
- 7.3.2.0
Fixed an issue with deploying the Spark 3 Client Advanced Configuration Snippet (Safety Valve) for spark3-conf/spark-env.sh.
For more information, see New Cloudera Manager configuration parameter spark_pyspark_executable_path has been added to the Livy for Spark 3 in Behavioral Changes In Cloudera Manager 7.13.2.
- OPSAPS-76539: Cloudera Manager UI allowed adding multiple Spark 3 service instances within a single cluster
- 7.3.2.0
- Prevented multiple Spark 3 service instances within a single cluster by implementing a
maxInstances flag in Cloudera Manager.
- OPSAPS-74281: Disabled the live Spark UI when the
ENCRYPT_ALL_PORTS feature flag is enabled
- 7.3.2.0
Enhanced the security of Spark by implementing new default configuration settings:
spark.ui.enabled=false: preventing initiation of an HTTP service that can be accessed from external hosts.spark.io.encryption.keySizeBits=256: the default Spark keySizeBits has been increased from 128 to 256
