Fixed Issues in Spark
Review the list of Spark issues that are resolved in Cloudera Runtime 7.2.17.
- CDPD-55243: Fix case sensitivity of Iceberg's CachingCatalog
- Previously, using inconsistent casing for database and table names of Iceberg tables in queries can lead to Spark reading a stale cached snapshot after a write to the table (append, update, delete) in the same Spark session. Now the cache is insensitive to the case of database and table names and is always refreshed on a write in the session.
- CDPD-50862: Extend usage of fire_listener_event API to HS2/Spark to generate events on DML queries
- The data written from Spark will now generate an "Insert" event in the HMS notification log table. This is useful for external services (listeners) like Impala to know the current condition of an external table.
- CDPD-46530: Spark - Upgrade Apache Ivy to 2.5.1 due to CVE-2022-37865, CVE-2022-37866
- Apache Ivy upgraded to 2.5.1 to avoid CVE
- [SPARK-39505][UI] Escape log content rendered in UI
Apache patch information