Fixed Issues in Cloudera Runtime 7.2.18.400

You can review the list of reported issues and their fixes in Cloudera Runtime 7.2.18.400.

CDPD-73488: Upgrade axios library version from 1.7.2 to 1.7.4 in Ranger Admin React JS for CVE-2024-39338

Upgraded Axios library version from 1.7.2 to 1.7.4 in Ranger Admin React JS due to CVE-2024-39338.

CDPD-73423: Ranger - Upgrade Spring Framework to 6.1.12/6.0.23/5.3.39 due to CVE-2024-38808 and CVE-2024-38809

Upgraded Spring-Framework version to 5.3.39 due to CVE-2024-38808 and CVE-2024-38809.

CDPD-73326: Reduce memory needed to create Ranger policy engine

An issue led to the creation of multiple RangerResourceMatchers with identical resource specification. This issue is now resolved and the creation of multiple RangerResourceMatcher objects is now avoided by maintaining a cache of them in the RangerPluginContext object associated with the Ranger policy engine, thereby reducing policy engine's memory needs.

CDPD-73282: Backport CALCITE-6530 HTTP Sessions are never expired in Avatica server

The http sessions created by the Avatica server did not expire and this caused the Avatica server to run out of memory. This issue is now resolved.

CDPD-73217: Requirement to add security-related HTTP headers

Security-related HTTP headers are now added to the Kudu embedded webserver to comply with security scanner requirements.

CDPD-73147: [Ranger React UI] Admin audits for "Import Delete" operation type do not display service name field

In the Ranger React UI, in admin audits, the Service name field was missing for the audits of operation type Import Delete. This issue is now resolved and the Import Delete policy logs now display the service name.

CDPD-73144: Enhance trie to support processing of evaluators during traversal

Ranger policy engine uses trie data structure to organize resources for faster retrieval of policies/tags/zones associated with a given resource. When a resource consists of multiple elements, such as, database/table/column, many trie instances are consulted to retrieve policies/tags/zones associated with the resource.

Such multi-trie retrieval is optimized with a 2-pass traversal - first pass to get count and the second pass to get the actual objects. Thherefore, the trie data structure used in Ranger policy engine is now updated to support processing of evaluators during traversal.

CDPD-72555: Ranger react UI some modules shown hardcoded time zone string "Indian Standard Time"

Removed the hardcoded Indian Standard Time string and added time zone base dynamic string.

CDPD-72536: Backport HBASE-28724 BucketCache.notifyFileCachingCompleted may throw IllegalMonitorStateException

When the prefetch thread completed reading the file blocks faster than the bucket cache writer, threads were able to drain it from the writer queues. And BucketCache.notifyFileCachingCompleted displayed the IllegalMonitorStateException error. This issue is now resolved.

CDPD-72522: IMPALA-12582 Executors crash during runtime filter generation

Impala executors stopped responding when generating MIN_MAX RuntimeFilters for certain queries, due to an out-of-bounds access to input_vals in the ScalarFnCall::InterpretEval() function.

The issue is now resolved by ensuring the ScalarExprEvaluator properly invokes the Open() function, preventing the out-of-bounds access and stabilizing the RuntimeFilter generation process.

CDPD-72347: Backport SPARK-48946

There was a Null Piint Exception (NPE) in DataSourceV2ScanExecBase redact method when the session was null. This issue is now resolved.

CDPD-72180: calcite build failure in cdpd-master

Upgraded the vlsi-release-plugins to 1.90 and the earlier version was missing from the repository.

CDPD-72149: Upgrade requireJS due to CVE-2024-38998 and CVE-2024-38999

Upgraded the RequireJS version due to CVE-2024-38998 and CVE-2024-38999.

CDPD-72059: org.apache.spark.sql.catalyst.parser.ParseException: [PARSE_SYNTAX_ERROR]

There was a ParseException with the message Syntax error at or near end of input in PySpark when using the listTables() method. This occurred after upgrading to Spark 3.4.1 from Spark 3.3.1. This issue is now resolved.

CDPD-71959: Backport HBASE-28463 to 7.2.18.x branch.

A new feature of time-based data tiering is now introduced in HBase to optimize storage efficiency and access performance by segregating data based on its recency. By keeping recent data in the bucket cache (backed by faster storage types like SSDs) and evicting older data, the system aims to provide a more flexible control over the cache allocation and eviction logic through configuration, allowing to define time priorities for cached data.

CDPD-71931: Ranger - Upgrade commons-compress to 1.26.0 due to CVE-2024-25710 and CVE-2024-26308

Upgraded the Commons-Compress version to 1.26.0 due to CVE-2024-25710 and CVE-2024-26308.

CDPD-71764: XSS vulnerability in Zeppelin : Unsanitized HTML in Markdown Paragraphs

To enhance security, Zeppelin now integrates HTML sanitization using JSoup within the markdown interpreter. This ensures that any HTML embedded in markdown is sanitized according to a configurable blacklist.

CDPD-71709: Pagination on the Ranger Admin - Plugin Status page

Added Pagination in the Ranger Admin Plugin Status page.

CDPD-71703: RANGER-4737: The inactivityTimeout is getting reset when user updates its profile from UserProfile page

In Ranger Admin with React JS, the inactivityTimeout was getting reset to a default value of 15 minutes only when the user updated the profile from UserProfile page. This issue is now resolved.

CDPD-71508: Backport HBASE-28596 Optimise BucketCache usage upon regions splits/merges.

A new configuration property hbase.rs.evictblocksonsplit is now added, with the default value set to true, to optimise BucketCache usage upon regions splits/merges.

CDPD-71447: Audit to S3 is failing for kafka

Kafka plugin needs AWS V2 SDK bundle on the classpath to push the audits to S3.

CDPD-71309: Enhance the audit generated in Ranger during data discovery call from REST Catalog API

The audit generated in Ranger during data discovery call from the REST Catalog API is now enhanced. Calls such as list Databases / ListTables did not have the correct access Types and are enhanced to provide details on the operation.

CDPD-71279: Proposal to Upgrade All React.js Dependent Libraries

Upgrade react.js related library.

CDPD-70952, CDPD-70950: Iceberg - Upgrade Aircompressor to 0.27 due to CVE-2024-36114

Upgraded the Aircompressor version to 0.27 due to CVE-2024-36114.

CDPD-69700: Ranger - remove jwtprovider-knox dependency due to CVE

Removed Knox jwt support from Ranger Client due to a CVE.

CDPD-69400: Need Virtual Group for Default Group

Extended the current virtual group syntax and implementation in Knox to allow the creation of a Unix primary group for an authenticated user. Thereby, creating a virtual group with the same name as the user.

CDPD-69039: Metastore schema version compatibility error during upgrade setup

The cluster creation process was failing with a Metastore schema version is not compatible error during the upgrade, but this issue is now resolved.

CDPD-68950: [DLM] REST API support for interacting with DLM service

The Data Lifecycle Management Service (DLM) now has a user-facing API that allows various personas to perform different things such as, creating new policies/associating tables to policies, deleting policies, executing adhoc action on a table, monitoring running jobs etc.

CDPD-67597: Hive - Upgrade PostgreSQL to Address CVE-2024-1597 vulnerability

Upgraded the PostgreSQL versions 42.5.5, 42.6.1, and 42.7.2 to address CVE-2024-1597, which involves a SQL injection vulnerability.

CDPD-66968: Enhance IDBroker API to create down scoped permission / policy used in cloud access token

Enhanced the IDBroker API to create down scoped permission / policy used in cloud access token.

CDPD-66915: Livy3 server logs are missing due to reload4j on classpath

Excluded reload4j from dependencies for Spark 3.3+.

CDPD-66797: Skip showing 'Page not found' for wrong value is provided to a API parameter in Login Session Tab

From server side the API used in Audit Login Sessions Tab -/service/xusers/authSessions added a validation to requestIP API query parameter.

When a user enters a text value, a page not found error message was displayed. This issue is now resolved and the server-side response is displayed as an alert on Login Session Tab.

CDPD-66795: Skip showing 'Page not found' page for INVALID_INPUT_DATA validation in User Profile

When an invalid form value is provided during profile update, the Ranger React UI displays Page not found message. This issue is now resolved and the server-side response is displayed as an alert on User Profile window.

CDPD-66783: Update the execution of setServiceDef call in App.jsx

Updated the execution of setServiceDef call in App.jsx.

CDPD-66780: Audit logs for Masking policy is missing data mask type entry

Audit logs for Masking policy was missing data mask type entry. This issue is now resolved and UI label regression is now fixed.

CDPD-66401: [Ranger React UI] Audit UI improvements with respect to values overflowing into other columns

In the Ranger react UI, in the audits, if the length of certain fields was long, the value was overflowing into other columns. This issue is now resolved and the values are clipped in the audit display tables.

CDPD-66395: HMS Iceberg REST Catalog enhancements to support OAuth2 Flow

Extended the existing TokenResource for KNOXTOKEN service to include OAuth specifics such as expected URL, error messages and flows to support Token Exchange Flow and Token Refresh.

CDPD-66271: Updating the "Something went wrong" page in Ranger React UI

If there was an error or code break in the Ranger react file, the Something went wrong error message was displayed. This issue is now resolved and buttonns are added for reloading and go to profile page.

CDPD-66095: Checkbox selection issue when clicking on permission label in tag-based permissions policy

There was an inconsistent behaviour in the selection of checkbox when clicking on permission label in tag-based permissions policy.For example, when HDFS, HIVE was selected and the permission was selected by clicking the permission label such as read/write, it was observed that any change in permission for HIVE was impacted on HDFS permission selection also. This issue is now resolved.

CDPD-65923: Audit logs for Mask Row policy does not show policy condition under policy item

Policy condition is now displayed under policy item for Mask & Row policy Audit logs.

CDPD-64854: Backport of RANGER-4513

There was an issue on the Policy Listing page where, an unexpected reset to Access tab occurred when attempting to filter the service and zone dropdown options. This issue is now resolved.

CDPD-64849: Optimize policy listing loader after session timeout and Audit Admin session ID modal loader

After sa ession timeout, when navigated to the Policy Listing page, the Something went wrong error message was displyed for a fraction of seconds. Also, in the Audit admin session Id modal, the loader was not in sync. These issues are now resolved and the loader logic in both above the scenarios is now improvised.

CDPD-64845: Optimize "plugins/definitions" API Call for Initial Load in Multiple Ranger-React Modules

In Ranger React, the "plugins/definitions" API call was implemented at the initial load for optimization. This optimization was implemented only on the Service Manager page and is now extended to modules such as, Audit, Report, Security Zone and Key Manager.

This enhancement aims to improve the initial load performance by efficiently utilizing the "plugins/definitions" API call across multiple modules within Ranger-React.

CDPD-63092: Avro - CVE-2023-39410

Upgraded the Avro version to 1.11.3 due to CVE-2023-39410.

CDPD-60845: Unable to write data to the non-default database using HWC.

Due an issue, data could not be written to the non-default database using Hive Warehouse Connector (HWC). This issue is now resolved.

CDPD-60505: "Select All permissions for all components." checkbox missing in tag based policy permission popup

In the permissions selector popup for tag based policies in the Backbone UI, there is a checkbox that allows users to select all permissions for all components selected. But in React UI, this checkbox was missing. This issue is now fixed.

CDPD-58846, CDPD-58844: Spark3 - Upgrade Janino to 3.1.10 due to CVE-2023-33546

Upgraded Janino version to 3.1.10 due to CVE-2023-33546.