Fixed Issues in Cloudera Runtime 7.2.18.300

CDPD-73217: Backport 'add security-related HTTP headers'

Security-related HTTP headers are now added to the Kudu embedded webserver to comply with security scanner requirements.

CDPD-72776: Regression: Hive select like query fails for Parquet table

There was an issue caused by Parquet-Hadoop version used by newly introduced rest catalog service. This issue is now resolved by correcting the Parquet-Hadoop version.

CDPD-72180: Calcite build failure in cdpd-master

Upgraded the vlsi-release-plugins to 1.90 and the earlier version was missing from the repository.

CDPD-72008: SMM UI - Upgrade node.js to 22.4.1/20.15.1/18.20.4 due to multiple CVEs

Upgraded the Node.js version in the Streams Messaging Manager UI to 20.15.1, due to CVE-2024-27980, CVE-2024-22020, CVE-2024-36137, CVE-2024-22018 and CVE-2024-37372.

CDPD-71847: Fix KConnect openapi descriptor file path

The Kafka Connect openapi descriptor file path is now fixed. An output format modification was necessary to publish Kafka Connect REST API references in JSON format. Kafka's build configuration is also modified to receive this newly added JSON formatted artifacts.

CDPD-71639: [7.2.18.300 CLONE] - Policy Engine initialization failed due to NPE

When policy deltas were enabled, and there was no material change in policy-set after the previous policy download processed by the Ranger admin, the ServicePolicies object downloaded contained null values instead of an empty list.

As the plugin considers empty-list value differently than null value, the policy-engine built by the plugin incorrectly reflects the existing policy-set, leading to incorrect authorization results.A material change to policy-set indicates that there is at least one policy added/deleted/updated to previous policy-set.

This issue is now resolved the policyDelta attribute is annotated in ServicePolicies and SecurityZone class with@JsonSerialize(include=JsonSerialize.Inclusion.NON_NULL)

CDPD-71580: workaround needed for Bootbox due to CVE-2023-46998

Upgraded the Bootbox.js library due to CVE-2023-46998.

CDPD-71508: Backport HBASE-28596 Optimise BucketCache usage upon regions splits/merges.

A new configuration property hbase.rs.evictblocksonsplit is now added, with the default value set to true, to optimise BucketCache usage upon regions splits/merges.

CDPD-71447: Audit to s3 is failing for Kafka

Kafka plugin needs AWS V2 SDK bundle on the classpath to push the audits to s3.

CDPD-71358: [7.2.18.300] Temporarily disable the tasks tab on Entity Detail page

The Entity detail page displayed Something went wrong because, on loading the Entity detail page, an API call (/api/atlas/admin/tasks) was made to get all the tasks that were created when deferred actions features were enabled. This issue is now resolved. The Entity detail page task tab and task API are now displayed in the UI depending on the server side property atlas.tasks.ui.tab.enabled. Previuosly, it was set to false, temporarilydisabling the task tab on Entity detail page in UI.

CDPD-71309: Enhance the audit generated in Ranger during data discovery call from REST Catalog API

The audit generated in Ranger during data discovery call from the REST Catalog API is now enhanced. Calls such as list Databases / ListTables did not have the correct access Types and are enhanced to provide details on the operation.

CDPD-71294: PARQUET-2498 Hadoop vector IO API doesn't handle empty list of ranges

Hadoop VectorIO API could not handle empty list of ranges and were rejected. This issue is now resolved.

CDPD-71293: HADOOP-19204. VectorIO regression: empty ranges are now rejected

The validation in VectorIO now rejects a read vectored with an empty range, whereas before it was a no-op (no-operation).

CDPD-71255: Backport IMPALA-12580 to 7.2.18.200

Previously, predicates were not pushed down to Impala scanners if they were already applied by Iceberg and no further rows were filtered. This issue is now resolved and a subset of the predicates are now pushed down to Impala Scan nodes.

CDPD-71193: Add backend config to restrict data file locations for Iceberg tables

A backend flag iceberg_restrict_data_file_location is now added. When the flag is set to true, Impala raises an error when at least one data file of an Iceberg table is outside of the table directory.The default value of the flag is true.

CDPD-70951: Hive - Upgrade Aircompressor to 0.27 due to CVE-2024-36114

Upgraded the Aircompressor version to 0.27 due to CVE-2024-36114.

CDPD-70908: IMPALA-12552 impala-shell should not call encode on kerberos_host_fqdn in python 3 env

Fixed a Kerberos authentication issue in the Impala-shell, that was experienced in Python3 environment when using the kerberos_host_fqdn option.

CDPD-70336: Disable basic auth for /api/atlas/admin/prometheus

Basic authorization is now disabled for Prometheus API to enable CDL to scrape metrics data.

CDPD-70053: Ranger - Upgrade Commons-configuration2 to 2.10.1 due to CVE-2024-29133 and CVE-2024-29131

Upgraded the Commons-configuration2 version to 2.10.1 due to CVE-2024-29133 and CVE-2024-29131.

CDPD-69333: PARQUET-2171: Support Hadoop vectored IO -final merged PR

Added a new feature called Vectored IO in Hadoop for improving read performance for seek heavy readers.

CDPD-68793: Hadoop - Upgrade Kafka Clients due to CVEs

Upgraded the Kafa Clients due to CVE-2023-25194, CVE-2021-38153 and CVE-2018-17196.

CDPD-67834: Hive - Upgrade Nimbus-JOSE-JWT to 9.37.3 due to CVE-2023-52428

Upgraded Nimbus-JOSE-JWT version to 9.37.3 due to CVE-2023-52428.

CDPD-67711: We are unable to access AFBS folder in Hue

Previously, the URL parameters were encoded only for small set of use-cases. But the parameters must be encoded always to cover all use-cases. This issue is now resolved and the

the
                                    _make_url

method of HttpClient class is overrid and its UrlEncode method is changed to use quote() method instead of the default


                                    quote_plus()

. This also fixed the scenarios of whitespaces present in the path that regressed after the above change.

CDPD-67570: Exception during re-analyze can be lost

Impala now displayes a meaningful error message when it faces an exception during the re-analyze phase.

CDPD-67514: Enhance UGI for group look up for the external user in data sharing environment

Enhanced the User Group Information (UGI) to do group look up for the external users in data sharing environment.

CDPD-67341: Refactor and improve IDBroker support in Hue

Refactored the IDBroker support and more preference is now given to Ranger Authorization Service (RAZ) when both are configured in Hue. Improved IDBroker HA code section to switchover to healthy instance correctly and not depend only on the first one for every scenario. This fix also improves Hue page loading performance.

CDPD-67224: Ozone - Upgrade Spring Framework to 6.1.6/6.0.19/5.3.34 due to CVE-2024-22243, CVE-2024-22259 and CVE-2024-22262

Upgraded the Spring Framework to 5.3.34 due to CVE-2024-22243, CVE-2024-22259 and CVE-2024-22262.

CDPD-67114: [7.2.18] Backport KAFKA-13988: Mirrormaker 2 auto.offset.reset=latest not working

The auto.offset.reset=latest configuration was not working in the Streams Replication Manager (SRM). This issue is now resolved.

CDPD-60267: Backport HIVE-27595 to CDP

Fixed slow filtering on Hive/HMS for large number of tables that used cartesian-product table filtering by sort + binary search.

CDPD-60257: REST API for Hive Metastore

Iceberg provides a REST catalog implementation that allows other query engines to integrate with Iceberg tables. A compatible REST implementation I snow provided for Hive Metastore (HMS) for the tables hosted in HMS that allow non-thrift-speaking other engines to integrate with HMS.

CDPD-31172: Hive: Intermittent ConcurrentModificationException in HiveServer2 during mondrian testset

Fixed an exception by using ConcurrentHashMap instead of HashMap to avoid the race condition between threads occurring because of concurrent modification of PerfLogger endTimes/startTimes maps.