Fixed issues in 7.1.9 SP1 CHF 2

Know more about the cumulative hotfix 2 for 7.1.9 SP1. This cumulative hotfix was released on 30 Sep, 2024.

Following are the list of fixes that were shipped for CDP Private Cloud Base version 7.1.9-1.cdh7.1.9.p1010.57518810.

COMPX-17702: Backport - YARN-10345 - HsWebServices containerlogs does not honor ACLs for completed jobs
Previously, the following rest APIs did not have any authorization:
  • /ws/v1/history/containerlogs/{containerid}/{filename}
  • /ws/v1/history/containers/{containerid}/logs
This is now resolvedx and the APIs now have an ACL authorization.
CDPD-73943: HIVE-27994: Optimized renaming of partitioned tables
Previously, renaming partitioned tables in Hive required fetching, deleting, and re-inserting every row in PART_COL_STATS. This is now optimized to improve performance, especially for tables with large datasets.
CDPD-73942: HIVE-25225: Fix for NPE in update column stats with disabled direct SQL
In batch loading scenarios, overhead during partition column stat updates is now reduced.
CDPD-73941: HIVE-24663: Optimized partition column stats update for large batches
When large number of partitions (>20K) were processed, the ColStatsProcessor ran into DB issues. This issue is now resolved by introducing smaller batches for stats gathering, preventing bulk update issues.
CDPD-73795: Backport CALCITE-6530 to 7.1.9 SP1 CHF2
Http Sessions created by the Avatica server never expired causing Out of Memory issues. This issue is now resolved.
CDPD-73761: [7.1.9 SP1 CHF2] Backport KAFKA-15391 Delete topic may lead to directory offline
Fixed an issue where deleting a topic caused the entire log directory to go offline.
CDPD-73678: Atlas - [7.1.9 SP1 CHFx]Upgrade Spring Framework to 6.1.12/6.0.23/5.3.39 due to CVE-2024-38808 and CVE-2024-38809
Upgraded the Spring Framework version to 6.1.12 due to CVE-2024-38808 and CVE-2024-38809.
CDPD-73643: Filter out unused CM_USER parameter
The unused CM_USER field is now removed from the /cm-configs internal endpoint.
CDPD-73442: IMPALA-13313 Resolution of potential deadlock
When idle_query_timeout was set in a session, new queries stopped responding for that session and failed. This deadlock issue occurred in long-running sessions and is now resolved.
CDPD-73423: Ranger - Upgrade Spring Framework to 6.1.12/6.0.23/5.3.39 due to CVE-2024-38808 and CVE-2024-38809
Upgraded the Spring-framework version to 5.3.39 due to CVE-2024-38808 and CVE-2024-38809.
CDPD-73326: Reduce memory needed to create Ranger policy engine
An issue led to the creation of multiple RangerResourceMatchers with identical resource specification. This issue is now resolved and the creation of multiple RangerResourceMatcher objects is now avoided by maintaining a cache of them in the RangerPluginContext object associated with the Ranger policy engine, thereby reducing policy engine's memory needs.
CDPD-73282: Backport CALCITE-6530 HTTP Sessions are never expired in Avatica server
The http sessions created by the Avatica server did not expire and this caused the Avatica server to run out of memory. This issue is now resolved.
CDPD-73226: Add /tmp noexec support for Zstd and Snappy compression
Zstandard and Snappy compression now support /tmp mounted as noexec.
CDPD-73147: [Ranger React UI] Admin audits for "Import Delete" operation type do not display service name field
In the Ranger React UI, in admin audits, the Service name field was missing for the audits of operation type Import Delete. This issue is now resolved and the Import delete policy logs now display the service name.
CDPD-73144: Enhance trie to support processing of evaluators during traversal

Ranger policy engine uses trie data structure to organize resources for faster retrieval of policies/tags/zones associated with a given resource. When a resource consists of multiple elements, such as, database/table/column, many trie instances are consulted to retrieve policies/tags/zones associated with the resource.

Such multi-trie retrieval is optimized with a 2-pass traversal - first pass to get count and the second pass to get the actual objects. Thherefore, the trie data structure used in Ranger policy engine is now updated to support processing of evaluators during traversal.

CDPD-72812: SSTFilteringService was disabled by default causing snapshot chain corruption.
SSTFilteringService was disabled by default causing snapshot chain corruption. The Ozone snapshots SSTFilteringService is now enabled by default.
CDPD-72766: CLONE - /api/atlas/admin/version works only once after atlas restart
API, Atlas, Admin, Version now works without Atlas restart.
CDPD-72621: Support for default constraints in HWC table writes
Added functionality in Hive Warehouse Connector to support default constraints during table writes, enhancing data integrity and management.
CDPD-72596: IMPALA-12921 IMPALA-12985: Support for running Impala with locally built Ranger
The fix adds support for locally built Ranger and for using the new constructor when instantiating RangerAccessRequestImpl.
CDPD-72292: [Private Cloud Releases] Upgrade RequireJS due to CVE-2024-38998 and CVE-2024-38999
Upgraded the RequireJS version due to CVE-2024-38998 and CVE-2024-38999.
CDPD-71764: XSS vulnerability in Zeppelin : Unsanitized HTML in Markdown Paragraphs
To enhance security, Zeppelin now integrates HTML sanitization using JSoup within the markdown interpreter. This ensures that any HTML embedded in markdown is sanitized according to a configurable blacklist.
CDPD-71279: Proposal to Upgrade All React.js Dependent Libraries
Upgraded the React.JS related libraries.
CDPD-71063: Hue - Upgrade eventlet to 0.35.2 and dnspython to 2.6.1 due to CVE-2023-29483
Security fix for CVE-2023-29483.
CDPD-69767: Backport fix to address CVE-2023-50291
This ticket backports the fix for CVE-2023-50291 from upstream Solr repository.
CDPD-69634, CDPD-69412: [719 SP1 CHFx] DB constraint violation error for grant and revoke command execution
Multiple column revoke resulted in incorrect number of columns in the generated policies, and the revoke statement only revoked if there was only one column. This issue is now resolved and multiple column revoke now generates correct results.
CDPD-69411: [719 SP1 CHFx] Impala's authorization-related tests failed due to PSQLException due to Ranger's backend database
Impala's authorization-related tests failed due to PSQLException caused by Ranger's backend database. This issue is now resolved.
CDPD-67913: IMPALA-12554 Consolidation of Ranger policies for GRANT statements in Impala
Impala now creates a single Ranger policy for the GRANT statement, even when multiple columns are specified, reducing the overall number of policies on the Ranger server.
CDPD-67552: [7.1.9] Add optional wait time for replication-records-lag calculation
Streams Replication Manager (SRM) added a new metric (replication-records-lag) about replication flows. The calculation of this metric has an impact on the performance of the SRM, so the following 3 properties were introduced to be this feature configurable. These properties can be specified in the Cloudera Manager, Configuration tab of the Streams Replication Manager service, where new lines has to be added to the "streams.replication.manager.config" property, where key-value pairs can be defined. The "replication.records.lag.calc.enabled" flag helps to turn on or off the new feature. The calculation of this metric is enabled by default. The "replication.records.lag.calc.period.ms" provides an opportunity to lower the frequency of the calculation of replication-records-lag. The default value is 0, so the metric will be provided every time, but for example with 15 sec or with longer periods, the calculation will not have a huge effect on the SRM replication latency, but the new metric will be available, so it will be not entirely switched off. The "replication.records.lag.end.offset.timeout.ms" is a way to specify the Kafka end offset timeout value in case of replication-records-lag calculation. The default value is 1 min, but lower value provides the opportunity to reduce latency, beside the risk that replication-records-lag calculation may fail. On the other hand, the higher value helps to avoid the failure of the metric calculation, but it may have bigger effect on the SRM replication latency.
CDPD-67460: Container Balancer must move only containers with size greater than 0 bytes.
Introduced a check on the size of the containers allowed to leave the source node during the balancing process. Added a unit test TestContainerBalancerTask#balancerShouldMoveOnlyPositiveSizeContainers to ensure that a balancer selects only the positive size containers. This is tested using the existing parameter TestContainerBalancerTask#balancerShouldObeyMaxSizeLeavingSourceLimit.
CDPD-67113: [7.1.9] Backport KAFKA-13988: Mirrormaker 2 auto.offset.reset=latest not working
Streams Replication Manager (SRM) did not respond to auto.offset.reset=latest config. This issue is now resolved.
CDPD-66886: Solr - Backport fix for CVE-2023-50298, CVE-2023-50292, CVE-2023-50386 and CVE-2023-50291
This ticket backports the fix for CVE-2023-50298, CVE-2023-50292, CVE-2023-50386 and CVE-2023-50291 from upstream Solr repository.
CDPD-64939: [719 SP1 CHFx] RANGER-4585 Support multiple columns policy creation in ranger for Grant / Revoke request
Upgrading from Cloudera Runtime 7.1.7 Service Pack 3 to 7.1.9 Service Pack 1, for a GRANT statement involving multiple columns in a table, Impala service used to create one Ranger policy for each column. This issue is now resolved.
CDPD-63596: There are a few compile-time dependencies on spotbugs-annotation, which is LGPL and not allowed by Apache Software Foundation under Category X.
The spotbugs-annotation, an LGPL thirdparty dependency from the Ozone package is now removed.
CDPD-62620: OM checkpoint request pauses all the background workers and then creates a checkpoint after it acquires all the necessary locks. There is a deadlock between OMDBCheckpointServlet#getCheckpoint request and RocksDBCheckpoint#pruneSstFiles background process causing a halt in the OM and no new read/write requests are taken after some time.
This issue is now resolved. The double locking is removed and now there is no deadlock between OMDBCheckpointServlet#getCheckpoint request and RocksDBCheckpoint#pruneSstFiles background process.
Common Vulnerabilities and Exposures (CVE) that is fixed in this CHF:
  • CVE-2022-31129 - Moment JS
  • CVE-2023-45857 - Axios
  • CVE-2024-38999 - Require JS
  • CVE-2024-38998 - Require JS
  • CVE-2023-29483 - Eventlet