Fixed issues in 7.1.9 CHF 5

COMPX-16140: CDPD - Upgrade Spring Security to 5.7.11/5.8.7/6.0.7/6.1.4 due to CVE-2023-34042

Upgraded Spring Security to 5.7.11/5.8.7/6.0.7/6.1.4 due to CVE 2023-34042.

COMPX-11263: QM UI: Configuration modification shows old value temporarily

The configuration page displayed old values intermittently during an update. This issue is now fixed and only the new values entered by the user is displayed during the update.

CDPD-67558, YARN-11639: YARN RM stops assigning resources either because of ConcurrentModificationException or NPE in PriorityUtilizationQueueOrderingPolicy

When dynamic queue creation was enabled in weight mode and the deletion policy coincides with the PriorityQueueResourcesForSorting, YARN RM stopped assigning resources because of either ConcurrentModificationException or NPE in PriorityUtilizationQueueOrderingPolicy. This issue is now fixed.

CDPD-67507: Use Name validation regex instead of service name validation regex for Display name

The regex validator for validating the service display name in the service edit form did not allow a space character. This fix now allows a space character in the service display name.

CDPD-67433: IMPALA-12878 TestResultSpoolingCancellation.test_cancellation failed in UBSAN build

A rare scenario where a query is closed by the client then closed again (most clients prevent this happening), resulted in an error message Query not yet running. This fix restores the previous message Invalid or unknown query handle.

CDPD-67313: [7.1.x] Timezone value not updated in Livy

While serialising the Spark results from the Java object into JSON text, Livy was setting the timezone to UTC. The Livy service code is now fixed to use a custom timezone based on the configuration instead of always using UTC.

CDPD-67278: Backport KNOX-3012: Fix the DN links on the Ozone SCM UI

There was a change in Ozone, which caused the links of the DataNodes (DN) not to route through Knox on the Ozone SCM UI. With this fix the DN links redirect to the correct Knox URLs again.

CDPD-67225: Zeppelin - Upgrade Spring Framework to 6.1.4/6.0.17/5.3.32 due to CVE-2024-22243

Upgraded Spring Framework to 6.1.4/6.0.17/5.3.32 due to CVE-2024-22243.

CDPD-67220: [Regression] Oozie HTTPS notification fails if SSL is not set in Oozie

Oozie fails to assemble the notification request if the notification URL is secure (uses HTTPS) but no SSL is configured for Oozie server. This issue is now resolved.

CDPD-67193: The inactivityTimeout is reset when user updates the profile from UserProfile page

Fixed an issue of not resetting inactivityTimeout to a default value of 15 minutes when the user updated the profile on the User Profile page of the Ranger Admin UI.

CDPD-67023: [Ranger React UI] Audit UI improvements with respect to values overflowing into other columns

In the Ranger React UI, for certain columns in the audits pages, the value overflowed into the next columns if the text length was long. This issue is now fixed, and the following columns in the specified audit pages are modified to prevent the overflow into next column:

Access Audits - Service Name and Cluster Name
Plugin Status Audits - Service Name field
Login sessions Audits - Login Id field

CDPD-66997: [AUTOSYNC] Recon - UnsupportedOperationException while merging Incremental Container Reports

A UnsupportedOperationException was displayed while merging incremental container reports. This issue is now fixed.

CDPD-66963, CDPD-66773: [AUTOSYNC] NPE causes OM crash in Snapshot Purge request

Ozone now ignores a purge request if there is a snapshot purge request for an already purged snapshot.

CDPD-66934: Display query information for Show databases/schemas command on Ranger Admin UI

In the Ranger React UI, if the resource type for certain commands were logged as null in the audits, then in the access audits, the information of the query/operations performed did not display. This issue is now fixed and the UI now displays the query / operation information for access audits even if the resource type is null.

CDPD-66927: HDFS authorization logic for directory hierarchy rooted at "/" is incorrect

There was an issue with the Ranger authorization logic for the HDFS commands that required authorization of the entire directory hierarchy rooted at the specified directory argument. This argument was incorrect due to incorrect computation of the sub-directory paths. The paths of sub-directories to be authorized contained an extra / character, leading to incorrect authorization result. This issue is now fixed.

CDPD-66917: [AUTOSYNC] Upgrade aws-java-sdk to 1.12.661

Upgrading aws-java-sdk to 1.12.661 version removes ion-java dependency from aws-java-sdk which caused CVE-2024-21634.

CDPD-66843: [7.1.9 CHF5 CLONE] - Provide an option to bypass evaluation of chained plugin if the parent plugin has applicable policy

When a chained plugin (such as Hive) is configured, every access request processed by the parent plugin (such as HDFS/Ozone/S3) is also processed by the chained plugin. This feature now supports a configuration parameter ranger.plugin.bypass.chained.plugin.evaluation.if.access.is.determined with the default value as false. When set to true, the evaluation of the chained plugin is skipped when an applicable policy is found by the parent plugin. This issue is now fixed.

CDPD-66842: Ranger Admin server provides empty response when user with user-role tries to update lastname or email address

An error response with a message is now displayed when a user with a user-role tries to add or update last name or email address.

CDPD-66839: Enhance perf-tracer to get CPU time when possible

Ranger module is instrumented with performance measurement code. Enabling performance logging for the module helps in measuring the amount of time spent during execution of various methods or functions during its operation. For achieving more precise time measurement, this feature supports nanosecond precision when the JVM version supports it.

CDPD-66798: [7.1.9 CHF5] Skip showing Page not found for wrong value is provided to a API parameter in Login Session Tab

There was an issue where Page Not Found was displayed when a user entered a text value to a search an API parameter IP in the Login Sessions under Audits. This issue is fixed and a server-side response is now displayed for invalid values as an alert on the Login Sessions tab.

CDPD-66796: [7.1.9 CHF5] Skip showing Page not found page for INVALID_INPUT_DATA validation in User Profile

A Page Not Found error message was displayed when a user provided invalid form values during profile update. This issue is now fixed and a server-side response is displayed as an alert on the User Profile window.

CDPD-66784: Update the execution of setServiceDef call in App.jsx

Removed unused code related to setServiceDef call in App.jsx.

CDPD-66782: Updating the Something went wrong page in Ranger React UI

The Something went wrong message was displayed when there was an error in the React JS code that was used to load Ranger Admin UI. This issue is now fixed.

CDPD-66781: Audit logs for Masking policy is missing data mask type entry

Fixed issue of showing Audit log for custom data mask type when added or updated into a policyItem of Masking policy.

CDPD-66725: Knox - Upgrade Okio to 3.4.0 due to CVE-2023-3635

Upgraded Okio to 3.4.0 due to CVE-2023-3635.

CDPD-66719: Ranger - Upgrade Spring Security to 5.7.11/5.8.7/6.0.7/6.1.4 due to CVE-2023-34042

Upgraded Spring Security to 5.7.11

CDPD-66568: Export/Import : changeMarker is not set to entity's lastupdatetime or its closer timestamp value

When a Hive table entity was exported using a fetch type incremental with changeMarker 0, after exporting, the changeMarker in the export response was not set to a recent timestamp. This issue is now fixed, and the changeMarker is now set to a closer timestamp value during an export or import.

CDPD-66538: [AUTOSYNC] Metadata are not updated when keys are overwritten

There was an issue when an object was created with the same key name as one already present in the database. The request was forwarded to the Ozone Manager (OM) side of the code, specifically to the OmKeyRequest class, containing a method called prepareFileInfo(). This method persists the data to the openKeyTable. Initially, the method checked if a key with the same name exists. If exists,new data size, modification time, updateID, and replicationConfig was updated. However, the metadata of the overridden file was not updated. Consequently, the old metadata stored earlier is retained.

This issue is now fixed and the changes involve extracting new metadata from the KeyArgs object and comparing it with the existing metadata in the OmKeyInfo object. Any new or modified metadata entries are then updated in the OmKeyInfo object. Also, metadata entries not mentioned in the overwrite operation are retained, ensuring the preservation of existing metadata.

CDPD-66509: [7.1.9]BackPort to 7.1.9x branch

Upgraded common-dbcp2 to 2.1.0 and commons-pool2 to 2.12.0

CDPD-66423: Backport HIVE-25986 to 7.1.9.x branches

The statement ID was incorrect if the table was an insert only ACID table and the LOAD IN PATH command was used to load the data. Because of this incorrect statement ID, the delta file path also contained a name, which was incompatible with other systems such as Impala. This issue is now fixed and the correct statement ID is now generated.

CDPD-66417: Upgrade Prometheus to 2.45.3 due to CVEs

Upgraded Prometheus to 2.45.3 to address CVE-2023-44487 and CVE-2023-45142.

CDPD-66358: HS2 logs having WARN logs from RangerHiveAuthorizer regarding connection to HMS for fetching hive object owner

This fix addresses the issue of HS2 logs having huge number of WARN logs.

CDPD-66243: [Knox] Invalid binary character logged in gateway.log

Upgraded the libpam4j dependency to fix a bug that resulted in group names with invalid characters.

CDPD-65969: A change in the message for ozone admin cert list subcommand count limits

Listing of certificates is performed in a batch with a default size of 20. A few certificates were not displayed if there were more than 20. This issue is now fixed and a warning message is displayed if the batch size is limiting the amount of certificates displayed and an option is provided to increase the batch size.

CDPD-65808: [7.1.9 CHF5 CLONE] - Performance degradation while retrieving mapped Hive resource for S3 location.

This fix improves the performance of RMS access evaluation while retrieving mapped Hive resource for Ozone locations (that is, Ozone keys).

CDPD-65616: Not able to access Zeppelin UI through Knox

Added JVM arguments to expose hidden internal classes required by the Ranger plug-in.

CDPD-65001: [AUTOSYNC] Pass TransactionInfo in OzoneManagerRequestHandler.handleWriteRequest

In OzoneManagerRequestHandler.handleWriteRequest, only transactionLogIndex was passed without the term. This issue is now fixed and the TransactionInfo (includes both term and index) is now passed. Thus, avoiding the recalculation of the term later on.

CDPD-64938: [AUTOSYNC] Remove RatisSnapshotInfo

Fixed the inconsistency with Ratis term and index when values are printed using the toString() command and when operations are running in parallel.

CDPD-64822: [AUTOSYNC] Move add response in doubleBuffer from validateAndUpdateCache to handleWriteRequest

Fixed an issue to ensure every response returned from validateAndUpdateCache is added to DoubleBuffer.

CDPD-64626: CLONE - Ranger - Upgrade aws-java-sdk-bundle to 1.12.599 due to CVE-2023-44487

Upgraded aws-java-sdk-bundle to 5.7.11

CDPD-64394: [AUTOSYNC] OzoneManagerStateMachine should put all failed write requests into OzoneManagerDoubleBuffer

Fixed the OzoneManager (OM) restart failure issue due to failed OM write request's response not added to OzoneManagerDoubleBuffer.

CDPD-64153: [AUTOSYNC] Tool to fix corrupted snapshot chain

This tool is a workaround to fix the snapshot chain corruption issue until the root cause is identified and fixed for the snapshot chain corruption.

CDPD-63747: Cache the results of access evaluation

This feature trades off more memory requirement against a potential faster evaluation of policies when chained-plugin (as when RMS is enabled) is configured for HDFS storage authorization. If the configuration parameter ranger.plugin.hdfs.useResultCache (default:false) is set to true, then the result of Hive policy authorization for a HDFS storage location is cached and is reused in subsequent accesses of that HDFS location.

CDPD-63687: Deleted resource mapping is not removed from the plugin's cache

When a storage (HDFS/Ozone/S3) is configured to use RMS, the storage locations of Hive/Impala database/table objects are maintained by the RMS server, and provided to the Ranger authorizer running in the storage service. This feature ensures that when a Hive database/table is removed, mapping information for the removed object is cleared from the resource-mappings provided to the storage service.

CDPD-63039: IMPALA-12528 test_hdfs_scanner_thread_non_reserved_bytes may occasionally fail

Fixed unit test issue at test_hdfs_scanner_thread_non_reserved_bytes.

CDPD-60459: HueQP - Fixing NPE for adminUser in facets api

A NullPointerException was displayed in facets/ API. This affected users of Hive job browser (using QP) with impact on all Hue/Hive users, and no impact to other components. This issue is now fixed.

CDPD-48298: CLONE - Knox - Upgrade Guava: Google Core Libraries for Java to v28.2/31.1-jre due to low CVEs

Updated the Guava dependency to get the fix for CVE-2020-8908.

CDPD-46225: Security Zone policies version increases by two when you update its resource.

The issue where updating a resource resulted in the Security Zone policies version incrementing by two is now resolved.

CDPD-44220: Livy - Missing deploy mode param at Spark submit

Fixed occasional issues with session recovery/HA failover on FIPS clusters.