Fixed issues in 7.1.9 CHF 5
Know more about the cumulative hotfix 5 for 7.1.9. This cumulative hotfix was released on April 08, 2024.
Following is the list of fixes that were shipped for CDP Private Cloud Base version 7.1.9-1.cdh7.1.9.p7.51778342
- COMPX-16140: CDPD - Upgrade Spring Security to 5.7.11/5.8.7/6.0.7/6.1.4 due to CVE-2023-34042
- Upgraded Spring Security to 5.7.11/5.8.7/6.0.7/6.1.4 due to CVE 2023-34042.
- COMPX-11263: QM UI: Configuration modification shows old value temporarily
- The configuration page displayed old values intermittently during an update. This issue is now fixed and only the new values entered by the user is displayed during the update.
- CDPD-67558, YARN-11639: YARN RM stops assigning resources either because of ConcurrentModificationException or NPE in PriorityUtilizationQueueOrderingPolicy
- When dynamic queue creation was enabled in weight mode and the deletion policy coincides with the PriorityQueueResourcesForSorting, YARN RM stopped assigning resources because of either ConcurrentModificationException or NPE in PriorityUtilizationQueueOrderingPolicy. This issue is now fixed.
- CDPD-67507: Use Name validation regex instead of service name validation regex for Display name
- The regex validator for validating the service display name in the service edit form did not allow a space character. This fix now allows a space character in the service display name.
- CDPD-67433: IMPALA-12878 TestResultSpoolingCancellation.test_cancellation failed in UBSAN build
- A rare scenario where a query is closed by the client then closed again (most clients prevent this happening), resulted in an error message Query not yet running. This fix restores the previous message Invalid or unknown query handle.
- CDPD-67313: [7.1.x] Timezone value not updated in Livy
- While serialising the Spark results from the Java object into JSON text, Livy was setting the timezone to UTC. The Livy service code is now fixed to use a custom timezone based on the configuration instead of always using UTC.
- CDPD-67278: Backport KNOX-3012: Fix the DN links on the Ozone SCM UI
- There was a change in Ozone, which caused the links of the DataNodes (DN) not to route through Knox on the Ozone SCM UI. With this fix the DN links redirect to the correct Knox URLs again.
- CDPD-67225: Zeppelin - Upgrade Spring Framework to 6.1.4/6.0.17/5.3.32 due to CVE-2024-22243
- Upgraded Spring Framework to 6.1.4/6.0.17/5.3.32 due to CVE-2024-22243.
- CDPD-67220: [Regression] Oozie HTTPS notification fails if SSL is not set in Oozie
- Oozie fails to assemble the notification request if the notification URL is secure (uses HTTPS) but no SSL is configured for Oozie server. This issue is now resolved.
- CDPD-67193: The inactivityTimeout is reset when user updates the profile from UserProfile page
- Fixed an issue of not resetting inactivityTimeout to a default value of 15 minutes when the user updated the profile on the User Profile page of the Ranger Admin UI.
- CDPD-67023: [Ranger React UI] Audit UI improvements with respect to values overflowing into other columns
- In the Ranger React UI, for certain columns in the audits pages,
the value overflowed into the next columns if the text length was long. This issue is
now fixed, and the following columns in the specified audit pages are modified to
prevent the overflow into next
column:
- Access Audits - Service Name and Cluster Name
- Plugin Status Audits - Service Name field
- Login sessions Audits - Login Id field
- CDPD-66997: [AUTOSYNC] Recon - UnsupportedOperationException while merging Incremental Container Reports
- A UnsupportedOperationException was displayed while merging incremental container reports. This issue is now fixed.
- CDPD-66963, CDPD-66773: [AUTOSYNC] NPE causes OM crash in Snapshot Purge request
- Ozone now ignores a purge request if there is a snapshot purge request for an already purged snapshot.
- CDPD-66934: Display query information for Show databases/schemas command on Ranger Admin UI
- In the Ranger React UI, if the resource type for certain commands were logged as null in the audits, then in the access audits, the information of the query/operations performed did not display. This issue is now fixed and the UI now displays the query / operation information for access audits even if the resource type is null.
- CDPD-66927: HDFS authorization logic for directory hierarchy rooted at "/" is incorrect
- There was an issue with the Ranger authorization logic for the
HDFS commands that required authorization of the entire directory hierarchy rooted at
the specified directory argument. This argument was incorrect due to incorrect
computation of the sub-directory paths. The paths of sub-directories to be authorized
contained an extra
/
character, leading to incorrect authorization result. This issue is now fixed. - CDPD-66917: [AUTOSYNC] Upgrade aws-java-sdk to 1.12.661
- Upgrading aws-java-sdk to 1.12.661 version removes ion-java dependency from aws-java-sdk which caused CVE-2024-21634.
- CDPD-66843: [7.1.9 CHF5 CLONE] - Provide an option to bypass evaluation of chained plugin if the parent plugin has applicable policy
- When a chained plugin (such as Hive) is configured, every access
request processed by the parent plugin (such as HDFS/Ozone/S3) is also processed by the
chained plugin. This feature now supports a configuration parameter
ranger.plugin.bypass.chained.plugin.evaluation.if.access.is.determined
with the default value as false. When set to true, the evaluation of the chained plugin is skipped when an applicable policy is found by the parent plugin. This issue is now fixed. - CDPD-66842: Ranger Admin server provides empty response when user with user-role tries to update lastname or email address
- An error response with a message is now displayed when a user with a user-role tries to add or update last name or email address.
- CDPD-66839: Enhance perf-tracer to get CPU time when possible
- Ranger module is instrumented with performance measurement code. Enabling performance logging for the module helps in measuring the amount of time spent during execution of various methods or functions during its operation. For achieving more precise time measurement, this feature supports nanosecond precision when the JVM version supports it.
- CDPD-66798: [7.1.9 CHF5] Skip showing Page not found for wrong value is provided to a API parameter in Login Session Tab
- There was an issue where Page Not Found was displayed when a user entered a text value to a search an API parameter IP in the Login Sessions under Audits. This issue is fixed and a server-side response is now displayed for invalid values as an alert on the Login Sessions tab.
- CDPD-66796: [7.1.9 CHF5] Skip showing Page not found page for INVALID_INPUT_DATA validation in User Profile
- A Page Not Found error message was displayed when a user provided invalid form values during profile update. This issue is now fixed and a server-side response is displayed as an alert on the User Profile window.
- CDPD-66784: Update the execution of setServiceDef call in App.jsx
- Removed unused code related to setServiceDef call in App.jsx.
- CDPD-66782: Updating the Something went wrong page in Ranger React UI
- The Something went wrong message was displayed when there was an error in the React JS code that was used to load Ranger Admin UI. This issue is now fixed.
- CDPD-66781: Audit logs for Masking policy is missing data mask type entry
- Fixed issue of showing Audit log for custom data mask type when added or updated into a policyItem of Masking policy.
- CDPD-66725: Knox - Upgrade Okio to 3.4.0 due to CVE-2023-3635
- Upgraded Okio to 3.4.0 due to CVE-2023-3635.
- CDPD-66719: Ranger - Upgrade Spring Security to 5.7.11/5.8.7/6.0.7/6.1.4 due to CVE-2023-34042
- Upgraded Spring Security to 5.7.11
- CDPD-66568: Export/Import : changeMarker is not set to entity's lastupdatetime or its closer timestamp value
- When a Hive table entity was exported using a fetch type incremental with
changeMarker
0, after exporting, thechangeMarker
in the export response was not set to a recent timestamp. This issue is now fixed, and thechangeMarker
is now set to a closer timestamp value during an export or import. - CDPD-66538: [AUTOSYNC] Metadata are not updated when keys are overwritten
-
There was an issue when an object was created with the same key name as one already present in the database. The request was forwarded to the Ozone Manager (OM) side of the code, specifically to the OmKeyRequest class, containing a method called prepareFileInfo(). This method persists the data to the openKeyTable. Initially, the method checked if a key with the same name exists. If exists,new data size, modification time, updateID, and replicationConfig was updated. However, the metadata of the overridden file was not updated. Consequently, the old metadata stored earlier is retained.
This issue is now fixed and the changes involve extracting new metadata from the KeyArgs object and comparing it with the existing metadata in the OmKeyInfo object. Any new or modified metadata entries are then updated in the OmKeyInfo object. Also, metadata entries not mentioned in the overwrite operation are retained, ensuring the preservation of existing metadata.
- CDPD-66509: [7.1.9]BackPort to 7.1.9x branch
- Upgraded common-dbcp2 to 2.1.0 and commons-pool2 to 2.12.0
- CDPD-66423: Backport HIVE-25986 to 7.1.9.x branches
- The statement ID was incorrect if the table was an insert only ACID table and the
LOAD IN PATH
command was used to load the data. Because of this incorrect statement ID, the delta file path also contained a name, which was incompatible with other systems such as Impala. This issue is now fixed and the correct statement ID is now generated. - CDPD-66417: Upgrade Prometheus to 2.45.3 due to CVEs
- Upgraded Prometheus to 2.45.3 to address CVE-2023-44487 and CVE-2023-45142.
- CDPD-66358: HS2 logs having WARN logs from RangerHiveAuthorizer regarding connection to HMS for fetching hive object owner
- This fix addresses the issue of HS2 logs having huge number of WARN logs.
- CDPD-66243: [Knox] Invalid binary character logged in gateway.log
- Upgraded the libpam4j dependency to fix a bug that resulted in group names with invalid characters.
- CDPD-65969: A change in the message for ozone admin cert list subcommand count limits
- Listing of certificates is performed in a batch with a default size of 20. A few certificates were not displayed if there were more than 20. This issue is now fixed and a warning message is displayed if the batch size is limiting the amount of certificates displayed and an option is provided to increase the batch size.
- CDPD-65808: [7.1.9 CHF5 CLONE] - Performance degradation while retrieving mapped Hive resource for S3 location.
- This fix improves the performance of RMS access evaluation while retrieving mapped Hive resource for Ozone locations (that is, Ozone keys).
- CDPD-65616: Not able to access Zeppelin UI through Knox
- Added JVM arguments to expose hidden internal classes required by the Ranger plug-in.
- CDPD-65001: [AUTOSYNC] Pass TransactionInfo in OzoneManagerRequestHandler.handleWriteRequest
- In
OzoneManagerRequestHandler.handleWriteRequest
, only transactionLogIndex was passed without the term. This issue is now fixed and the TransactionInfo (includes both term and index) is now passed. Thus, avoiding the recalculation of the term later on. - CDPD-64938: [AUTOSYNC] Remove RatisSnapshotInfo
- Fixed the inconsistency with Ratis term and index when values are printed
using the
toString()
command and when operations are running in parallel. - CDPD-64822: [AUTOSYNC] Move add response in doubleBuffer from validateAndUpdateCache to handleWriteRequest
- Fixed an issue to ensure every response returned from validateAndUpdateCache is added to DoubleBuffer.
- CDPD-64626: CLONE - Ranger - Upgrade aws-java-sdk-bundle to 1.12.599 due to CVE-2023-44487
- Upgraded aws-java-sdk-bundle to 5.7.11
- CDPD-64394: [AUTOSYNC] OzoneManagerStateMachine should put all failed write requests into OzoneManagerDoubleBuffer
- Fixed the OzoneManager (OM) restart failure issue due to failed OM write request's response not added to OzoneManagerDoubleBuffer.
- CDPD-64153: [AUTOSYNC] Tool to fix corrupted snapshot chain
- This tool is a workaround to fix the snapshot chain corruption issue until the root cause is identified and fixed for the snapshot chain corruption.
- CDPD-63747: Cache the results of access evaluation
- This feature trades off more memory requirement against a
potential faster evaluation of policies when chained-plugin (as when RMS is enabled) is
configured for HDFS storage authorization. If the configuration parameter
ranger.plugin.hdfs.useResultCache
(default:false) is set to true, then the result of Hive policy authorization for a HDFS storage location is cached and is reused in subsequent accesses of that HDFS location. - CDPD-63687: Deleted resource mapping is not removed from the plugin's cache
- When a storage (HDFS/Ozone/S3) is configured to use RMS, the storage locations of Hive/Impala database/table objects are maintained by the RMS server, and provided to the Ranger authorizer running in the storage service. This feature ensures that when a Hive database/table is removed, mapping information for the removed object is cleared from the resource-mappings provided to the storage service.
- CDPD-63039: IMPALA-12528 test_hdfs_scanner_thread_non_reserved_bytes may occasionally fail
- Fixed unit test issue at
test_hdfs_scanner_thread_non_reserved_bytes
. - CDPD-60459: HueQP - Fixing NPE for adminUser in facets api
- A NullPointerException was displayed in facets/ API. This affected users of Hive job browser (using QP) with impact on all Hue/Hive users, and no impact to other components. This issue is now fixed.
- CDPD-48298: CLONE - Knox - Upgrade Guava: Google Core Libraries for Java to v28.2/31.1-jre due to low CVEs
- Updated the Guava dependency to get the fix for CVE-2020-8908.
- CDPD-46225: Security Zone policies version increases by two when you update its resource.
- The issue where updating a resource resulted in the Security Zone policies version incrementing by two is now resolved.
- CDPD-44220: Livy - Missing deploy mode param at Spark submit
- Fixed occasional issues with session recovery/HA failover on FIPS clusters.
- CVE-2023-36478
- CVE-2023-26048
- CVE-2023-26049
- CVE-2023-40167
- CVE-2023-41900
Technical Service Bulletins
- TSB 2024-752: Dangling delete issue in Spark rewrite_data_files procedure causes incorrect results for Iceberg V2 tables
- For the latest update on this issue see the corresponding Knowledge article: TSB 2024-752: Dangling delete issue in Spark rewrite_data_files procedure causes incorrect results for Iceberg V2 tables.