Fixed Issues in Apache Ranger

Review the list of Ranger issues that are resolved in Cloudera Runtime 7.1.9.

CDPD-53435: [7.1.9.x] Add/ Update metric details for Ranger TagSync

Add Metrics APIs for Ranger Tagsync.

CDPD-44451: Add/ Update metric details for Ranger UserSync

Add Metrics APIs for Ranger Usersync.

CDPD-58506: User is not allowed delete directory in ozone even though user has permissions

User is not allowed delete directory in ozone even though user has permissions

CDPD-69608: Ranger TagSync is out of memory

Ranger TagSync process runs out of heap memory. The out of memory issue could be because the Kafka messages that are of no interest to TagSync are stored in TagSync process and shall not be cleared until a Kafka message that must be uploaded to Ranger admin is encountered. Leads to build up of messages in a Kafka queue ("ATLAS_ENTITIES") under the same condition.

CDPD-50662: [7.1.9.x] - Groups are not visible in mask and row level policy listing tables.

Groups listing are not visible in mask and row-level policy listing tables.

CDPD-51892: CLONE - Tag-based policy UI to not show permissions in deny/exception for services that don't support deny/exception

tag-based policy UI should not show permissions in deny and exception policy-items for service-types that don’t support deny and exceptions i.e., service-defs having options.enableDenyAndExceptionsInPolicies=false.

CDPD-55048: KafkaAuthorization ACL operation Interface implementation in RangerKafkaAuthorizer

KafkaAuthorization ACL operation Interface implementation in RangerKafkaAuthorizer

CDPD-57073: RangerClient#createRole singletonMap causes Ozone tenant creation failure in custom-kerberos-principal-option4

Reverted a change(part of another review for JWT changes) to fix tenant creation in Ozone. Fixes the REST API call, passing auth_type as kerberos in the request was the issue.

CDPD-49182: [7.1.9] Ranger AD User Sync - support for AD group names containing slashes

Adds support for LDAP user and group names with special characters.

CDPD-44902: Ranger admin feature to delete all external users

Introduced new feature with the addition of 2 new REST APIs to force delete external users at scale.

CDPD-46248: Ranger RMS Field issues

Fixed issues listed in the description below. Please ensure that before applying the patch, RMS service is stopped and the existing RMS resource-mapping is cleaned up. This can be achieved by updating the RMS database tables with the following SQL commands. delete from x_rms_resource_mapping; delete from x_rms_service_resource; delete from x_rms_notification; update x_rms_mapping_provider set last_known_version=-1; After applying the patch and restarting RMS server, the resource-mappings will be re-synced from HiveMetaStore.

CDPD-50668: CLONE 7.1.9 - HA support for Ranger User Sync

This is a new feature which enables support for Ranger usersync in HA(Active-Passive) mode.

CDPD-48978: kms get currentversion api is returning old keymaterial after key migration from KTS to KMS

while exporting keys for KTS migration, key version should be in opposite order

CDPD-49334: Key migration from KTS to RangerKMS

Key migration from KTS to RangerKMS DB

CDPD-55419: Ranger - Upgrade json-smart to 2.4.10 due to CVE-2023-1370

Upgrade json-smart to 2.4.10

CDPD-53858: metrics are not getting dumped in /var/log/ranger/kms/ranger_kms_metric.log file when KMS is stopped

After discussion internally, it was agreed to dump the metric state in the same regular kms log file when service goes down.

CDPD-57318: Ranger - Upgrade jackson-dataformat-xml to 2.13.5 due to multiple CVEs in woodstox

Use woodstox-core to 5.4.0 version

CDPD-56463: [7.1.9] - Ranger - Upgrade Spring Security to 5.7.8+/5.8.3+/6.0.3+ due to CVE-2023-20862

Upgrade Spring Security to 5.7.8

CDPD-50537: [7.1.9.x] - Ranger - Upgrade Kerby to 2.0.3 due to CVE-2023-25613

Upgrade Kerby to 2.0.3

CDPD-55561: Ranger - Upgrade bcpkix-jdk15on to 1.70+ due to CVE-2019-17359

Upgrade bcpkix-jdk15on to 1.70

CDPD-15744: HA support for Ranger Tag Sync/User Sync

HA support for Ranger TagSync and UserSync added as part of this new feature enhancement.

CDPD-54854: CLONE [7.1.9]- Ranger audit metrics deletion is failing

Code fix for Ranger audit metrics deletion failing.

CDPD-50648: CLONE [7.1.9] - Ranger is opening a lot of zk connections when solr is down

Making sure that Ranger closes the Zookeeper connection in case when Solr service is not reachable. Also following the configured number of retries to connect to Solr and on given time intervals.

CDPD-49503: [Ranger UI] [React JS] If the url to edit a policy, service or permissions for a module, and the url to view user/group/roles contains an invalid id, then page should display an error

1) If the user enters the Wrong URL in ranger UI It will give 404 Page not found Error page. 2) If the user enters the wrong ID that is not present in the database It will show 400 Data not found page

CDPD-54619: [7.1.9.x]- Regression caused by CDPD-45891

Fix uri for getDeletedGroups() in PolicyMgrUserGroupBuilder

CDPD-44227: Ranger improvement - Roles Import/export API for ranger admin

Add Roles Import/export API for ranger admin

CDPD-44198: shell script to export, transform, import of ranger tags for ranger replication

shell script to export, transform, import of ranger tags for ranger replication

CDPD-50457: [719 CLONE] - Provide option to update group memberships when same users/groups are synced from different sync sources

Update group memberships when same users/groups are synced from different sync sources

CDPD-56737: Ranger - Upgrade Tomcat to 8.5.89 due to CVE-2023-28709

Upgrade Tomcat to 8.5.89

CDPD-50454: [7.1.9.x]- Unable to delete the user if policy is created by same user and added in the policy item

Allow delete user operation if policy is created by same user and added in the policy item

CDPD-56300: Introduce config within Ranger to control retention period of x_auth_session data

Add config within Ranger to control retention period of x_auth_session table data

CDPD-55459: Ranger - Upgrade Spring Framework to 5.3.27/6.0.8 due to CVE-2023-20863

Upgrade Spring Framework to 5.3.27

CDPD-49638: [7.1.9.x] - Log4j2 support in Ranger

Added Log4j2 support in Ranger

CDPD-11878: Support for avoiding multiple access request enrichment

Optimization to enrich the request only once to alleviate the performance overhead.

CDPD-50533: [7.1.9.x] - Add unique constraint on resource_signature column of x_rms_service_resource table

Add unique constraint on resource_signature column of x_rms_service_resource table

CDPD-50605: ArrayIndexOutOfBounds exception may be thrown while processing events

Fix to handle ArrayIndexOutOfBounds exception while processing events

CDPD-49650: [7.1.9.x] - Add Oracle SSL support in ranger

Oracle SSL Connection support in ranger

CDPD-58569: Ranger - Upgrade Guava to 32.0.1 due to CVE-2023-2976

Upgrade Guava library version to 32.0.1

CDPD-52749: [7.1.9.x]- [Ranger][UserSync]Enumerate Groups will give error on executing 'getent group' command

Fix for Enumerate Groups will give error on executing 'getent group' command

CDPD-50368: [7.1.9]- Ranger - Upgrade snakeyaml due to CVE-2022-1471

Upgrade snakeyaml to 2.0

CDPD-50433: [7.1.9.x] - No policy found for given version in Ranger Audit page

Record policy data history during ranger upgrade

CDPD-49704: deleteUserGroupUtil.py fails to delete username with space

Allow deletion of users having space in username

CDPD-58493: Ranger - Upgrade Netty Project to 4.1.94.Final due CVE-2023-34462

Upgrade Netty Project to 4.1.94.Final

CDPD-56457: [7.1.9] - Ranger - Upgrade Nimbus-JOSE-JWT to 9.24 due to CVEs coming from json-smart

Upgrade Nimbus-JOSE-JWT to 9.31

CDPD-40385: Ranger RMS for Ozone

This is a new feature introduced in CDP 7.1.9. Ranger RMS will support authorization for Ozone storage locations. RMS for Ozone will co-exist with Hive-HDFS ACL sync and provide authorization for both HDFS and Ozone file systems.

CDPD-53830: [7.1.9.x] Add/ Update metric details for Ranger RMS

Add Metrics APIs for Ranger RMS

CDPD-50564: Add/ Update Additional metric details for Ranger RMS

Add Additional Metrics for Ranger RMS.

CDPD-55050: Support SELF_OR_PREFIX resource matching scope in Ranger Authorization

API to find whether a user/group/role is authorized to the given operation on any resource of give type

CDPD-50670: CLONE 7.1.9 - HA support for Ranger TagSync

This is a new feature which enables support for Ranger TagSync in HA(Active-Passive) mode.

CDPD-35034: [SDX/SaaS Migration] Utilities to migrate Ranger Service Tags

Utilities to migrate Ranger Service Tags

CDPD-47989: Ranger - Upgrade Netty to 4.1.86.Final due to CVE-2022-41881, CVE-2022-41915

Upgrade Netty to 4.1.86.Final

CDPD-49711: assignPermissionToUser in XUserMgr creates entries with NULL moduleId in x_user_module_perm

Fixed assignPermissionToUser in XUserMgr to correct the bug which assigns permissions for a module (which does not exist) to users with Auditor role.

CDPD-39208: Review and remove unused RDBMS tables used by Ranger admin service

Remove unused RDBMS tables used by Ranger admin service

CDPD-53805: Ozone_key tag based policies are not working

What was the Root Cause? Ozone qualified name parsing had a issue wherein '/' was getting included in the key name which resulted in wrong key matching while enforcing policy How was this Issue Resolved? Logic for parsing ozone qualified name changed such that '/' is not included in the key name which was causing issue previously.

CDPD-55572: shell script to export, transform, import of ranger Roles for ranger replication

Shell script to export, transform, import of ranger Roles for ranger replication

CDPD-43132: Allow roles, tagrest & xaudit Ranger Admin APIs via knox proxy

This fix allows access to ranger role, tagrest and xaudit ranger admin APIs from knox proxy.

CDPD-57018: Ranger - Upgrade aws-java-sdk to 1.12.367+

Upgrade aws-java-sdk to 1.12.481

CDPD-48119: Ranger - Upgrade OWASP Java HTML Sanitizer due to security CVEs

Upgrade OWASP Java HTML Sanitizer

CDPD-50588: [719 CLONE] - Update dependencies to support macOS aarch64 M1 (Apple Silicon) environment

Support ranger build on macOS aarch64 M1 (Apple Silicon) environment

OPSAPS-67025: CM changes for Key migration from KTS to RangerKMS

Migrating hadoop keys from Ranger KMS KTS database to Ranger KMS database

OPSAPS-67374: [7.1.9.x] Unable to locate appender KMS-AUDIT & KMS-METRICS error shown during Ranger KMS start task

Resolved log4j2 appender issues for Ranger KMS.

OPSAPS-65704: Alert or notification has to done when Solr is down resulting in audit pile up in spool directory

A health alert will be shown on Cloudera Manager for Ranger plugin supported services, when the used space of ranger plugin spool directory (local directory) is greater than the threshold value.

OPSAPS-65894: Support LunaClient 10.3 for Ranger KMS DB

New doc created for the Luna 10.5

CDPD-50726: [7.1.9.x]- Need to update Knox re-write rules to allow access to newer APIs introduced in Ranger

Update Knox re-write rules to allow access to newer APIs introduced in Ranger

CDPD-29102: Ranger - Remove log4j 1.x dependencies due to EOL

Log4j 1.x dependency is removed and upgraded to log4j2

CDPD-54698: [7.1.9.x] - Ranger - Upgrade Scala to 2.13.9 due to CVE-2022-36944

Upgrade scala to 2.13.9 as part of CVE fix

CDPD-55164: ranger policy replication transform step is not printing logs

Improve ranger policy replication transformation logs

CDPD-56462: [7.1.9] - Ranger - Upgrade BeanShell to 2.1b5 due to high CVEs

Upgrade BeanShell to 2.1b5 by upgrading testNG to 7.0.0

CDPD-56454: [7.1.9]- Ranger - Upgrade Apache Derby due to critical CVEs

Upgrade Apache Derby to 10.14.2.0

CDPD-56455: [7.1.9] - Ranger - Upgrade Spring LDAP to 2.4.1 due to high CVEs

Upgrade Spring LDAP to 2.4.1

CDPD-55920: Turning usersync debug logging on results in users not getting synced due to NPE

Fix NPE while logging debug messages

CDPD-46256: Ranger Audit metrics page broken in New UI

Fixed Audit metrics not loading in New UI

CDPD-48041: Ranger - Upgrade commons-net to 3.9.0 due to CVE-2021-37533

Upgrade commons-net to 3.9.0

CDPD-47900: Log4j2 support in Ranger

Log4j 1.x dependency is removed and upgraded to log4j2

CDPD-46233: knox plugin is not working

Knox service was failing when Audit metrics was enabled. Fix was done to handle the CNF error in knox ranger plugin which took care of this error

CDPD-53826: Ranger - Upgrade jettison to 1.5.4 due to CVE-2023-1436

Upgrade jettison to 1.5.4

CDPD-53804: Ranger - Upgrade Spring Framework to 5.3.26/6.0.7 due to CVE-2023-20861 and CVE-2023-20860

Upgrade Spring Framework to 5.3.27

CDPD-48032: Ranger - Upgrade jettison to 1.5.2 due to CVE-2022-45685 and CVE-2022-45693

Upgrade jettison to 1.5.2