HDP-2.3.6 Release Notes
Also available as:
PDF

Ranger

In addition to any Apache patches ported over from the 2.4.x core, HDP 2.3.6 provides Ranger 0.5.2 and the following Apache patches:

  • RANGER-204: Not able to delete user or group if user/group has any policy defined..

  • RANGER-205: Delete rest api of User not deleting user completely from system.

  • RANGER-617: Handle Search by status on policy search filter.

  • RANGER-843: Add indexes to improve Postgres query performance.

  • RANGER-875: Restrict Grantor privileges of Ranger db user for Oracle DB Flavour.

  • RANGER-888: Provide support to delete Users and Groups from Ranger Admin UI.

  • RANGER-891: Audit shutdown hook to be registered with Hadoop ShutdownHookManager, instead of directly with Java Runtime.

  • RANGER-893: Ranger ugsync with LDAP is not able to fetch group information when member attribute contains user's short name instead of full DN.

  • RANGER-921: Improve implementation of internal SQL calls and make it more generic.

  • RANGER-926: Ranger UI validation changes to allow . [dot] in firstname and lastname fields.

  • RANGER-934: Ranger should use released version of EclipseLink (EclipseLink 2.6.2).

HDP 2.3.6 provides Ranger 0.5.2 and ports the following Apache patches from the 2.4.x core:

  • RANGER-173: Utility scripts to create HDFS audit folders and policies.

  • RANGER-218: LDAP Groups incorrectly labelled internal.

  • RANGER-725: Add the right .gitignore file to the newly added projects so that directory listing is clean after a build.

  • RANGER-746: Addressing suggestions from Review - Add wildcard, multiple CN & SAN support when validating plugins' SSL certs.

  • RANGER-767: Refactor UserGroupSink implementation and consolidate performance improvements.

  • RANGER-771: Fix 4+ Log entries upon login in X_AUTH_SESS.

  • RANGER-772: Hive plugin Update Ranger authorizer to mimic changes made by Hive standard authorizer for the case when IMPORT can end up creating a table.

  • RANGER-773: Fix newly found Coverity scan issues for Ranger KMS.

  • RANGER-777: Remove the maven profile that was added for building Kafka plugin.

  • RANGER-778: Fix user update issue.

  • RANGER-789: Fix incorrect policy list paging for non-admin users.

  • RANGER-794: Ranger policy engine performance measurement.

  • RANGER-798: Handle different timezone issue while saving audit logs to Solr.

  • RANGER-799: Ranger UI fixes - partial search not working on Policy listing page.

  • RANGER-804: Delete groups associated with User causes Exception in UserSync.

  • RANGER-809: Audit framework need to cache the getHostName() values to reuse for successive calls.

  • RANGER-809: Audit framework need to cache the getHostName() values to reuse for successive calls.

  • RANGER-821: Ranger shutdown hook should not only do its processing asynchronously but also terminate itself if it runs more than a configurable amount of time.

  • RANGER-825: groupId in Ranger s child poms should be consistent and follow convention.

  • RANGER-831: policy version incremented twice when resources are updated.

  • RANGER-833: In Ranger UI add support for usernames containing a plus + symbol.

  • RANGER-834: Correct the excludes flag's treatment when resource value denotes everything.

  • RANGER-836: Optimize policy retrieval.

  • RANGER-844: Optimize policy retrieval for non-admin users.

  • RANGER-848: Policy Listing page fix: users column is empty for non-admin users.

  • RANGER-857: Unify (and update) Tomcat versions.

  • RANGER-863: Make parameters like maxHttpHeaderSize configurable for EmbeddedServer.

  • RANGER-882: Scrub received policies before policy engine uses it to guard against inadvertent data corruption: remove null policy resource values.

  • RANGER-889: Policy engine API to find list of users/groups having access to a resource.

  • RANGER-894: Fixing few issues in the ldap tool when user search base or group search base is configured as part of input.

HDP 2.3.4.7 provided Ranger 0.5.0 and the following Apache patch:

  • RANGER-834: Modify Exclude logic evaluation to handle wildcard characters.

HDP 2.3.4 provided Ranger 0.5.0 and the following Apache patches:

  • RANGER-246: Need to update the current implementation for recent changes in Kafka.

  • RANGER-526: Provide REST API to change user role.

  • RANGER-586: Ranger plugins should not add dependent libraries to component's CLASSPATH.

  • RANGER-590: Escape spaces in the user and group names which are part of rest call URI in UserSync process.

  • RANGER-602: Solr client in SolrCloud mode should work with ZooKeeper settings also.

  • RANGER-607: Unable to create multiple policyItems for same user or group.

  • RANGER-608: Denied access to list a directory does not generate audit.

  • RANGER-652: LDAP configuration tool.

  • RANGER-656: Ranger UI - KMS Need to handle 404 error when clicked on breadcrumb.

  • RANGER-658: Package ranger_credential_helper.py with Ranger Usersync assembly.

  • RANGER-661: Plugin receives empty policy list though the service has policies.

  • RANGER-663: Race condition during policy update causes policy to get in an bad state.

  • RANGER-664: Ranger PolicyRefresh REST Client timeout parameter should be configurable.

  • RANGER-665: ranger.ldap.ad.referral property is not getting updated in RANGER-admin-site.xml.

  • RANGER-666: Ranger to support Azure SQL Database.

  • RANGER-671: Add support to retrieve permissions for the logged in user from UserSession rather going to database every time.

  • RANGER-673: Setup changes to allow Ranger service to installed using custom service user.

  • RANGER-674: Ranger public rest api gives 200 response for wrong credential instead of 401.

  • RANGER-677: Ranger Admin fails to render policies referring to groups that contain "." in name.

  • RANGER-680: Remove public group by default in default policy for KMS repo.

  • RANGER-681: Update default sync intervals for LDAP and UNIX.

  • RANGER-682: Ranger to support Azure Blob Datastore as an audit destination via HDFS audit handler.

  • RANGER-684: Ranger Usersync - Add Ability to transform user/group names.

  • RANGER-687: after each 30 seconds audit is getting updated in plugin tab.

  • RANGER-688: Handle scenario where ids of XUser and XPortalUser are not in sync.

  • RANGER-697: KeyAdmin role user should see only KMS related audit access logs in Audit tab.

  • RANGER-700: Provide a wrapper shell script to run the FileSourceUserGroupBuilder process.

  • RANGER-701: Update setup scripts to allow special characters in passwords.

  • RANGER-702: Optimize policy download performance.

  • RANGER-705: Ranger Usersync should provide summary logs on the sync progress instead of not logging any details after 2000 users.

  • RANGER-706: Optimize audit db upgrade patches to minimize timeout issues.

  • RANGER-712: Create a new project which can serve as a template to write Ranger extensions.

  • RANGER-713: Knox-plugin failed to enable after plugin modification for not to add dependent libraries to component's CLASSPATH.

  • RANGER-714: Enhancements to the db admin setup scripts.

  • RANGER-715: Fix issues reported by coverity test in Ranger Plugin ClassLoader.

  • RANGER-717: Hive and HBase Ranger plugin Audit to DB failed to log after plugin modification for not to add dependent libraries to component's CLASSPATH.

  • RANGER-720: Ldap discovery tool doesn't seem to be working as expected.

  • RANGER-724: AuditBatchQueue: prevQueueSize not recomputed after initial assignment - static code analyzer flagged issue.

  • RANGER-725: Add the right .gitignore file to the newly projects so that directory listing is clean after a build.

  • RANGER-727: Knox Plugin failed to AuditToSpool file when Audit Destination is down.

  • RANGER-731: Ranger plugin for YARN doesn't seem to be able to write audit to Kerberized HDFS.

  • RANGER-733: Implement best coding practices to resolve issues found during code scan.

  • RANGER-739: Ranger HBase Plugin returning null for RegionObserver.preCompact calls causing HBase:ACL issue.

  • RANGER-740: Kafka Authorizer interface has added close() method. Ranger should also implement it.

  • RANGER-741: Fix installation script to skip Audit DB password check if audit source is SOLR.

  • RANGER-742: Ranger usersync fails after syncing 500 users from AD or ldap server when paged results is enabled.

  • RANGER-743: External users with Admin Role should be allowed to create/update users.

  • RANGER-744: Kafka Authorizer has updated how IP/Host is passed.

  • RANGER-745: Upgrade Apache commons-collections.

  • RANGER-747: RangerAdmin is considering "none" as valid ZK Host Name for Solr.

  • RANGER-748: Users in policy got changed after upgrade.

  • RANGER-749: Ranger KMS to support multiple KMS instances with keys across multiple clusters.

  • RANGER-754: Ranger YARN Plugin lookup and test connection should support SPENGO enabled HTTP Authentication.

  • RANGER-755: ldap run.sh script fails since auth directory does not exist.

  • RANGER-756: LdapTool fails with -r option to retrieve only users/group/all.

  • RANGER-757: [LDAP tool] authentication fails if use -d option to search only users.

  • RANGER-758: Handle special characters in passwords starting from -r.

  • RANGER-761: Transaction logs not getting generated under audit menu admin tab if policy name is changed.

  • RANGER-766: Yarn Plugin Config hadoop.security.authentication should be non-mandatory with default value.

  • RANGER-767: Refactor UserGroupSink implementation and consolidate performance improvements.

HDP 2.3.2 provided Ranger 0.5.0 and the following Apache patches:

  • RANGER-551 Policy Validation: If resource levels are not valid for any hierarchy then checks about missing mandatory levels should be skipped

BUG FIXES

  • RANGER-560 Policy validation: Provide user friendly error messages about validation failures

  • RANGER-580 HBase plugin: Plugin may not work after upgrade

  • RANGER-584 Service validation: Provide user friendly error messages about validation failures

  • RANGER-587 ranger-admin-site.xml not getting updated when ranger.authentication.method is changed

  • RANGER-588 Take care of Ranger KMS installation even if 'java' is not in PATH

  • RANGER-593 Service def validation: Provide user friendly error messages about validation failures

  • RANGER-594 Policy Validation: Change the logic to generate friendly error messages to be like used for Service and Service def

  • RANGER-598 Update Ranger config migration script to work with Ranger 0.5

  • RANGER-615 Audit to db: Truncate all string values of audit record so that writing of audit does not fail

  • RANGER-618 KMS gets slower in key creation once Database grows

  • RANGER-621 Solr service-def JSON has incorrect impliedGrants for solr_admin permission

  • RANGER-622 Hive plugin: Add jar via Beeline throws NPE

  • RANGER-623 Enable plugin scripts should handle file permissions for certain umask value

  • RANGER-624 Windows installation broken after SQLAnywhere support

  • RANGER-625 Change db flavor input parameter value from SQLAnywhere to SQLA

  • RANGER-627 Processing done by Audit Shutdown hooks can confuse someone looking at logs to think that shutdown of a service is held up due to Ranger plugin

  • RANGER-628 Make filters for ranger-admin search binds configurable

  • RANGER-630 Data consistency across API and UI

  • RANGER-632 Policy validation error messages produced by the server are not seen by the user

  • RANGER-637 Make REFERRAL property in Ranger User sync configurable

  • RANGER-638 Ranger admin should redirect back to login page when session cookies expires

  • RANGER-639 Storm plugin - commons-lang is a required dependency and hence should be packaged as part of Storm plugin

  • RANGER-641 Ranger kms start fails if java is not set and started using service keyword

  • RANGER-642 Update USERSEARCHFILTER for Ranger Authentication on Windows

  • RANGER-653 Move delegated admin check to mgr layer from service layer for XPermMap and XAuditMap

HDP 2.3.0 provided Ranger 0.5.0 and the following Apache patches:

  • RANGER-422 Add additional database columns to support aggregation

  • RANGER-423 Support audit log aggregation in Ranger Admin UI

  • RANGER-513 Policy validation: resource hierarchies check does not work with single-node hierarchies as in HDFS

  • RANGER-551 Policy Validation: If resource levels are not valid for any hierarchy then checks about missing mandatory levels should be skipped.

  • RANGER-564 Add incubating to the release name

BUG FIXES

  • RANGER-219 Autocomplete behavior of Hive tables/columns

  • RANGER-524 HBase plugin: list command should prune the tables returned on user permissions

  • RANGER-529 Policy Validation: resources of a policy must match one of the resource hierarchies of the service def.

  • RANGER-533 HBase plugin: if user does not have family-level access to any family in a table then user may be incorrectly denied access done at table/family level during get or scan

  • RANGER-539 Rolling downgrade changes

  • RANGER-545 Fix js error for lower versions of FF (less than 30)

  • RANGER-548 Key rollover command fails

  • RANGER-550 Hive plugin: Add audit logging support for metadata queries that have filtering support from Hive.

  • RANGER-553 Default policy creation during service creation should handle service defs with multiple hierarchies, e.g. Hive properly

  • RANGER-554 Ranger KMS keys listing page does not support pagination

  • RANGER-555 Policy view page (from access audit page) gives 404 with Oracle DB

  • RANGER-558 HBase plugin: unless user has READ access at some level under the table/family being accessed (via scan/get) authorizer should throw an exception and audit

  • RANGER-565 Ranger Admin install fails (sometimes) with IO Error when DB used in Oracle

  • RANGER-566 Installation of Ranger on Oracle 12c with shared database needs to use private synonym instead of public synonym

  • RANGER-569 Enabling Ranger plugin for HBase should not modify hbase.rpc.protection value

  • RANGER-570 Knox plugin: after upgrading Ranger from 0.4 to 0.5 the Knox plugin won't work because classes with old names are missing

  • RANGER-571 Storm plugin: after upgrading Ranger from 0.4 to 0.5 the plugin won't work because classes with old names are missing

  • RANGER-575 Allow KMS policies to be assigned to all users

  • RANGER-576 Storm audit not showing access type in the Ranger Admin Audit UI

HDP CHANGES

  • RANGER-450 Failed to install Ranger component due to Ranger policyManager script failures