Ranger Audit Filters (Technical Preview)

You can use Ranger audit filters to control the amount of audit log data collected and stored on your cluster.

About Ranger audit filters

Ranger audit filters allow you to control the amount of audit log data for each Ranger service. Audit filters are defined using a JSON string that is added to each service configuration. The audit filter JSON string is a simplified form of the Ranger policy JSON.

Ranger audit filters format

Audit filters consist of a subset of Ranger policy attributes, along with access results attributes. These attributes define the audit filter rules.

{'resources':
 {
   'database':{'values':['data','data2']},
   'table':{'values':['*']},
   'column':{'values':['*']}
 },
 'accessTypes':['select','insert'],
 'users':['user1','user2],
 'groups':['no_audit_group'],
 'roles' :['role1','role2'],
 'actions': ['METADATA OPERATION','SHOW_ROLES'],
 'accessResult': 'DENIED' or 'ALLOWED' or 'NOT_DETERMINED' or empty
 'isAudited' : false
}

Ranger audit filters configuration

Ranger audit filters are configured by adding the audit filter JSON string as the value of the ranger.plugin.audit.filters configuration property in each Ranger service.

To configure resource-based audit filters, click the Edit icon for the applicable service in the Ranger Admin web UI. On the Edit Service page, click the Add (+) icon under Add New Configurations, then enter ranger.plugin.audit.filters in the Name box, and enter the audit filter JSON string in the Value box.

To configure tag-based audit filters, click the Edit icon for the applicable tag-based service in the Ranger Admin web UI. On the Edit Service page, click the Add (+) icon under Add New Configurations, then enter ranger.plugin.audit.filters in the Name box, and enter the audit filter JSON string in the Value box.

HDFS service audit filters example:

"serviceConfig": {
   "ranger.plugin.audit.filters": 
   "[
     {'accessResult': 'DENIED', 'isAudited': true},

     {'users': ['unaudited-user1'], 'groups': ['unaudited-group1'], 'roles':['unaudited-role1'], 'isAudited': false}, 

     {'actions': [ 'listStatus', 'getfileinfo' ], 'accessTypes':['execute'], 'isAudited': false}, 

 {'resources':{'path':{'values':['/audited'],'isRecursive':true}},
'isAudited': true},
     
{'resources':{'path':{'values':['/unaudited'],'isRecursive':true}},'isAudited': false} 
     ]"
   }

Hive service audit filters example:

"serviceConfig": {
  	"ranger.plugin.audit.filters":

"[ {'accessResult': 'DENIED', 'isAudited': true},
 {'resources':{'database':{'values':['temp']},'table':{'values':['tempdata']},'column':{'values':['*']}},'isAudited' : false},
 {'resources':{'database':{'values':['sys']},'table':{'values':['dump']}},users':['user2']','isAudited': false }, 

{'actions':['METADATA OPERATION'], 'isAudited': false}, 

{'users':['superuser1'],'groups':['supergroup1'], 'isAudited': false} ]"
}

Tag-based service audit filters example:

"serviceConfig": {
 "ranger.plugin.audit.filters": "[ {'resources':{'tag':{'values':['NO_AUDIT']}},'isAudited': false}, 

{'resources':{'tag':{'values':['SYS_DATA']}},'users':['user1'],
'isAudited':false},

 {'resources':{'tag':{'values':['HIPPA']}},'users':['user1'], 'isAudited':true} ]"
}

Audit filters details

As shown in the examples above, audit filters are defined as a JSON list.

When adding the JSON string to the Value box in each service configuration, you should only enter the text within the top-level brackets, as ranger.plugin.audit.filters is already set in the Name box in the UI. For the HDFS example above, you would enter:

[ {'accessResult': 'DENIED', 'isAudited': true},

     {'users': ['unaudited-user1'], 'groups': ['unaudited-group1'], 'roles':['unaudited-role1'], 'isAudited': false}, 

     {'actions': [ 'listStatus', 'getfileinfo' ], 'accessTypes':['execute'], 'isAudited': false}, 

 {'resources':{'path':{'values':['/audited'],'isRecursive':true}},
'isAudited': true},
     
{'resources':{'path':{'values':['/unaudited'],'isRecursive':true}},'isAudited': false} ]

Each value in the list is an audit filter, which takes the format of a simplified Ranger policy, along with access results fields.
Audit filters are defined with rules on Ranger policy attributes and access result attributes.
- Policy attributes: resources, users, groups, roles, accessTypes
- Access result attributes: isAudited, actions, accessResult
The following audit filter specifies that accessResult=DENIED will be audited. The isAudited flag specifies whether or not to audit.
```
{'accessResult': 'DENIED', 'isAudited': true}
```
The following audit filter specifies that “resource => /unaudited” will not be audited.
```
{'resources':{'path':{'values':['/unaudited'],'isRecursive':true},
'iAudited': false}
```
The following audit filter specifies that access to resource database=> sys table=> dump by user “use2” will not be audited.
```
{'resources':{'database':{'values':['sys']},'table':{'values':dump']},
'users':['user2'],'isAudited': false }
```
The following audit filter specifies that access result in actions => listStatus, getfileInfo and accessType => execute will not be audited.
```
{{'actions': [ 'listStatus', 'getfileinfo' ], 'accessTypes':['execute'], 'isAudited': false}
```
The following audit filter specifies that access by user "superuser1" and group "supergroup1" will not be audited.
```
{'users':['superuser1'],'groups':['supergroup1'], 'isAudited': false}
```
The following audit filter specifies that access to any resource tagged as NO_AUDIT will not be audited.
```
{'resources':{'tag':{'values':['NO_AUDIT']}},'isAudited': false}
```

Default audit filters

You can create default audit filters for each Ranger service, which can then be modified as needed by Admin users.

HDFS service:

"ranger.plugin.audit.filters": 
"[ {'accessResult': 'DENIED', 'isAudited': true}, 
{'resources':{'path':{'values':['*/hive-staging','*/staging', '*/sparkStaging', 
'*/_impala_insert_staging', '/user/history/done_intermediate','/user/spark/spark2ApplicationHistory',
'/user/spark/ApplicationHistory', '/user/hue/.cloudera_manager_hive_metastore_canary',
'/user/hue/.Trash/Current/user/hue/.cloudera_manager_hive_metastore_canary',
'/tmp', '/user/oozie/share/lib', '/hbase/archive', '/hbase/oldWALs', '/hbase/MasterProcWALs'],
'isRecursive':true}}, 'isAudited': false}, {'actions':['delete','rename*'],'isAudited':true}, 
{'users':['cloudera-scm','dr.who','hbase','hive','impala','mapred','solr','spark', 'hue'], 
'isAudited':false}, {'users':['hdfs'], 'actions': ['listStatus', 'getfileinfo', 'listCachePools',
'listCacheDirectives'], 'isAudited': false}, {'actions': ['getfileinfo'], 'isAudited':true} ]"

HBase service:

"ranger.plugin.audit.filters":"[{'resources':{'table':{'values':['*-ROOT-*','*.META.*', '*_acl_*', 
'hbase:meta', 'hbase:acl']}}, 'isAudited': false }, {'resources':{'table':{'values':['atlas_janus',
'ATLAS_ENTITY_AUDIT_EVENTS']},'column-family':{'values':['*']},'column':{'values':['*']}},
'isAudited':false},{'users':['hbase'], 'actions':['balance'],'isAudited':false}]"

Hive service:

"ranger.plugin.audit.filters": "[ {'accessResult': 'DENIED', 'isAudited': true}, {'accessTypes':['_any'],
'isAudited':false}, {'actions':['METADATA OPERATION'], 'isAudited': false}, {'users':['hive','hue'],
'actions':['SHOW_ROLES'],'isAudited':false} ]"

Kafka service:

"ranger.plugin.audit.filters":"[{'resources':{'consumergroup':{'values':['atlas','ranger_entities_consumer']}},
'isAudited':false},{'resources':{'topic':{'values':['ATLAS_*']}},'isAudited':false},{'users':['rangertagsync',
'kafka','atlas'],'isAudited':false}]"

ADLS service:

"ranger.plugin.audit.filters": "[ {'accessResult': 'DENIED', 'isAudited': true}, 
{'users':['hive','hdfs','zeppelin','hbase','solr','kafka'],'isAudited':false} ]"