Service Auditing Properties

Each service (with exceptions noted) that supports auditing has the following properties:
  • Enable Audit Collection - See Audit Log Properties.
  • Audit Event Filter - A set of rules that capture properties of auditable events and actions to be performed when an event matches those properties. The Cloudera Manager Agent uses this property to filter events out before they are sent to Cloudera Navigator. This property is not supported for Sentry, Solr, or the Navigator Metadata Server. The default filter settings discard the following events:
    • HDFS - generated by the internal Cloudera and Hadoop users (cloudera-scm, hdfs, hbase, hive, impala, mapred, solr, spark, and dr.who), events generated by the hdfs user running the listStatus, listCachePools, listCacheDirectives, and getfileinfo operations, and that affect files in the /tmp directory.
    • HBase - that affect the -ROOT-, .META., and acl tables
    • Hive - generated by Hive MapReduce jobs in the /tmp directory
    • Impala - no default filter.
  • Audit Event Tracker - A set of rules for tracking and coalescing events. This feature is used to define equivalency between different audit events. Tracking works by keeping a reference to events when they first appear, and comparing other incoming events against the tracked events according to the rules defined. When events match, according to a set of configurable parameters, only one entry in the audit list is generated for all the matching events. This property is not supported for the Navigator Metadata Server.
  • Audit Queue Policy - The action to take when the audit event queue is full. The options are Drop or Shutdown. When a queue is full and the queue policy of the service is Shutdown, before shutting down the service, N audits will be discarded, where N is the size of the Cloudera Navigator Audit Server queue. This property is not supported for the Navigator Metadata Server.

The Audit Event Filter and Audit Event Tracker rules for filtering and coalescing events are expressed as JSON objects.

You can edit these rules using a rule editor:


or in a JSON text field:


For information on the structure of the objects, and the properties for which you can set filters, display the description on the configuration page as follows:
  1. In the Cloudera Manager Admin Console, go to a service that supports auditing.
  2. Click the Configuration tab.
  3. Select Scope > Service (Service-Wide).
  4. Select Category > Cloudera Navigator category.
  5. In Audit Event Tracker row, click . For example, the Hive properties are:
    • userName: the user performing the action.
    • ipAddress: the IP from where the request originated.
    • operation: the Hive operation being performed.
    • databaseName: the databaseName for the operation.
    • tableName: the tableName for the operation.

Continue reading:

Configuring Service Auditing Properties

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

Follow this procedure for all cluster services that support auditing. In addition, for Impala and Solr auditing, perform the steps in Configuring Impala Daemon Logging, Enabling Solr Auditing.
  1. Go to a service that supports auditing.
  2. Click the Configuration tab.
  3. Select Scope > Service (Service-Wide).
  4. Select Category > Cloudera Navigator category.
  5. Edit the properties.
  6. Click Save Changes to commit the changes.
  7. Restart the service.

Configuring Impala Daemon Logging

Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)

To control whether the Impala Daemon role logs to the audit log:
  1. Click the Impala service.
  2. Click the Configuration tab.
  3. Select Scope > Impala Daemon.
  4. Select Category > Logs.
  5. Edit the Enable Impala Audit Event Generation.
  6. Click Save Changes to commit the changes.
  7. Restart the service.
To set the log file size:
  1. Click the Impala service.
  2. Select Scope > Impala Daemon.
  3. Select Category > Logs.
  4. Set the Impala Daemon Maximum Audit Log File Size property.
  5. Click Save Changes to commit the changes.
  6. Restart the service.

Enabling Solr Auditing

Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)

Solr auditing is disabled by default. To enable auditing:
  1. Enable Sentry authorization for Solr following the procedure in Enabling Sentry Authorization for Solr.
  2. Go to the Solr service.
  3. Click the Configuration tab.
  4. Select Scope > Solr Service (Service-Wide)
  5. Select Category > Policy File Based Sentry category.
  6. Select or deselect the Enable Sentry Authorization checkbox.
  7. Select Category > Cloudera Navigator category.
  8. Select or deselect the Enable Audit Collection checkbox. See Audit Log Properties.
  9. Click Save Changes to commit the changes.
  10. Restart the service.

Enabling and Disabling Navigator Metadata Server Auditing

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

Navigator Metadata Server auditing is enabled by default. To enable or disable auditing:
  1. Do one of the following:
    • Select Clusters > Cloudera Management Service > Cloudera Management Service.
    • On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link.
  2. Click the Configuration tab.
  3. Select Scope > Navigator Metadata Server
  4. Select Category > Cloudera Navigator category.
  5. Select or deselect the Enable Audit Collection checkbox.
  6. Click Save Changes to commit the changes.
  7. Restart the service.

Audit Logging to Syslog

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

The Audit Server logs all audit records into a Log4j logger called auditStream. The log messages are logged at the TRACE level, with the attributes of the audit records. By default, the auditStream logger is inactive because the logger level is set to FATAL. It is also connected to a NullAppender, and does not forward to other appenders (additivity set to false).

To record the audit stream, configure the auditStream logger with the desired appender. For example, the standard SyslogAppender allows you to send the audit records to a remote syslog.

The Log4j SyslogAppender supports only UDP. An example syslog configuration would be: 
$ModLoad imudp
$UDPServerRun 514 
# Accept everything (even DEBUG messages) local2.* /my/audit/trail.log
It is also possible to attach other appenders to the auditStream to provide other integration behaviors.
You can audit events to syslog in two formats: JSON and RSA EnVision. To configure audit logging to syslog, do the following:
  1. Do one of the following:
    • Select Clusters > Cloudera Management Service > Cloudera Management Service.
    • On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link.
  2. Click the Configuration tab.
  3. Locate the Navigator Audit Server Logging Advanced Configuration Snippet property by typing its name in the Search box.
  4. Depending on the format type, enter: 
    log4j.logger.auditStream = TRACE,SYSLOG
log4j.appender.SYSLOG = org.apache.log4j.net.SyslogAppender
log4j.appender.SYSLOG.SyslogHost = hostname
log4j.appender.SYSLOG.Facility = Local2
log4j.appender.SYSLOG.FacilityPrinting = true
    To configure the specific stream type, enter:
    Format Properties
    JSON 
    log4j.additivity.auditStream = false
    RSA EnVision 
    log4j.additivity.auditStreamEnVision = false
  5. Click Save Changes to commit the changes.

Example Log Messages

Format Log Message Example
JSON 
Jul 23 11:05:15 hostname local2:
{"type":"HDFS","allowed":"true","time":"1374602714758",
"service":"HDFS-1",
"user":"root","ip":"10.20.93.93","op":"mkdirs","src":"/audit/root","perms":"rwxr-xr-x"}
RSA EnVision 
Cloudera|Navigator|1|type="Hive",allowed="false",time="1382551146763",
service="HIVE-1",user="systest",impersonator="",ip="/10.20.190.185",op="QUERY",
opText="select count(*) from sample_07",db="default",table="sample_07",path="/user/hive/warehouse/sample_07",objType="TABLE"
If a particular field is not applicable for that audit event, it is omitted from the message.