Service Auditing Properties

Describes service auditing properties and how to configure the properties.

Each service (with exceptions noted) that supports auditing configuration has the following properties:
  • Enable Collection - Controls whether the Cloudera Manager Agent tracks a service's audit log file. A validation check is performed for all life cycle actions (stop/start/restart). If the Enable Collection flag is selected and the Audit Log Directory property is not set, the validator displays a message that says that the Audit Log Directory property must be set to enable auditing.
  • Event Filter - A set of rules that capture properties of auditable events and actions to be performed when an event matches those properties. This property is not supported for Sentry.
  • Event Tracker - A set of rules for tracking and coalescing events. This feature is used to define equivalency between different audit events. When events match, according to a set of configurable parameters, only one entry in the audit list is generated for all the matching events.
  • Queue Policy - The action to take when the audit event queue is full. The options are Drop or Shutdown. When a queue is full and the queue policy of the service is Shutdown, before shutting down the service, N audits will be discarded, where N is the size of the Cloudera Navigator Audit Server queue.

The Event Filter and Event Tracker rules for filtering and coalescing events are expressed as JSON objects. For information on the structure of the objects, see the description on the configuration page within the Cloudera Manager Admin Console.

The default event filter discards events generated by the internal Cloudera and Hadoop users (cloudera-scm, hdfs, hbase, hive, mapred, solr, and dr.who) and that affect files in the /tmp directory.

Continue reading:

Configuring Service Auditing Properties

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

  1. Click a service that supports auditing.
  2. Click the Configuration tab.
  3. Click the Cloudera Navigator category. The Service-Wide category displays.
  4. Edit the properties.
  5. Click Save Changes.
  6. Restart the service.

Configuring Impala Daemon Logging

Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)

To control whether the Impala daemon logs to the audit log:
  1. Click the Impala service.
  2. Click the Configuration tab.
  3. Expand the Impala Daemon Default Group > Logs category.
  4. Edit the Enable Impala Audit Event Generation checkbox setting.
  5. Click Save Changes.
  6. Restart the service.

Audit Logging to Syslog

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

The Audit Server logs all audit records into a Log4j logger called auditStream. The log messages are logged at the TRACE level, with the attributes of the audit records. By default, the auditStream logger is inactive because the logger level is set to FATAL. It is also connected to a NullAppender, and does not forward to other appenders (additivity set to false).

To record the audit stream, configure the auditStream logger with the desired appender. For example, the standard SyslogAppender allows you to send the audit records to a remote syslog.

The Log4j SyslogAppender supports only UDP. An example syslog configuration would be: 
$ModLoad imudp 
$UDPServerRun 514 
# Accept everything (even DEBUG messages) local2.* /my/audit/trail.log
It is also possible to attach other appenders to the auditStream to provide other integration behaviors.
You can audit events to syslog in two formats: JSON and RSA EnVision. To configure audit logging to syslog, do the following:
  1. Do one of the following:
    • Select Clusters > Cloudera Management Service > Cloudera Management Service.
    • On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link.
  2. Click the Configuration tab.
  3. Search for Navigator Audit Server Logging Advanced Configuration Snippet.
  4. Click the Value field and depending on the format type, enter: 
    log4j.logger.auditStream = TRACE,SYSLOG
log4j.appender.SYSLOG = org.apache.log4j.net.SyslogAppender
log4j.appender.SYSLOG.SyslogHost = hostname
log4j.appender.SYSLOG.Facility = Local2
log4j.appender.SYSLOG.FacilityPrinting = true
    To configure the specific stream type, enter:
    Format Properties
    JSON 
    log4j.additivity.auditStream = false
    RSA EnVision 
    log4j.additivity.auditStreamEnVision = false
  5. Click Save Changes to commit the changes.

Example Log Messages

Format Log Message Example
JSON 
Jul 23 11:05:15 hostname local2: 
{"type":"HDFS","allowed":"true","time":"1374602714758",
"service":"HDFS-1",
"user":"root","ip":"10.20.93.93","op":"mkdirs","src":"/audit/root","perms":"rwxr-xr-x"}
RSA EnVision 
Cloudera|Navigator|1|type="Hive",allowed="false",time="1382551146763",
service="HIVE-1",user="systest",impersonator="",ip="/10.20.190.185",op="QUERY",
opText="select count(*) from sample_07",db="default",table="sample_07",path="/user/hive/warehouse/sample_07",objType="TABLE"
If a particular field is not applicable for that audit event, it is omitted from the message.