Audit Events and Audit Reports

Required Role:

An audit event is an event that describes an action of accessing a service. An audit report , is a collection of audit events that satisfy a set of filters.

Audit events are recorded by the Cloudera Navigator Audit Server. Audit report metadata is recorded by the Cloudera Navigator Metadata Server.

The following properties can appear in an audit event entry:

Timestamp - Date and time the action was performed. The server stores the timestamp in the timezone of the server and the UI displays the timestamp converted to the local timezone.
Operation - The action performed.
- HBase - createTable, deleteTable, modifyTable, addColumn, modifyColumn, deleteColumn, enableTable, disableTable, move, assign, unassign, balance, balanceSwitch, shutdown, stopMaster, flush, split, compact, compactSelection, getClosestRowBefore, get, exists, put, delete, checkAndPut, checkAndDelete, incrementColumnValue, append, increment, scannerOpen, grant, revoke
- HDFS - setPermission, setOwner, open, concat, setTimes, createSymlink, setReplication, create, append, rename, delete, getfileinfo, mkdirs, listStatus, fsck
- Hive - EXPLAIN, LOAD, EXPORT, IMPORT, CREATEDATABASE, DROPDATABASE, SWITCHDATABASE, DROPTABLE, DESCTABLE, DESCFUNCTION, MSCK, ALTERTABLE_ADDCOLS, ALTERTABLE_REPLACECOLS, ALTERTABLE_RENAMECOL, ALTERTABLE_RENAMEPART, ALTERTABLE_RENAME, ALTERTABLE_DROPPARTS, ALTERTABLE_ADDPARTS, ALTERTABLE_TOUCH, ALTERTABLE_ARCHIVE, ALTERTABLE_UNARCHIVE, ALTERTABLE_PROPERTIES, ALTERTABLE_SERIALIZER, ALTERPARTITION_SERIALIZER, ALTERTABLE_SERDEPROPERTIES, ALTERPARTITION_SERDEPROPERTIES, ALTERTABLE_CLUSTER_SORT, SHOWDATABASES, SHOWTABLES, SHOW_TABLESTATUS, SHOW_TBLPROPERTIES, SHOWFUNCTIONS, SHOWINDEXES, SHOWPARTITIONS, SHOWLOCKS, CREATEFUNCTION, DROPFUNCTION, CREATEVIEW, DROPVIEW, CREATEINDEX, DROPINDEX, ALTERINDEX_REBUILD, ALTERVIEW_PROPERTIES, LOCKTABLE, UNLOCKTABLE, ALTERTABLE_PROTECTMODE, ALTERPARTITION_PROTECTMODE, ALTERTABLE_FILEFORMAT, ALTERPARTITION_FILEFORMAT, ALTERTABLE_LOCATION, ALTERPARTITION_LOCATION, CREATETABLE, CREATETABLE_AS_SELECT, QUERY, ALTERINDEX_PROPS, ALTERDATABASE, DESCDATABASE, ALTER_TABLE_MERGE, ALTER_PARTITION_MERGE, GRANT_PRIVILEGE, REVOKE_PRIVILEGE, SHOW_GRANT, GRANT_ROLE, REVOKE_ROLE, SHOW_ROLE_GRANT, CREATEROLE, DROPROLE
- Impala - Query, Insert, Update, Delete, GRANT_PRIVILEGE, REVOKE_PRIVILEGE, SHOW_GRANT, GRANT_ROLE, REVOKE_ROLE, SHOW_ROLE_GRANT, CREATEROLE, DROPROLE
- Sentry - GRANT_PRIVILEGE, REVOKE_PRIVILEGE, ADD_ROLE_TO_GROUP, DELETE_ROLE_FROM_GROUP, CREATE_ROLE, DROP_ROLE
Username - The name of the user that performed the action.
Impersonator - If the action was requested by another service, the name of the user that invoked the service action on behalf of the user.
- When Sentry is not enabled, the Impersonator field always shows.
- When Sentry is enabled, the Impersonator field shows for services other than Hive.
IP Address - The IP address of the host where the service action occurred.
Service Name - The name of the service that performed the service action.

Continue reading:

Viewing Audit Events
Filtering Audit Events
Creating Audit Reports
Editing Audit Reports
Downloading Audit Events

Viewing Audit Events

Start and log into the Navigator UI.
Click the Audits tab. The Audit Events report displays all audit events that occurred during the last hour.

Filtering Audit Events

Specifying a Time Range

Click the date-time range at the top right of the audits page.
Do one of the following:
- Click a Last n hours link.
- Specify a custom range:
  1. Click Custom range.
  2. In the Selected Range endpoints, click each endpoint and specify a date and time in the date control fields.
    - Date - Click the down arrow to display a calendar and select a date, or click a subfield and click the spinner arrows or up and down arrow keys.
    - Time - Click the hour, minute, and AM or PM fields and click the spinner arrows or up and down arrow keys to specify the value.
    - Move between fields using the right and left arrow keys.
Click Apply.

Adding a Filter

Click the icon that displays next to a property when you hover in one of the event entries. A filter containing the property, operator, and its value is added to the list of filters at the top and Cloudera Navigator redisplays all events that match the filter.
Click the Filters link. The filters pane displaysand a filter control with property, operation, and value fields is added to the list of filters.
1. Choose a property in the drop-down list. You can search by properties such as Username, Service Name, or Operation. The properties vary depending on the service or role.
2. Choose an operator in the operator drop-down list.
3. Type a property value in the value text field. To match a substring, use the like operator. For example, to see all the audit events for files created in the folder /user/joe/out specify Source like /user/joe/out.
4. Click Apply. The property, operation, and value display above the list of audit events and the list of events displays all events that match the filter criteria.
5. Click Add New Filter to add more filters and repeat steps 1 through 4.

Removing a Filter

Do one of the following:
- Click the x next to the filter above the list of events.
- Remove from the Filters pane:
  1. Click the Filters link. The filters pane displays.
  2. Click the at the right of the filter.
  3. Click Apply.The filter is removed from above the list of audit event and the list of events displays all events that match the filter criteria.

Creating Audit Reports

Start and log into the Navigator UI.
Click the Audits tab. The Audit Events report displays all audit events that occurred during the last hour.
Do one of the following:
- Save a filtered version of the Audit Events report:
  1. Optionally specify filters.
  2. Click Save As Report.
- Create a new report:
  1. Click Create New Report.
Enter a report name.
In the Default time range field, specify a relative time range. If you had specified a custom absolute time range before selecting Save As Report, the custom absolute time range is discarded.
Optionally add filters.
Click Save.

Editing Audit Reports

Start and log into the Navigator UI.
Click the Audits tab. The Audit Events report displays all audit events that occurred during the last hour.
In the left pane, click a report name.
Click Edit Report.
In the Default time range field, specify a relative time range. If you had specified a custom absolute time range before selecting Save As Report, the custom absolute time range is discarded.
Optionally add filters.
Click Save.

Downloading Audit Events

You can download audit events in the Audit UI or using the Audit API. An audit event contains the following fields: timestamp, service, username, ipAddress, command, resource, allowed, [operationText], serviceValues. The structure of the resource and serviceValues fields depends on the type of the service. Hive, Hue, Impala, and Sentry events have the operationText field, which contains the operation string.

Downloading Audit Events Using the Audit UI

Start and log into the Navigator UI.
Click the Audits tab. The Audit Events report displays all audit events that occurred during the last hour.
In the left pane, click a report name.
Select Export > format, where format is CSV or JSON.

HDFS Audit Log Example

{
  "items" : [ {
  "timestamp" : "2014-10-10T16:39:25.656Z",
  "service" : "HDFS-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "setPermission",
  "resource" : "/user/hive/warehouse/sample_09/000000_0",
  "allowed" : true,
  "serviceValues" : {
    "dest" : null,
    "delegation_token_id" : null,
    "permissions" : "rwxrwxrwt",
    "src" : "/user/hive/warehouse/sample_09/000000_0"
  }
}, {
  "timestamp" : "2014-10-10T16:39:25.632Z",
  "service" : "HDFS-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "setPermission",
  "resource" : "/user/hive/warehouse/sample_09",
  "allowed" : true,
  "serviceValues" : {
    "dest" : null,
    "delegation_token_id" : null,
    "permissions" : "rwxrwxrwt",
    "src" : "/user/hive/warehouse/sample_09"
  }
}, {
  "timestamp" : "2014-10-10T16:39:25.606Z",
  "service" : "HDFS-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "setOwner",
  "resource" : "/user/hive/warehouse/sample_09",
  "allowed" : false,
  "serviceValues" : {
    "dest" : null,
    "delegation_token_id" : null,
    "permissions" : null,
    "src" : "/user/hive/warehouse/sample_09"
  }
}, {
  "timestamp" : "2014-10-10T16:39:25.590Z",
  "service" : "HDFS-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "delete",
  "resource" : "/user/hive/warehouse/sample_09",
  "allowed" : true,
  "serviceValues" : {
    "dest" : null,
    "delegation_token_id" : null,
    "permissions" : null,
    "src" : "/user/hive/warehouse/sample_09"
  }
}, {
  "timestamp" : "2014-10-10T16:39:25.581Z",
  "service" : "HDFS-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "getfileinfo",
  "resource" : "/user/hive/warehouse",
  "allowed" : true,
  "serviceValues" : {
    "dest" : null,
    "delegation_token_id" : null,
    "permissions" : null,
    "src" : "/user/hive/warehouse"
  }
}, {
  "timestamp" : "2014-10-10T16:39:25.575Z",
  "service" : "HDFS-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "getfileinfo",
  "resource" : "/user/hive/warehouse/sample_09",
  "allowed" : true,
  "serviceValues" : {
    "dest" : null,
    "delegation_token_id" : null,
    "permissions" : null,
    "src" : "/user/hive/warehouse/sample_09"
    }
  } ]
}

In this example, the first event access was denied, and therefore the allowed field has the value false.

Hive Example - via downloaded JSON file

The following records list Hive operations to create and load a table:

[ {
  "timestamp" : "2014-10-10T16:39:26.184Z",
  "service" : "HIVE-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "QUERY",
  "resource" : "default:sample_09",
  "operationText" : "INSERT OVERWRITE \n  TABLE sample_09 \nSELECT \n  sample_07.code,sample_08.description \n  FROM sample_07 \n  JOIN sample_08 \n  WHERE sample_08.code = sample_07.code",
  "allowed" : true,
  "serviceValues" : {
    "object_type" : "TABLE",
    "database_name" : "default",
    "operation_text" : "INSERT OVERWRITE \n  TABLE sample_09 \nSELECT \n  sample_07.code,sample_08.description \n  FROM sample_07 \n  JOIN sample_08 \n  WHERE sample_08.code = sample_07.code",
    "resource_path" : "/user/hive/warehouse/sample_09",
    "table_name" : "sample_09"
  }
}, {
  "timestamp" : "2014-10-10T16:39:26.183Z",
  "service" : "HIVE-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "QUERY",
  "resource" : "default:sample_07",
  "operationText" : "INSERT OVERWRITE \n  TABLE sample_09 \nSELECT \n  sample_07.code,sample_08.description \n  FROM sample_07 \n  JOIN sample_08 \n  WHERE sample_08.code = sample_07.code",
  "allowed" : true,
  "serviceValues" : {
    "object_type" : "TABLE",
    "database_name" : "default",
    "operation_text" : "INSERT OVERWRITE \n  TABLE sample_09 \nSELECT \n  sample_07.code,sample_08.description \n  FROM sample_07 \n  JOIN sample_08 \n  WHERE sample_08.code = sample_07.code",
    "resource_path" : "/user/hive/warehouse/sample_07",
    "table_name" : "sample_07"
  }
}, {
  "timestamp" : "2014-10-10T16:39:26.182Z",
  "service" : "HIVE-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "QUERY",
  "resource" : "default:sample_08",
  "operationText" : "INSERT OVERWRITE \n  TABLE sample_09 \nSELECT \n  sample_07.code,sample_08.description \n  FROM sample_07 \n  JOIN sample_08 \n  WHERE sample_08.code = sample_07.code",
  "allowed" : true,
  "serviceValues" : {
    "object_type" : "TABLE",
    "database_name" : "default",
    "operation_text" : "INSERT OVERWRITE \n  TABLE sample_09 \nSELECT \n  sample_07.code,sample_08.description \n  FROM sample_07 \n  JOIN sample_08 \n  WHERE sample_08.code = sample_07.code",
    "resource_path" : "/user/hive/warehouse/sample_08",
    "table_name" : "sample_08"
  }
}, {
  "timestamp" : "2014-10-10T16:38:18.604Z",
  "service" : "HIVE-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "CREATETABLE",
  "resource" : "default:sample_09",
  "operationText" : "CREATE TABLE sample_09 (code string,description string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\\t' STORED AS TextFile",
  "allowed" : true,
  "serviceValues" : {
    "object_type" : "TABLE",
    "database_name" : "default",
    "operation_text" : "CREATE TABLE sample_09 (code string,description string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\\t' STORED AS TextFile",
    "resource_path" : "",
    "table_name" : "sample_09"
  }
}, {
  "timestamp" : "2014-10-10T16:38:18.602Z",
  "service" : "HIVE-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "CREATETABLE",
  "resource" : "default:",
  "operationText" : "CREATE TABLE sample_09 (code string,description string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\\t' STORED AS TextFile",
  "allowed" : true,
  "serviceValues" : {
    "object_type" : "DATABASE",
    "database_name" : "default",
    "operation_text" : "CREATE TABLE sample_09 (code string,description string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\\t' STORED AS TextFile",
    "resource_path" : "/user/hive/warehouse",
    "table_name" : ""
  }
}, {
  "timestamp" : "2014-10-10T16:37:06.836Z",
  "service" : "HIVE-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "LOAD",
  "resource" : ":",
  "operationText" : "LOAD DATA INPATH\n      '/user/admin/sample_08' OVERWRITE INTO TABLE sample_08",
  "allowed" : true,
  "serviceValues" : {
    "object_type" : "DFS_DIR",
    "database_name" : "",
    "operation_text" : "LOAD DATA INPATH\n      '/user/admin/sample_08' OVERWRITE INTO TABLE sample_08",
    "resource_path" : "/user/admin/sample_08",
    "table_name" : ""
  }
}, {
  "timestamp" : "2014-10-10T16:37:06.836Z",
  "service" : "HIVE-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "LOAD",
  "resource" : "default:sample_08",
  "operationText" : "LOAD DATA INPATH\n      '/user/admin/sample_08' OVERWRITE INTO TABLE sample_08",
  "allowed" : true,
  "serviceValues" : {
    "object_type" : "TABLE",
    "database_name" : "default",
    "operation_text" : "LOAD DATA INPATH\n      '/user/admin/sample_08' OVERWRITE INTO TABLE sample_08",
    "resource_path" : "/user/hive/warehouse/sample_08",
    "table_name" : "sample_08"
  }
}, {
  "timestamp" : "2014-10-10T16:37:05.752Z",
  "service" : "HIVE-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "DESCTABLE",
  "resource" : "default:sample_08",
  "operationText" : "DESCRIBE EXTENDED sample_08",
  "allowed" : true,
  "serviceValues" : {
    "object_type" : "TABLE",
    "database_name" : "default",
    "operation_text" : "DESCRIBE EXTENDED sample_08",
    "resource_path" : "/user/hive/warehouse/sample_08",
    "table_name" : "sample_08"
  }
}, {
  "timestamp" : "2014-10-10T16:37:05.379Z",
  "service" : "HIVE-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "LOAD",
  "resource" : "default:sample_07",
  "operationText" : "LOAD DATA INPATH\n      '/user/admin/sample_07' OVERWRITE INTO TABLE sample_07",
  "allowed" : true,
  "serviceValues" : {
    "object_type" : "TABLE",
    "database_name" : "default",
    "operation_text" : "LOAD DATA INPATH\n      '/user/admin/sample_07' OVERWRITE INTO TABLE sample_07",
    "resource_path" : "/user/hive/warehouse/sample_07",
    "table_name" : "sample_07"
  }
}, {
  "timestamp" : "2014-10-10T16:37:05.377Z",
  "service" : "HIVE-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "LOAD",
  "resource" : ":",
  "operationText" : "LOAD DATA INPATH\n      '/user/admin/sample_07' OVERWRITE INTO TABLE sample_07",
  "allowed" : true,
  "serviceValues" : {
    "object_type" : "DFS_DIR",
    "database_name" : "",
    "operation_text" : "LOAD DATA INPATH\n      '/user/admin/sample_07' OVERWRITE INTO TABLE sample_07",
    "resource_path" : "/user/admin/sample_07",
    "table_name" : ""
  }
}, {
  "timestamp" : "2014-10-10T16:37:00.002Z",
  "service" : "HIVE-1",
  "username" : "admin",
  "ipAddress" : "10.20.190.241",
  "command" : "DESCTABLE",
  "resource" : "default:sample_07",
  "operationText" : "DESCRIBE EXTENDED sample_07",
  "allowed" : true,
  "serviceValues" : {
    "object_type" : "TABLE",
    "database_name" : "default",
    "operation_text" : "DESCRIBE EXTENDED sample_07",
    "resource_path" : "/user/hive/warehouse/sample_07",
    "table_name" : "sample_07"
  }
} ]

Downloading Audit Events Using the Audit API

You can filter and download audit events using the Cloudera Navigator API.

Hive Example - via audit API

To download the audits events using the API, issue the request http://host-1.ent.cloudera.com:7187/api/v3/audits?query=service==*HIVE*, which could return the following JSON items:

{
  "items" : [ {
    "timestamp" : "2014-10-07T21:09:05.804Z",
    "service" : "HIVE-1",
    "username" : "test",
    "impersonator" : "",
    "ipAddress" : "20.10.191.128",
    "command" : "CREATEROLE",
    "resource" : ":",
    "operationText" : "CREATE ROLE bad_role",
    "allowed" : false,
    "serviceValues" : {
      "object_type" : "UNKNOWN",
      "database_name" : "",
      "operation_text" : "CREATE ROLE bad_role",
      "resource_path" : "",
      "table_name" : ""
    }
  }, {
    "timestamp" : "2014-10-07T21:08:52.036Z",
    "service" : "HIVE-1",
    "username" : "test",
    "ipAddress" : "20.10.191.128",
    "command" : "DROPTABLE",
    "resource" : "default:ratings_sum",
    "operationText" : "DROP TABLE ratings_sum",
    "allowed" : true,
    "serviceValues" : {
      "object_type" : "TABLE",
      "database_name" : "default",
      "operation_text" : "DROP TABLE ratings_sum",
      "resource_path" : "/user/hive/warehouse/ratings_sum",
      "table_name" : "ratings_sum"
    }
  } ]
}

Auditing

Downloading HDFS Directory Access Permission Reports