Cloudera Navigator Auditing

Minimum Required Role: Auditing Viewer (also provided by Full Administrator)

An audit event is an event that describes an action that has been taken for a cluster, host, license, parcel, role, service or user.

Cloudera Manager records cluster, host, license, parcel, role, and service lifecycle events (activate, create, delete, deploy, download, install, start, stop, update, upgrade, and so on), user security-related events (add and delete user, login failed and succeeded), and provides an audit UI and API to view, filter, and export such events. For information on Cloudera Manager auditing features, see Lifecycle and Security Auditing.

The Cloudera Navigator Audit Server records service access events and the Cloudera Navigator Metadata Server provides an audit UI and API to view, filter, and export both service access events and the lifecycle and security events retrieved from Cloudera Manager.

Continue reading:

Viewing Audit Events

  1. Start and log in to the Cloudera Navigator data management component UI.
  2. Click the Audits tab. The Audit Events report displays all audit events that occurred during the last hour.

Filtering Audit Events

You filter audit events by specifying a time range or adding one or more filters containing an audit event field, operator, and value.

Specifying a Time Range

  1. Click the date-time range at the top right of the Audits tab.
  2. Do one of the following:
    • Click a Last n hours link.
    • Specify a custom range:
      1. Click Custom range.
      2. In the Selected Range endpoints, click each endpoint and specify a date and time in the date control fields.
        • Date - Click the down arrow to display a calendar and select a date, or click a field and click the spinner arrows or press the up and down arrow keys.
        • Time - Click the hour, minute, and AM/PM fields and click the spinner arrows or press the up and down arrow keys to specify the value.
        • Move between fields by clicking fields or by using the right and left arrow keys.
  3. Click Apply.

Adding a Filter

  1. Do one of the following:
    • Click the icon that displays next to a field when you hover in one of the event entries.
    • Click the Filters link. The Filters pane displays.
      1. Click Add New Filter to add a filter.
      2. Choose a field in the Select Property... drop-down list. You can search by fields such as username, service name, or operation. The fields vary depending on the service or role. The service name of the Navigator Metadata Server is Navigator.
      3. Choose an operator in the operator drop-down list.
      4. Type a field value in the value text field. To match a substring, use the like operator. For example, to see all the audit events for files created in the folder /user/joe/out, specify Source like /user/joe/out.
    A filter control with field, operation, and value fields is added to the list of filters.
  2. Click Apply. A field, operation, and value breadcrumb is added above the list of audit events and the list of events displays all events that match the filter criteria.

Removing a Filter

  1. Do one of the following:
    • Click the x next to the filter above the list of events. The list of events displays all events that match the filter criteria.
    • Click the Filters link. The Filters pane displays.
      1. Click the at the right of the filter.
      2. Click Apply. The filter is removed from above the list of audit event and the list of events displays all events that match the filter criteria.

Verifying that Auditing is Running

Navigator auditing captures a complete and immutable record of all activity within a system. Depending on how you use this information, you may want to know as soon as possible if auditing is not running. To monitor auditing and ensure that it has not failed without generating a warning message, you can configure an auditing pipeline health check to verify that auditing is working and is not silently down. The health check uses the following metrics to determine if auditing is running:
  • Number of bytes of audits processed.
  • Number of bytes of audits remaining.
  • Number of errors when sending audits from the Cloudera Manager Agent to the Cloudera Manager server.
The audit generates a warning in Cloudera Manager if:
  • The number of bytes of audits processed is zero.
  • The number of bytes of audits remaining is not zero.
  • Errors occur when audits are sent.
The health check is run for each role that can generate audits.

Configuring the Audit Pipeline Health Check

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

In Cloudera Manager, you configure the audit pipeline health check as follows:
  1. Go to the service you want to configure.
  2. Click the Configuration tab.
  3. Search for mgmt.navigator.
  4. Edit the following configuration items:
    • Navigator Audit Pipeline Health Check - Select or deselect the check box to enable the audit health check. You can enable the health check for specific groups. By default, all groups are selected.
    • Monitoring Period for Audit Failures - Set the period of time that elapses before a failure warning is sent. The default time period is 20 minutes.
    • Navigator Audit Failure Thresholds - Set the size of the audit failure, in bytes, that triggers a Warning or Critical error message. The value that you specify for this threshold is the number of bytes of audit data that have not been sent to audit server. You can specify different thresholds for Warning and Critical errors. By default, Critical errors are sent for failures of any size.
  5. Click Save Changes.

For example, in the following graphic, the pipeline health check is enabled for all groups in the service. The failure period is set to 15 minutes, and the health check sends a warning for failures of any size and a critical error when more than 2 KiB of audit events have not been sent.



Service Audit Event Fields

The following fields can appear in a service audit event:
Display Name Field Description
Additional Info additional_info JSON text that contains more details about an operation performed on entities in Navigator Metadata Server.
Allowed allowed Indicates whether the request to perform an operation failed or succeeded. A failure occurs if the user is not authorized to perform the action.
Collection Name collection_name The name of the affected Solr collection.
Database Name database_name For Sentry, Hive, and Impala, the name of the database on which the operation was performed.
Delegation Token ID delegation_token_id Delegation token identifier generated by HDFS NameNode that is then used by clients when submitting a job to JobTracker.
Destination dest Path of the final location of an HDFS file in a rename or move operation.
Entity ID entity_id Identifier of a Navigator Metadata Server entity. The ID can be retrieved using the Navigator Metadata Server API.
Event Time timestamp Date and time an action was performed. The Navigator Audit Server stores the timestamp in the timezone of the Navigator Audit Server. The Navigator UI displays the timestamp converted to the local timezone. Exported audit events contain the stored timestamp.
Family family HBase column family.
Impersonator impersonator If an action was requested by another service, the name of the user that invoked the action on behalf of the user.
  • When Sentry is enabled, the Impersonator field displays for services other than Hive.
  • When Sentry is not enabled, the Impersonator field always displays.
IP Address ipAddress The IP address of the host where an action occurred.
Object Type object_type For Sentry, Hive, and Impala, the type of the object (TABLE, VIEW, DATABASE) on which operation was performed.
Operation command The action performed.
  • HBase - createTable, deleteTable, modifyTable, addColumn, modifyColumn, deleteColumn, enableTable, disableTable, move, assign, unassign, balance, balanceSwitch, shutdown, stopMaster, flush, split, compact, compactSelection, getClosestRowBefore, get, exists, put, delete, checkAndPut, checkAndDelete, incrementColumnValue, append, increment, scannerOpen, grant, revoke
  • HDFS - setPermission, setOwner, open, concat, setTimes, createSymlink, setReplication, create, append, rename, delete, getfileinfo, mkdirs, listStatus, fsck, listSnapshottableDirectory
  • HiveServer2 - EXPLAIN, LOAD, EXPORT, IMPORT, CREATEDATABASE, DROPDATABASE, SWITCHDATABASE, DROPTABLE, DESCTABLE, DESCFUNCTION, MSCK, ALTERTABLE_ADDCOLS, ALTERTABLE_REPLACECOLS, ALTERTABLE_RENAMECOL, ALTERTABLE_RENAMEPART, ALTERTABLE_RENAME, ALTERTABLE_DROPPARTS, ALTERTABLE_ADDPARTS, ALTERTABLE_TOUCH, ALTERTABLE_ARCHIVE, ALTERTABLE_UNARCHIVE, ALTERTABLE_PROPERTIES, ALTERTABLE_SERIALIZER, ALTERPARTITION_SERIALIZER, ALTERTABLE_SERDEPROPERTIES, ALTERPARTITION_SERDEPROPERTIES, ALTERTABLE_CLUSTER_SORT, SHOWDATABASES, SHOWTABLES, SHOW_TABLESTATUS, SHOW_TBLPROPERTIES, SHOWFUNCTIONS, SHOWINDEXES, SHOWPARTITIONS, SHOWLOCKS, CREATEFUNCTION, DROPFUNCTION, CREATEVIEW, DROPVIEW, CREATEINDEX, DROPINDEX, ALTERINDEX_REBUILD, ALTERVIEW_PROPERTIES, LOCKTABLE, UNLOCKTABLE, ALTERTABLE_PROTECTMODE, ALTERPARTITION_PROTECTMODE, ALTERTABLE_FILEFORMAT, ALTERPARTITION_FILEFORMAT, ALTERTABLE_LOCATION, ALTERPARTITION_LOCATION, CREATETABLE, CREATETABLE_AS_SELECT, QUERY, ALTERINDEX_PROPS, ALTERDATABASE, DESCDATABASE, ALTER_TABLE_MERGE, ALTER_PARTITION_MERGE, GRANT_PRIVILEGE, REVOKE_PRIVILEGE, SHOW_GRANT, GRANT_ROLE, REVOKE_ROLE, SHOW_ROLE_GRANT, CREATEROLE, DROPROLE
  • Hue - USER_LOGIN, USER_LOGOUT, EDIT_USER, ADD_LDAP_USERS, ADD_LDAP_GROUPS, SYNC_LDAP_USERS_GROUPS, EDIT_GROUP, EDIT_PERMISSION, CREATE_USER, CREATE_GROUP, DELETE_USER, DELETE_GROUP
  • Impala - Query, Insert, Update, Delete, GRANT_PRIVILEGE, REVOKE_PRIVILEGE, SHOW_GRANT, GRANT_ROLE, REVOKE_ROLE, SHOW_ROLE_GRANT, CREATEROLE, DROPROLE, DML (Data Manipulation Language statements)
  • Navigator Metadata Server - auditReport, authorization, metadata, policy, search, savedSearch. For the operation subtypes, see Sub Operation.
  • Sentry - GRANT_PRIVILEGE, REVOKE_PRIVILEGE, ADD_ROLE_TO_GROUP, DELETE_ROLE_FROM_GROUP, CREATE_ROLE, DROP_ROLE
  • Solr - add, commit, deleteById, deleteByQuery, finish, query, rollback, CREATE, CREATEALIAS, CREATESHARD, DELETE, DELETEALIAS, DELETESHARD, LIST, LOAD, LOAD_ON_STARTUP, MERGEINDEXES, PERSIST, PREPRECOVERY, RELOAD, RENAME, REQUESTAPPLYUPDATES, REQUESTRECOVERY, REQUESTSYNCSHARD, SPLIT, SPLITSHARD, STATUS, SWAP, SYNCSHARD, TRANSIENT, UNLOAD
Operation Params operation_params Solr query or update parameters used when performing the action.
Operation Text operation_text For Sentry, Hive, and Impala, the SQL query that was executed by user. For Hue, the user or group that was added, edited, or deleted.
Permissions permissions HDFS permission of the file or directory on which the HDFS operation was performed.
Privilege privilege Privilege needed to perform an Impala operation.
Qualifier qualifier HBase column qualifier.
Query ID query_id The query ID for an Impala operation.
Resource resource A service-dependent combination of multiple fields generated during fetch. This field is not supported for filtering as it is not persisted.
Resource Path resource_path HDFS URL of Hive objects (TABLE, VIEW, DATABASE, and so on)
Service Name service The name of the service that performed the action.
Session ID session_id Impala session ID.
Solr Version solr_version Solr version number.
Source src Path of the HDFS file or directory present in an HDFS operation.
Status status Status of an Impala operation providing more information on success or failure.
Stored Object Name stored_object_name Name of a policy, saved search, or audit report in Navigator Metadata Server.
Sub Operation sub_operation Subtype of operation performed in Navigator Metadata Server. Valid values are:
  • auditReport - fetchAllReports, createAuditReport, deleteAuditReport, updateAuditReport
  • authorization - searchGroup, deleteGroup, fetchGroup, fetchRoles, updateRoles
  • metadata - updateMetadata, fetchMetadata, fetchAllMetadata
  • policy - fetchAllPolicies, createPolicy, deletePolicy, updatePolicy, fetchPolicySchedule, updatePolicySchedule, deletePolicySchedule
  • savedSearch - fetchAllSavedSearches, fetchSavedSearch, createSavedSearch, deleteSavedSearch, updateSavedSearch
Table Name table_name For Sentry, HBase, Hive, and Impala, the name of the table on which action was performed.
Username username The name of the user that performed the action.