Enable Audit Logging in Non-Ambari Clusters

How to enable audit logging for HDFS.

It is recommended that Ranger audits be written to both Solr and HDFS. Audits to Solr are primarily used to enable queries from the Ranger Admin UI. HDFS is a long-term destination for audits; audits stored in HDFS can be exported to any SIEM system, or to another audit store.

Enable audit logging:
1. Set the XAAUDIT.HDFS.ENABLE value to "true" for the component plug-in in the install.properties file, which can be found here:/usr/hdp/<version>/ranger-<component>=plugin.
2. Configure the NameNode host in the XAAUDIT.HDFS.HDFS_DIR field.
3. Create a policy in the HDFS service from the Ranger Admin for individual component users (hive/hbase/knox/storm/yarn/kafka/kms) to provide READ and WRITE permissions for the audit folder (i.e., for enabling Hive component to log Audits to HDFS, you need to create a policy for the hive user with Read and WRITE permissions for the audit directory).
4. Set the Audit to HDFS caches logs in the local directory, which can be specified in XAAUDIT.HDFS.LOCAL_BUFFER_DIRECTORY (this can be like /var/log/<component>/**), which is the path where the audit is stored for a short time. This is similar for archive logs that need to be updated.
Enable auditing reporting from the Solr database:
1. Modify the following properties in the Ranger service install.properties to enable auditing to the Solr database in Ranger:
  - audit_store=solr
  - For HDP Search's Solr Instance: http:<solr_host>:8983/solr/ranger_audits
    
    For Ambari Infra's Solr Instance: http:<solr_host>:8886/solr/ranger_audits
  - audit_solr_user=ranger_solr
  - audit_solr_password-NONE
2. Restart Ranger: service ranger-admin restart.
Enable auditing to the Solr database for a plug-in (e.g., HBase):
1. Set the following properties in install.properties of the plug-in to begin audit logging to the Solr database:
  - XAAUDIT.SOLR.IS.ENABLED=true
  - XAAUDIT.SOLR.ENABLE=true
  - For HDP Search's Solr Instance: XAAUDIT.SOLR.URL=http://solr_host:8983/solr/ranger_audits
    
    For Ambari Infra's Solr Instance: XAAUDIT.SOLR.URL=http://solr_host:8886/solr/ranger_audits
  - XAAUDIT.SOLR.USER-ranger_solr
  - XAAUDIT.SOLR.PASSWORD=NONE
  - XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hadoop/hdfs/audit/solr/spool
2. Enable the Ranger HBase plug-in.
3. Restart the HBase component.