Managing Auditing
Also available as:
PDF

Migrating Audit Logs from DB to Solr in Ambari Clusters

How to migrate your audit logs from DB to Solr. It is recommended that you store audits in both HDFS and Solr.

Before you migrate your audit logs from DB to Solr, make sure your cluster meets the following requirements:
  • Solr must be installed and running (see “Using Apache Solr for Ranger Audits”).
  • All plug-ins must be upgraded and writing audit logs to Solr (i.e., plugins must not be writing audit logs to DB.)
  • The DB server must be running, and the credentials used to connect Audit to DB must be available.
  • Ranger must be running with the audit source as Solr, and the Solr URL must be configured.
Audit to DB is no longer recommended and the option is disabled in the Ambari UI. If your logs were previously stored on DB, you can migrate the logs to Solr.
  1. Configure the properties ranger.audit.source.type and ranger.audit.solr.urls:
    Property Name Sample Value Location
    ranger.audit.source.type solr Ranger>Configs>Advanced>Advanced ranger-admin-site
    ranger.audit.solr.urls Syntax: http://<solr_host>:<port>/solr/ranger_audits Example: http://192.168.0.2:8983/solr/ranger_auditsExample: http://192.168.0.2:8886/solr/ranger_audits Ranger>Configs>Ranger Audit
  2. Verify or enter the ranger.jpa.audit.jdbc.url value.

  3. After upgrading Ranger and changing the audit log destination from DB to Solr, Ambari may not automatically populate the required property values. If necessary, you can add these as custom properties from Ambari.
    1. Select Ranger>Configs>Advanced>Custom ranger-admin-site, then click Add Property....
    2. Enter the following information on the Add Property pop-up:
      • Type: preloaded with the value ranger-admin-site.xml
      • Key: enter ranger.jpa.audit.jdbc.url
      • Value: enter the JDBC audit string for your DB platform:
      Table 1. JDBC Audit String
      DB Platform Syntax Example Value
      MySQL jdbc:mysql://DB_HOST:PORT/audit_name jdbc:mysql://c6401.ambari.apache.org:3306/ranger_audit
      Oracle

      For Oracle SID:

      jdbc:oracle:thin:@AUDIT_HOST:PORT:SID

      jdbc:oracle:thin:@c6401.ambari.apache.org:1521:ORCL

      For Oracle Service Name:

      jdbc:oracle:thin:@//AUDIT_HOST[:PORT][/ServiceName]

      jdbc:oracle:thin:@//c6401.ambari.apache.org:1521/XE
      PostgreSQL jdbc:postgresql://AUDIT_HOST/audit_name jdbc:postgresql://c6401.ambari.apache.org:5432/ranger_audit
      MS SQL jdbc:sqlserver://AUDIT_HOST;databaseName=audit_name jdbc:sqlserver://c6401.ambari.apache.org:1433;databaseName=ranger_audit
      SQLA jdbc:sqlanywhere:host=AUDIT_HOST;database=audit_name jdbc:sqlanywhere:host=c6401.ambari.apache.org:2638;database=ranger_audit
  4. Restart Ranger Admin: service ranger-admin restart.
  5. Navigate to the Ranger admin directory and run the following command:
    $/path/to/java -Dlogdir=ews/logs -Dlog4j.configuration=db_patch.log4j.xml -cp ews/webapp/WEB-INF/classes/conf:ews/webapp/WEB-INF/classes/lib/*:ews/webapp/WEB-INF/:ews/webapp/META-INF/:ews/webapp/WEB-INF/lib/*:ews/webapp/WEB-INF/classes/:ews/webapp/WEB-INF/classes/META-INF:/usr/share/java/mysql-connector-java.jar org.apache.ranger.patch.cliutil.DbToSolrMigrationUtil
    If the script succeeds, it prints the following details on the screen:
    • Processing batch 'n' of total 'noOfBatches' (Where each batch contains 10000 rows.)
    • Total number of migrated audit logs.

    If the script fails to migrate data, it returns the error: Migration process failed, Please refer ranger_db_patch.log file.