Creating Ranger replication policies

Replicate Audit logs

• Cloud Credential On Source – Select a cloud credential to access the target cloud storage to write the audit logs on the target cluster. The cloud credentials that you register for Replication Manager on the Cloud Credentials page are displayed in this field.

  If the required cloud credential is not displayed, click Add Cloud Credential to add the credentials.

• Audit Logs location (on source) – Displays the source Ranger HDFS audit log path by default. For example, hdfs://[***SOURCE URL***]:8020/ranger/audit/

  You can edit the log directory path to replicate only a subset of logs by appending hdfs, hbase, or atlas to the end of the default path. For example, if you append hdfs to the end of the default path, Replication Manager replicates only the HDFS Ranger audit logs.

• Run As Username (on source) – Enter the username to run the replication job. Ensure that the user is in the supergroup group on the source cluster.

• Merge method (default) – Replication Manager merges the Ranger policies.

  For example, if a Ranger policy in the target Ranger service has user1 and the same Ranger policy on the source cluster has user2, both user1 and user2 are added in the target Ranger policy after replication.

• Override method – Replication Manager overwrites the existing Ranger policies.

  For example, if a Ranger policy in the target Ranger service has user1 and the same Ranger policy on the source cluster has user2, user1 is removed and user2 is added in the target Ranger policy after replication.

• Enable replication – Select the field to instruct the replication policy to replicate the Ranger data for the chosen source service to the chosen target service.
• Source service name – Cannot be edited.
• Destination service name – Retain the default service name or select the service name on your target Data Hub cluster. Select the destination service from the dropdown list of services in the target Cloudera Manager that has the same type as the selected source service. For example, you can replicate the source Hive service's Ranger policy to any target Hive service.

Choose one of the following options:

• Does Not Repeat
• Custom – In the Custom Recurrence dialog box, choose the time, date, and the frequency to run the policy.

  Replication Manager ensures that the same number of seconds elapses between the runs. For example, if you set the Start Time to January 19, 2022 11.06 AM and the Interval to 1 day, Replication Manager runs the replication policy for the first time at the specified time in the time zone in which you created the replication policy. It then runs the policy exactly one day (24 hours or 86400 seconds) later.

The map task dynamically throttles its bandwidth consumption during a copy operation so that the net used bandwidth aligns with the specified value, however, the exact net usage might fluctuate.

• Static – Distributes file replication tasks among the mappers in advance to achieve a uniform distribution based on file sizes.
• Dynamic – Distributes the file replication tasks to mappers in small sets. As a mapper completes its current set, it dynamically acquires and processes the next set of unallocated tasks.

• Skip Checksum Checks – Skips checksum checks on the copied files. If selected, checksums are not validated. Checksums are checked by default.

Checksums are used for the following purposes:

• To skip replication of files that have already been copied. If Skip Checksum Checks is selected, the replication job skips copying a file if the file lengths and modification times are identical between the source and target clusters. Otherwise, the job copies the file from the source to the target.
• To redundantly verify the integrity of data. However, checksums are not required to guarantee accurate transfers between clusters. HDFS data transfers are protected by checksums during transfer and storage hardware also uses checksums to ensure that data is stored accurately. These two mechanisms work together to validate the integrity of the copied data.

• Skip Listing Checksum Checks – Skips checksum verification when comparing two files to see if they are identical. If selected, the file size and last modified time are used to compare the files. Skipping the check improves performance during the mapper phase.
• Abort on Error– Aborts the job upon encountering an error. If selected, files copied up to that point remain on the target, but no additional files are copied. By default, this is not selected.
• Abort on Snapshot Diff Failures – Aborts the replication if a snapshot diff fails. By default, if a snapshot diff fails, the replication policy uses a complete copy to replicate data. If selected, the policy aborts the replication entirely instead.

Select one or more of the following attributes you want to keep from the source file system:

• Block Size
• Replication Count
• Permissions – Replication preserves ACLs, when both the source and destination clusters support ACLs. Otherwise, ACLs are not replicated.
• Extended Attributes – Replication preserves extended attributes when both the source and destination clusters support extended attributes. This option is displayed only when both source and destination clusters support extended attributes.

If an option is not selected, the replication job uses the settings of the destination file system. By default, the source system settings are preserved.

If you select any Preserve options when replicating to S3 or ADLS, the values are saved in metadata files on S3 or ADLS. When you replicate from S3 or ADLS to HDFS, you can select which of these saved options to preserve.

• Keep Deleted Files – Retains the destination files even when they no longer exist at the source. This is the default option.
• Delete to Trash – Moves files to the trash folder, if the HDFS trash is enabled. This option is not supported when replicating to S3 or ADLS.
• Delete Permanently – Deletes files permanently, which uses the least amount of space. Use with caution.

You can configure alerts to be delivered by email or sent as SNMP traps. If alerts are enabled for events, you can search for and view the alerts on the Events tab, even if email notifications are not configured.