Cloudera Docs

Ozone replication policies

Apache Ozone is a scalable, distributed, and high performance object store optimized for big data workloads and can handle billions of objects of varying sizes. Ozone storage is co-located on HDFS. You can create Ozone replication policies in CDP Private Cloud Base Replication Manager to replicate data in Ozone buckets between CDP Private Cloud Base 7.1.8 clusters or higher using Cloudera Manager 7.7.1 or higher.

Cloudera supports the following types of Ozone storage:
  • Object store buckets (OBS), which are storage buckets where all the keys are written into a flat namespace and can be accessed using S3 interface provided by Ozone.
  • File System Optimization (FSO), which are Hadoop-compatible file system buckets where the rename and delete operations on the directories are atomic. These buckets can be accessed using Filesystem APIs and S3 interfaces.
  • Legacy buckets, which are Ozone buckets created prior to CDP Private Cloud Base 7.1.8 and use the Ozone File System (ofs) protocol or scheme.

You can use Ozone replication policies to replicate or migrate the required Ozone data to another cluster to run load-intensive workloads, back up data, or for backup-restore use cases.

Ozone replication policies support data replication between:
  • FSO buckets in source and target clusters using ofs protocol.
  • legacy buckets in source and target clusters using ofs protocol.
  • OBS buckets in source and target clusters that support S3A filesystem using the S3A scheme or replication protocol.

How Ozone replication works

Ozone snapshots are enabled for all the buckets and volumes. If the incremental replication feature is enabled on the source and target clusters, to replicate Ozone data you can choose one of the following methods during the Ozone replication policy creation process:

Full file listing
By default, the Ozone replication policies use the full file listing method which takes a longer time to replicate data. In this method, the first Ozone replication policy job run is a bootstrap job; that is, all the data in the chosen buckets are replicated. During subsequent replication policy runs, Replication Manager performs the following high-level steps:
  1. Lists all the files.
  2. Performs a checksum and metadata check on them to identify the relevant files to copy. This step depends on the advanced options you choose during the replication creation process. During this identification process, some unchanged files are skipped if they do not meet the criteria set by the chosen advanced options.
  3. Copies the identified files from the source cluster to the target cluster.
Incremental only
In this method, the first replication policy job run is a bootstrap job, and subsequent job runs are incremental jobs.
To perform the incremental job, Replication Manager leverages Ozone snapshots and the snapshot-diff capability to generate a diff report. The diff report contains the changed or new data from the source cluster. The subsequent replication policy replicates data based on the diff report.
By default, the ozone.replication.incremental.allow_safe_to_merge_target_side_changesOzone service configuration parameter is enabled to ensure that the metadata replication of Ozone storage-backed Hive external tables is successful. In case, the metadata-only replication creates a partition on the destination (this is normal behavior for a Hive external table. A partition-level DDL statement might result in Ozone directory keys being created/deleted/renamed.), this in turn creates an Ozone directory key. In such scenarios, the parameter ensures that the Ozone incremental replication does not fall back to full-file listing, and therefore remains incremental. This is accomplished by ensuring the ‘create entries’ in the target-side snapshot-diff are accepted during the Ozone incremental replication process.
Incremental with fallback to full file listing
In this method, the first replication policy job run is a bootstrap job, and subsequent job runs are incremental jobs. However, if the snapshot-diff fails during a replication policy job run, the next job run is a full file listing run. After the full file listing run succeeds, the subsequent runs are incremental runs. This method takes a longer time to replicate data if the replication policy job falls back to the full file listing method.