Cloudera Docs

Ozone replication policies using Cloudera Manager APIs

Apache Ozone is a scalable, distributed, and high performance object store optimized for big data workloads and can handle billions of objects of varying sizes. Ozone storage is co-located on HDFS. You can use Cloudera Manager APIs to create Ozone replication policies to replicate data in Ozone buckets between CDP Private Cloud Base 7.1.8 clusters or higher using Cloudera Manager 7.7.1 or higher.

Cloudera supports the following types of Ozone storage:

  • Object store buckets (OBS) are storage buckets where all the keys are written into a flat namespace and can be accessed using S3 interface provided by Ozone.

  • File System Optimization (FSO) are Hadoop-compatible file system buckets where the rename and delete operations on the directories are atomic. These buckets can be accessed using Filesystem APIs and S3 interfaces.

  • Legacy buckets are Ozone buckets created prior to CDP Private Cloud Base 7.1.8 and uses the Ozone File System (ofs) protocol or scheme.

Ozone replication policies use Cloudera Manager APIs to replicate data. You can use these policies to replicate or migrate the required Ozone data to another cluster to run load-intensive workloads, back up data, or for backup-restore use cases.

Ozone replication policies supports the following functionalities:

  • Replicates data between FSO buckets in source and target clusters using ofs protocol. Supports incremental replication using file checksums.

  • Replicates data between legacy buckets in source and target clusters using ofs protocol. Supports incremental replication using file checksums.

  • Replicates data between OBS buckets in source and target clusters that support S3A filesystem using the S3A scheme or replication protocol.