Configuring Ozone services

This section helps you to understand the purpose of the parameters you need to set and the disk type you require for services like DataNode, OM, Recon, and SCM.

DataNode

Setting name Default value Purpose Disktype Mount point Minimum size (GB) Notes
ozone.metadata.dirs /var/lib/hadoop-ozone/datanode/ozone-metadata Determines where the datanode security certificates will be stored on the local filesystem. HDD /var/app/lib/ 100
dfs.container.ratis.datanode.storage.dir /var/lib/hadoop-ozone/datanode/ratis/data One or more directories used for storing Datanode Ratis metadata. NVMe (Preferred) /var/app/lib/hadoop-ozone/datanode/ratis 1024 RAID 1 NVMe or SSDNVMe/SSD is not required but strongly recommended.
hdds.datanode.dir None List of directories where user data is stored on each DataNode. HDD /data/{1..N} Size and Number varies

The mount points are numeric values for each data drive 1 - n

These disks must not be shared with HDFS or another storage system.

If not configured this falls back to dfs.datanode.data.dir. However it is strongly recommended to configure this explicitly.

Ozone Manager

Setting name Default value Purpose Disktype Mount point Minimum size (GB) Notes
ozone.metadata.dirs /var/lib/hadoop-ozone/om/ozone-metadata Determines where the Ozone Manager security certificates will be stored on the local filesystem. NVMe (required) /var/app/lib/hadoop-ozone/om/ 4096 RAID 1 NVMe required
ozone.om.ratis.storage.dir /var/lib/hadoop-ozone/om/ratis This directory stores Ozone Manager's Ratis metadata like logs. NVMe (required) /var/app/lib/hadoop-ozone/om/
ozone.om.db.dirs /var/lib/hadoop-ozone/om/data Directory where the Ozone Manager stores its metadata. NVMe (required) /var/app/lib/hadoop-ozone/om/
ozone.om.snapshot.diff.db.dir Directory for Ozone snapshot diff scratch space. HDD or SSD /data/1 256 (One HDD with 256GB free space will be sufficient to store snapshot diffs.)

This is not available in Cloudera Manager. You must configure this through the safety valve.

By default, if no value is specified here then the same locations as specified in ozone.metadata.dirs will be used.

Recon

Setting name Default value Purpose Disktype Mount point Minimum size (GB) Notes
ozone.metadata.dirs /var/lib/hadoop-ozone/recon/ozone-metadata Determines where the Recon security certificates will be stored on the local filesystem. HDD /var/app/lib/ 4096
ozone.recon.db.dir /var/lib/hadoop-ozone/recon/data Directory where the Recon Server stores its metadata. NVMe (required) /var/app/lib/hadoop-ozone/recon These 3 settings can share the same NVMe dedicated to Recon
ozone.recon.om.db.dir /var/lib/hadoop-ozone/recon/om/data Where recon keeps OM snapshot DB. NVMe (required) /var/app/lib/hadoop-ozone/recon
ozone.recon.scm.db.dirs /var/lib/hadoop-ozone/recon/scm/data Directory where the Recon Server stores StorageContainerManager's metadata NVMe (required) /var/app/lib/hadoop-ozone/recon

Storage Container Manager

Setting name Default value Purpose Disktype Mount point Minimum size (GB) Notes
ozone.metadata.dirs /var/lib/hadoop-ozone/scm/ozone-metadata Determines where the SCM security certificates will be stored on the local filesystem. NVMe or SSD /var/app/lib/hadoop-ozone/scm 4096

These 3 settings can share the same NVMe dedicated to the SCM.

RAID 1 NVMe or SSD required.

ozone.scm.db.dirs /var/lib/hadoop-ozone/scm/data Directory where the Storage Container Manager stores its metadata NVMe or SSD /var/app/lib/hadoop-ozone/scm
ozone.scm.ha.ratis.storage.dir /var/lib/hadoop-ozone/scm/ratis Storage directory used by SCM to write Ratis logs. NVMe or SSD /var/app/lib/hadoop-ozone/scm