Configuring Ozone services
This section helps you to understand the purpose of the parameters you need to set and the disk type you require for services like DataNode, OM, Recon, and SCM.
DataNode
| Setting name | Default value | Purpose | Disktype | Mount point | Minimum size (GB) | Notes |
|---|---|---|---|---|---|---|
| ozone.metadata.dirs | /var/lib/hadoop-ozone/datanode/ozone-metadata | One or more directories used for storing Ozone metadata. OzoneManager, SCM, and Datanode will write the metadata to this path. | HDD | /var/app/lib/ | 100 | |
| dfs.container.ratis.datanode.storage.dir | /var/lib/hadoop-ozone/datanode/ratis/data | One or more directories used for storing Datanode Ratis metadata. | NVMe (Preferred) | /var/app/lib/hadoop-ozone/datanode/ratis | 1024 | RAID 1 NVMe or SSDNVMe/SSD is not required but strongly recommended. |
| hdds.datanode.dir | None | List of directories where user data is stored on each DataNode. | HDD | /data/{1..N} | Size and Number varies |
The mount points are numeric values for each data drive 1 - n These disks must not be shared with HDFS or another storage system. If not configured this falls back to dfs.datanode.data.dir. However it is strongly recommended to configure this explicitly. |
Ozone Manager
| Setting name | Default value | Purpose | Disktype | Mount point | Minimum size (GB) | Notes |
|---|---|---|---|---|---|---|
| ozone.metadata.dirs | /var/lib/hadoop-ozone/om/ozone-metadata | One or more directories used for storing Ozone metadata. OzoneManager, SCM, and Datanode will write the metadata to this path. | NVMe (required) | /var/app/lib/hadoop-ozone/om/ | 4096 | RAID 1 NVMe required |
| ozone.om.ratis.storage.dir | /var/lib/hadoop-ozone/om/ratis | This directory stores Ozone Manager's Ratis metadata like logs. | NVMe (required) | /var/app/lib/hadoop-ozone/om/ | ||
| ozone.om.db.dirs | /var/lib/hadoop-ozone/om/data | Directory where the Ozone Manager stores its metadata. | NVMe (required) | /var/app/lib/hadoop-ozone/om/ | ||
| ozone.om.snapshot.diff.db.dir | Directory for Ozone snapshot diff scratch space. | HDD or SSD | /data/1 | 256 (One HDD with 256GB free space will be sufficient to store snapshot diffs.) |
This is not available in Cloudera Manager. You must configure this through the safety valve. By default, if no value is specified here then the same locations as specified in ozone.metadata.dirs will be used. |
Recon
| Setting name | Default value | Purpose | Disktype | Mount point | Minimum size (GB) | Notes |
|---|---|---|---|---|---|---|
| ozone.metadata.dirs | /var/lib/hadoop-ozone/recon/ozone-metadata | One or more directories used for storing Ozone metadata. OzoneManager, SCM, and Datanode will write the metadata to this path. | HDD | /var/app/lib/ | 4096 | |
| ozone.recon.db.dir | /var/lib/hadoop-ozone/recon/data | Directory where the Recon Server stores its metadata. | NVMe (required) | /var/app/lib/hadoop-ozone/recon | These 3 settings can share the same NVMe dedicated to Recon | |
| ozone.recon.om.db.dir | /var/lib/hadoop-ozone/recon/om/data | Where recon keeps OM snapshot DB. | NVMe (required) | /var/app/lib/hadoop-ozone/recon | ||
| ozone.recon.scm.db.dirs | /var/lib/hadoop-ozone/recon/scm/data | Directory where the Recon Server stores StorageContainerManager's metadata | NVMe (required) | /var/app/lib/hadoop-ozone/recon |
Storage Container Manager
| Setting name | Default value | Purpose | Disktype | Mount point | Minimum size (GB) | Notes |
|---|---|---|---|---|---|---|
| ozone.metadata.dirs | /var/lib/hadoop-ozone/scm/ozone-metadata | One or more directories used for storing Ozone metadata. OzoneManager, SCM, and Datanode will write the metadata to this path. | NVMe or SSD | /var/app/lib/hadoop-ozone/scm | 4096 |
These 3 settings can share the same NVMe dedicated to the SCM. RAID 1 NVMe or SSD required. |
| ozone.scm.db.dirs | /var/lib/hadoop-ozone/scm/data | Directory where the Storage Container Manager stores its metadata | NVMe or SSD | /var/app/lib/hadoop-ozone/scm | ||
| ozone.scm.ha.ratis.storage.dir | /var/lib/hadoop-ozone/scm/ratis | Storage directory used by SCM to write Ratis logs. | NVMe or SSD | /var/app/lib/hadoop-ozone/scm |
