Ozone containers

Containers are the fundamental replication unit of Ozone and are managed by the Storage Container Manager (SCM) service. Containers are big binary units (5 Gb by default) that can contain multiple blocks.

Blocks are local information that are not managed by SCM. Therefore, even if billions of small files are created in the system (which means billions of blocks are created), only the status of the containers will be reported by the DataNodes and the containers will be replicated.

Whenever Ozone Manager requests a new block allocation from the SCM, SCM identifies the suitable container and generates a block id which contains a ContainerId and a LocalId. The client connects to the DataNode which stores the container, and the DataNode manages the separated block based on the LocalId.

Open versus closed containers

Open Closed
mutable immutable
replicated with RAFT (Ratis) Replicated with async container copy

Write: Allowed only on Raft Leader

Read: On all replicas

Deletes: Not allowed

Write: No writes are allowed

Read: On all replicas

Deletes: allowed