Ozone containers

Containers are the fundamental replication unit of Ozone and are managed by the Storage Container Manager (SCM) service. Containers are big binary units (5 Gb by default) that can contain multiple blocks.

Blocks are local information that are not managed by SCM. Therefore, even if billions of small files are created in the system (which means billions of blocks are created), only the status of the containers will be reported by the DataNodes and the containers will be replicated.

Whenever Ozone Manager requests a new block allocation from the SCM, SCM identifies the suitable container and generates a block id which contains a ContainerId and a LocalId. The client connects to the DataNode which stores the container, and the DataNode manages the separated block based on the LocalId.

Open versus closed containers

Open	Closed
mutable	immutable
replicated with RAFT (Ratis)	Replicated with async container copy
Write: Allowed only on Raft Leader Read: On all replicas Deletes: Not allowed	Write: No writes are allowed Read: On all replicas Deletes: allowed