Scaling Namespaces and Optimizing Data Storage
Also available as:
PDF
loading table of contents...

Federation terminology

A block pool is a set of data blocks that belongs to a single namespace. A namespace and its block pool together form a namespace volume.

Block pool

A block pool is a set of blocks that belong to a single namespace. DataNodes store blocks for all the block pools in the cluster. Each block pool is managed independently. This allows a namespace to generate Block IDs for new blocks without the need for coordination with the other namespaces. A NameNode failure does not prevent the DataNode from serving other NameNodes in the cluster.

Namespace volume

A namespace and its block pool together are called namespace volume. A namespace volume is a self-contained unit of management. When a NameNode or a namespace is deleted, the corresponding block pool at the DataNodes is deleted.

Cluster ID

A Cluster ID is an identifier for all the nodes in the cluster. When a NameNode is formatted, this identifier is either provided or automatically generated. The same Cluster ID is used for formatting the other NameNodes into the cluster.