Cloudera Docs

Scaling recommendations and limitations

Here are some of the recommended vales that you should consider while deciding the size of the tablet servers, masters, and other storage resources to get optimum performance.

Kudu can seamlessly run across a wide array of environments and workloads with minimal expertise and configuration at the following scale:

  • Recommended maximum number of masters is 3.

  • Recommended maximum number of tablet servers is 100.

  • Recommended maximum amount of stored data, post-replication and post-compression, per tablet server is 8 TiB.

  • Recommended number of tablets per tablet server is 1000 (post-replication) with 2000 being the maximum number of tablets allowed per tablet server.

  • Maximum number of tablets per table for each tablet server is 60, post-replication (assuming the default replication factor of 3), at table-creation time.

  • Recommended maximum amount of data per tablet is 50 GiB. Going beyond this can cause issues such a reduced performance, compaction issues, and slow tablet startup times.

    The recommended target size for tablets is under 10 GiB.

Staying within these limits will provide the most predictable and straightforward Kudu experience. However, experienced users who run on modern hardware, use the latest versions of Kudu, test and tune Kudu for their use case, and work closely with the community, can achieve much higher scales comfortably. Following are some anecdotal values that have been seen in real world production clusters:
  • Number of master servers: 3
  • More than 300 tablet servers
  • 10+ TiB of stored data per tablet server, post-replication and post-compression
  • More than 4000 tablets per tablet server, post-replication
  • 50 GiB of stored data per tablet. Going beyond this can cause issues such a reduced performance, compaction issues, and slower tablet startup time.