determines the maximum amount of memory that a Kudu tablet server may use. The amount of memory
used by a tablet server scales with data size, write workload, and read concurrency.
The following table provides numbers that can be used to compute a rough estimate of memory usage:
|Memory required per TB of data on disk||1.5GB per 1TB data on disk||Amount of memory per unit of data on disk required for basic operation of the tablet server.|
|Hot Replicas' MemRowSets and DeltaMemStores||minimum 128MB per hot replica||Minimum amount of data to flush per MemRowSet flush. For most use cases, updates should be rare compared to inserts, so the DeltaMemStores should be very small.|
|Scans||256KB per column per core for read-heavy tables||Amount of memory used by scanners, and which will be constantly needed for tables which are constantly read.|
||Amount of memory reserved for use by the block cache.|
Using this information for the example load gives the following breakdown of memory usage:
|8TB data on disk||8TB * 1.5GB / 1TB = 12GB|
|200 hot replicas||200 * 128MB = 25.6GB|
|1 40-column, frequently-scanned table||40 * 40 * 256KB = 409.6MB|
|Expected memory usage||38.5GB|
|Recommended hard limit||52GB|
Using this as a rough estimate of Kudu’s memory usage, select a memory limit so that the expected memory usage of Kudu is around 50-75% of the hard limit.