Controlling memory usage
When you enable WriteBufferManager for a Flink application, the block-cache size becomes the single most important option in limiting the memory size. You can set it 256 mb as a starting value, and later increase it based on your production experience.
state.backend.rocksdb.block.cache-size: 256mb
The block cache serves as the in-memory cache for both reads and writes for the RocksDB
instances. Flink creates an independent RocksDB instance for each stateful (keyed state)
operator and subtask, so the minimum RocksDB native memory requirement is
num_task_slots * num_stateful_ops * block_cache_size. The block size should
be chosen depending on the size of the state and number of keys, but 256 MB is a good starting
value.
If the read performance is insufficient, the block-size can be increased to 512 MB, 1 GB or even 2 GB on machines with large memory sizes.
