Resource configuration

For a Flink application, setting the number of parallel instances is an essential part of resource management.

In a Flink application, the different tasks are split into several parallel instances for execution. The number of parallel instances for a task is called parallelism. Parallelism can be defined at the operator, client, execution environment and system level. Cloudera recommends setting parallelism to a lower value at the initially and increasing it over time, if the job cannot keep up with the input rate.

Flink automatically identifies the needed resources for an application and requests them from the ResourceManager. New resources are requested in case of failure.