Back Pressure
NiFi provides two configuration elements for Back Pressure. These thresholds
indicate how much data should be allowed to exist in the queue before the component
that is the source of the Connection is no longer scheduled to run. This allows the
system to avoid being overrun with data. The first option provided is the "Back
pressure object threshold." This is the number of FlowFiles that can be in the
queue before back pressure is applied. The second configuration option is the
"Back pressure data size threshold." This specifies the maximum amount of
data (in size) that should be queued up before applying back pressure. This value is
configured by entering a number followed by a data size (B for
bytes, KB for kilobytes, MB for megabytes,
GB for gigabytes, or TB for terabytes).
|
By default each new connection added will have a default Back Pressure Object Threshold of 10,000 objects and Back Pressure Data Size Threshold of 1 GB. |
When back pressure is enabled, small progress bars appear on the connection label, so the DFM can see it at-a-glance when looking at a flow on the canvas. The progress bars change color based on the queue percentage: Green (0-60%), Yellow (61-85%) and Red (86-100%).
Hovering your mouse over a bar displays the exact percentage.
When the queue is completely full, the Connection is highlighted in red.
