Configuring your target processor
In a dataflow that is pulling data from a Snowflake database, configure the ExecuteSQLRecord processor to handle data pooling from remote tables. To do this, launch the processor configuration window and provide the configurations appropriate for your use case.
Configure ExecuteSQLRecord
.
The ExecuteSQLRecord
processor handles data pooling from the remote tables.
To
load balance the execution of the SQL queries across the NiFi nodes, you should update the
configuration of the relationship between the ListDatabaseTables
processor
and the ExecuteSQLRecord
processor and define a round robin load balancing
strategy. This way the flow files generated on the primary node by the
ListDatabaseTables
processors will be shuffled across the nodes of the
NiFi cluster to distribute the workload.
After you have configured your target processor, proceed to confirming your dataflow success.