Cloudera Docs

Creating Controller Services for your data flow

You can add controller services that can provide shared services to be used by the processors in your data flow. Create them after you build the NiFi data flow and before you configure the processors, so that they are available when you configure your NiFi processors.

For this data flow, create two controller services.

  • AvroWriter
  • DBCPConnectionPool

See the Related Information section below, for links to complete details about these Apache NiFi controller services.

You must have built your data flow.

  1. To add a controller service to your flow, right-click on the canvas and select Configure from the pop-up menu. This displays the Controller Services Configuration window.
  2. Select the Controller Services tab.
  3. Click the + button to display the Add Controller Service dialog.
  4. Select the required controller service and click Add.
  5. Click the Configure icon in the right-hand column and configure the necessary options.
  6. Click Apply to save the changes.
  7. Click the Enable button (flash) in the far-right column of the Controller Services tab to enable the controller service.

You require the following controller services for this example of a NiFi data flow moving data out of Snowflake.

AvroRecordSetWriter

You use this controller service as the writer, when writing the data as Flow Files in NiFi. Avro is a good option in most cases but you may want to choose another writer depending on your requirements.

In general, you may accept the default configurations.

DBCPConnectionPool

This controller service allows NiFi to interact with the Snowflake database through the JDBC interface.

Property Example value for a data flow pulling data from Snowflake
Name SnowflakeJDBCConnectionPool
Database Connection URL 
jdbc:snowflake://[***account_name***].snowflakecomputing.com/?[***connection_params***]

Where:

  • account_name is similar to cb56215.europe-west2.gcp
  • connection_parameters is similar to db=DEMO_DB
Database Driver Class Name net.snowflake.client.jdbc.SnowflakeDriver

After you have finished configuring your controller services:

  • Configure your source processor
  • Configure your target processor
  • Confirm your data flow is running