Cloudera Docs

Build your dataflow

Set up the elements of your NiFi dataflow that allows you to move data into Snowflake using Apache NiFi. This involves opening NiFi in CDP Public Cloud, adding processors to your NiFi canvas, and connecting the processors.

When you are building a data flow to move data into Snowflake using Apache NiFi, you can consider using the following processors to build your dataflow:

  • GenerateFlowFile
  • PutDatabaseRecord

You have reviewed and met the prerequisites.

  1. Launch the NiFi from your CDP Public Cloud or CDP Private Cloud Base cluster.
  2. Add the NiFi Processors to your canvas.
    1. Select the Processor icon from the Cloudera Flow Management Actions pane, and drag a processor to the canvas.
    2. Use the Add Processor filter box to search for the processor you want to add, and then click Add.
    3. Add each of the processors you want to use for your data flow.
  3. Connect the two processors to create a dataflow.
    1. Click the Connection icon in the first processor, and drag it to the second processor.
    2. A Create Connection dialog displays. It has Details and Settings tabs. You can configure the connection's name, FlowFile expiration time period, thresholds for back pressure, load balance strategy, and prioritization.
    3. Click Add to close the dialog box and add the connection to your flow. Optionally, you can add success and failure funnels to your data flow, which help you see where flow files are routed when your data flow is running.

Once you have finished building the dataflow, move on to the following steps:

  • Create Controller Services for your dataflow.
  • Configure your source Processor.
  • Configure your target Processor.
  • Confirm your dataflow success