Build Your First Data Flow

Apache NiFi offers a large number of components that help you to create data flows for any type of protocols or data sources. Generally, a data flow has a listener component at the start of the flow, which gets the data from a source system. On the other end of the flow, there is usually a transmitter component that writes the data to a target system, and there can be components in between, which process the data.

NiFi components

Apache NiFi components are the building blocks of data flows. The NiFi Component toolbar contains several data flow components that you can use when creating a new flow.

Create a data flow

This section walks you through the process of creating NiFi data flows from scratch using basic flow options.

For information about building advanced data flows (using parameters, custom properties, controller services, or reporting tasks), see the Apache NiFi User Guide.

  1. Add data flow components.

    To create a flow, drag the component icons to the canvas from the component toolbar in the top-left of the screen. You can add processors and other components as you need. For more information, see Adding Components to the Canvas in the Apache NiFi User Guide.

  2. Configure the processors.
    1. Right-click the processor on the canvas and choose Configure in the context menu. Alternatively, double-click on the processor.
      A configuration dialog with the following tabs is displayed: Settings, Scheduling, Properties, and Comments.
    2. Configure the processor according to the behavior you desire in your data flow.
    3. When you have finished configuring the options you need, click Apply to save the changes or cancel all changes by clicking Cancel.
  3. Connect data flow components.

    You can connect the data flow components by hovering over a component and dragging the connection icon to another component. You can then choose which relationships you want to include for this connection.

  4. Start your data flow.

    When all components are connected, the flow is still not working. To start the processors, click each one individually and then right-click and choose the Start menu item.

    Alternatively, you can select all processors and then click the Start icon on the Operate palette.

    After you start the data flow, the icon in the top-left corner of the processors changes from a stopped icon to a running icon. You can stop the processors by using the Stop icon on the Operate palette or the Stop context menu item.

By following the above steps, you can choose any processor and other NiFi components to create a flow suitable for your purpose.