Cloudera Docs

Build your first data flow

You can use Apache NiFi to create data flows for any protocols or data sources. Start your flow with a listener component, which gets the data from a source system. On the other end of the flow, add a transmitter component that writes the data to a target system. Additionally, you can add other components in between that process the data.

NiFi components

Apache NiFi components are the building blocks of data flows. The NiFi Component toolbar contains several data flow components that you can use when creating a new flow.

Create a data flow

This section walks you through the process of creating NiFi data flows from scratch using basic flow options.

For information about building advanced data flows (using parameters, custom properties, controller services, or reporting tasks), see the Apache NiFi User Guide.

  1. Add data flow components.

    You can add processors and other components to build your data flow as you need. To add a processor to your flow, drag the processor icon to the canvas from the component toolbar in the top-left of the screen. For more information, see Adding Components to the Canvas in the Apache NiFi User Guide.

  2. Configure the processors.
    1. Right-click the processor on the canvas and choose Configure in the context menu. Alternatively, double-click on the processor.
      A configuration dialog with the following tabs is displayed: Settings, Scheduling, Properties, and Comments.
    2. Configure the processor according to the behavior you desire in your data flow.
    3. When you have finished configuring the options you need, click Apply to save the changes or cancel all changes by clicking Cancel.
  3. Connect data flow components.

    You can connect the data flow components by hovering over a component and dragging the connection icon to another component. You can then choose which relationships you want to include for this connection.

  4. Start your data flow.

    When all components are connected, the flow is still not working. To start the processors, click each one individually and then right-click and choose the Start menu item.

    Alternatively, you can select all processors and then click the Start icon on the Operate palette.

    After you start the data flow, the icon in the top-left corner of the processors changes from a stopped icon to a running icon. You can stop the processors by using the Stop icon on the Operate palette or the Stop context menu item.

By following the above steps, you can choose any processor and other NiFi components to create a flow suitable for your purpose.