Cloudera Docs

Building your first data flow

You can use Apache NiFi to create data flows for any protocols or data sources. Start by constructing your flow with a listener component, retrieving data from a source system. and conclude it with a transmitter component that writes the data to a target system. NiFi provides a range of components for processing data efficiently that you can add in between your listener and transmitter.

NiFi component toolbar

The NiFi Component toolbar, located in the top-left corner, contains diverse components that you can use when constructing data flows. These components act as the building blocks, enabling you to design data flows tailored to your specific needs.

Creating a data flow

This section walks you through the process of creating NiFi data flows from scratch using basic flow options.

For information about building advanced data flows (using parameters, custom properties, controller services, or reporting tasks), see the Apache NiFi User Guide.

  1. Add data flow components.
    To add a processor to your flow, drag processor icons from the component toolbar to the canvas, located in the center of the screen. This step initiates the creation of your data flow.

    For more information, see Adding Components to the Canvas in the Apache NiFi User Guide.

  2. Configure the processors.
    1. Right-click or double-click a processor on the canvas and select Configure in the context menu.
      A configuration dialog with the following tabs is displayed: Settings, Scheduling, Properties, and Comments.
    2. Configure the processor according to the behavior you desire in your data flow.
    3. When you have finished configuring the options you need, click Apply to save the changes or cancel all changes by clicking Cancel.
  3. Connect data flow components.
    Hover over a component, drag the connection icon to another component, and select the relationships you want to include for this connection.
  4. Start your data flow.
    Even when components are connected, the flow is inactive. To activate processors, individually click each one, then right-click and choose Start from the menu.

    Alternatively, select all processors and click the Start icon on the Operate palette.

    After starting the data flow, the processor icons in the top-left corner change from a stopped to a running state. You can stop the processors by using the Stop icon on the Operate palette or the Stop context menu item.

By following these steps, leverage NiFi components to construct a dynamic data flow suitable for your specific use case.