Developing a dataflow for Stateless NiFi
Learn about the recommended process of building a dataflow that you can deploy with the Stateless NiFi Sink or Source connectors. This process involves building and designing a parameterized dataflow within a Process Group and then downloading the dataflow as a flow definition.
The general steps for building a dataflow are identical for both source and sink connector flows. For ease of understanding a dataflow example for a simple MQTT Source and MQTT Sink connector is provided without going into details (what processors to use, what parameters and properties to set, what relationships to define and so on).
- Ensure that you reviewed Dataflow development best practices for Stateless NiFi.
- You have access to a running instance of NiFi.
- Design and parameterize your dataflow.For example, you can build a dataflow that you deploy as a source connector. The following is an example dataflow for a simple MQTT source connector. It consists of a ConsumeMQTT Processor and an output port (representing a Kafka topic).
Alternatively, you can also build a dataflow to deploy as a sink connector. The following is an example dataflow for a simple MQTT sink connector. It consists of an input port (representing a Kafka topic), a PublishMQTT Processor, and two output ports.
In the case of this example, a Failure output port is required. This is done so that in case the destination is not available, the session can be rolled back and the Kafka message can be declined. The Failure port is specified in the configuration of the connector when deploying it through SMM.
When designing and building your dataflow, ensure that you update your Parameter Context and reference the parameters in your components. For example:
For more information on building dataflows, parameters, and referencing parameters see the Flow Management library.
