Depending on what kind of data flow you want to run, you may or may not want to output the data.
For example, a data flow may be designed to be called in response to an object being added to S3. This data flow may fetch the data from S3, parse the data, transform it, and then push the data into Apache Kafka. For such a data flow, it does not really make sense to output any data, as the data flow itself introduces a Kafka topic as the sink of the data.
You can design a different data flow to be called in response to a message being placed onto an SNS topic. The data flow may take that notification, filter out messages that do not match some criteria, and then send any messages that do match the criteria to an Output Port. That output may then be connected to another destination through an AWS Lambda destination.
Output ports can also be used when using the API Gateway trigger to output the data that should be sent back to the client that made the API call.
- No output port
If no Output Port is present in the data flow's root group, no output will be provided from the function invocation.
- One output port
If one Output Port is present, and its name is "failure", any data routed to that Output Port will cause the function invocation to fail. No output will be provided as the output of the function invocation.
- Two output ports, where one is failure
If two Output Ports are present, and one of them is named "failure", that Output Port will behave as the "failure" port described above. The other Output Port is considered a successful invocation. When the data flow is invoked, if a single FlowFile is routed to this Port, the contents of the FlowFile will be emitted as the output of the function. If more than one FlowFile is sent to the Output Port, the data flow will be considered a failure, as Lambda requires a single entity to be provided as its output. In this case a MergeContent or MergeRecord Processor may be used to assemble multiple FlowFiles into a single output FlowFile.
Any port with the name "failure" (ignoring case) will be considered a Failure Port. Additionally, if the FAILURE_PORTS environment variable is specified, any port whose name is in the comma-separated list will be considered a Failure Port.
- Two or more output ports with no failure port
If two or more Output Ports are present and none of them are failure ports, you must provide the OUTPUT_PORT environment variable. This environment variable must match the name of an Output Port at the data flow's root group (case is NOT ignored). Anything that is routed to the specified port is considered the output of the function invocation. Anything routed to any other port is considered a failure.