Output size constraints

When you design a data flow that will be deployed within AWS Lambda, you create a Process Group that contains a single Input Port (for example named Input) and two Output Ports (for example named success and failure). If data is routed to the failure port, the entire invocation is failed. If data is routed to the success port, the contents of the FlowFile become the output of the Lambda function. This allows you to set a destination for your AWS Lambda. For example, you can send the output to an SQS queue or another Lambda function.

Consider the following when designing your flow:
  • AWS Lambda limits the size of the output. While the maximum output size is configurable, the default is 6 MB. So, if the size of the FlowFile routed to success exceeds 6 MB, the result is a failure, even if you have no destination configured.

  • Take into account the size constraints on any downstream system that may be used as the Lambda destination, and do not send a large FlowFile as the output of the Process Group. Instead, you should have the last processor in the data flow auto-terminate the success relationship so that there is no output FlowFile. Or you can use a ReplaceText processor with the Replacement Strategy set to Always Replace. This allows you to set the contents to some predefined text, such as: { "action": "perform dataflow", "success": true, "fileSize": ${fileSize} }