Add the Quickstart NiFi Flow as a function in Cloudera DataFlow
Before you can run the NiFi flow in serverless mode on AWS Lambda, you must register it in Cloudrea DataFlow service and download the Lambda DF function handler libraries.
1. Upload the Quickstart NiFi Flow to the DataFlow Catalog
- After logging into the CDP Public Cloud console with the credentials that you used to register for the trial, select the DataFlow service on the CDP homepage.
- Click Catalog in the left navigation pane to display the Flow Catalog.
- Click Import Flow Definition.
Provide a name and a description and upload the data flow you downloaded from your
local NiFi instance.
- Click Import.
Copy the CRN # for version 1 of the flow you uploaded.
Make sure to select the version’s CRN in the orange box below. You will need it when configuring the function in AWS Lambda.
2. Download the Lambda DataFlow Function binaries zip and upload to S3
To be able to run the NiFi flow in AWS Lambda, you need the DataFlow function handler libraries.
Click Functions in the left navigation pane and download the
DataFlow Function binaries for AWS Lambda.
AWS Lambda will use these binaries to run the NiFi flow.
Upload this binary to an S3 bucket that you will later reference when creating the
function in AWS Lambda.
The S3 bucket needs to be in the same region as where the Lambda is being created/deployed.
Copy the S3 URI for later use.
3. Create a CDP Service Account
You need to create a CDP service account for the AWS Lambda function to be able to retrieve the DataFlow function from the Catalog where you uploaded it earlier.
- Go to the CDP homepage, select .
Under Actions, select Create Machine
User and provide a name for the service account.
and select the
- Click Update.
- Select .
Save the Access Key ID and Private Key, which will be used to configure the AWS