Add the Telemetry Tutorial NiFi Flow as a function in Cloudera Data Flow
Before you can run the Apache NiFi flow in serverless mode on AWS Lambda, you must
register it in Cloudera Data Flow service and download the Lambda DF function
handler libraries.
1. Upload the Telemetry Tutorial NiFi Flow to the Cloudera Data Flow
Catalog🔗
After logging into the Cloudera on cloud console with the
credentials that you used to register for the trial, select the Cloudera Data Flow service on the Cloudera on cloud homepage.
Click Catalog in the left navigation pane to display the
Flow Catalog.
Click Import Flow Definition.
Provide a name and a description and upload the data flow you downloaded from your
local NiFi instance.
Click Import.
Copy the CRN # for version 1 of the flow you uploaded.
Make sure to select the version’s CRN in the orange box below. You will need it when
configuring the function in AWS Lambda.
2. Download the Lambda Cloudera Data Flow Function binaries zip and
upload to S3🔗
To be able to run the NiFi flow in AWS Lambda, you need the Cloudera Data Flow function handler libraries.
Click Functions in the left navigation pane and download the
Cloudera Data Flow Function binaries for AWS Lambda.
AWS Lambda will use these binaries to run the NiFi flow.
Upload this binary to an S3 bucket that you will later reference when creating the
function in AWS Lambda.
The S3 bucket needs to be in the same region as where the Lambda is being
created/deployed.
Copy the S3 URI for later use.
3. Create a Cloudera on cloud Service Account🔗
You need to create a Cloudera on cloud service account
for the AWS Lambda function to be able to retrieve the Cloudera Data Flow
function from the Catalog where you uploaded it earlier.
Go to the Cloudera on cloud homepage, select Management Console > User Management.
Under Actions, select Create Machine
User and provide a name for the service account.
Click Roles > Update Roles and select the DFFunctionMachineUser.
Click Update.
Select Access Keys > Generate Access Key.
Save the Access Key ID and Private Key, which will be used to configure the AWS
Lambda function.
