Testing your Lambda function
Once you have built and verified your dataflow and uploaded it to the Cloudera DataFlow catalog, you can create your Lambda function.
After creating the Lambda function and configuring all necessary Environment Variables, you should test that you have configured all of the settings correctly before publishing the Lambda.
For this, you can use the Test tab of AWS Lambda Function details screen. Lambda provides you with many options for sample data, so you can easily create an example JSON document to simulate the events that your Lambda function will handle.
Click the Test button to run the data flow. If there are problems, you can inspect the log messages to understand the problem. If the logs do not provide enough details, you may need to adjust the log levels to gain more debugging information. Ror more information, see Adjusting logs levels.
The amount of time the Lambda function takes to run depends heavily on the data flow and the number of extensions it needs. Because the Lambda function may have to download some extensions from Nexus and perform initialization, a cold start may take longer time. Even 20-30 seconds is not uncommon for a data flow with several extensions, while other data flows may complete in 10 seconds.
After you run the function successfully, it may be helpful to run several additional iterations to understand how the function will perform when a cold start is not necessary. Again, this depends heavily upon the configured data flow and other services that it may interact with. A simple data flow may complete in as a short time as 10 milliseconds while a data flow that must perform complex transformations and reach out to one or more web services or databases for enrichment may take several seconds to complete.
It is important to adjust Lambda's timeout configuration on the tab if the cold start takes longer than the amount of time allocated.
