Testing your Azure Function App
Once you have built and verified your dataflow and uploaded it to the Cloudera DataFlow Catalog, you can create your Azure Function App.
After deploying the function, you should test if all settings have been properly configured.
-
Click the Functions blade of your Function App.
-
Select the Function you want to test.
-
Click the Code + Test blade on the left, and the Test/Run button at the top.
This brings up a side panel to input sample data and have the data flow run. If there are problems, you can inspect the log messages to understand the issue. If the logs do not provide enough details, you may need to adjust the log levels to gain more debugging information, and view the complete Log Stream. For more information, see Monitoring and logs.
As a workaround, you can test your Function without generating an actual Event Grid event as follows:
-
Click the Get function URL button at the top of the Code + Test view, and click the Copy button next to the URL.
-
Create a file, for example, named
event-grid.json
with the contents of the Event Grid schema JSON you want to test. -
Paste the URL in the place of the URL in the following curl command:
curl https://<app-function-name>.azurewebsites.net/runtime/webhooks/EventGrid?functionName=StatelessNiFiEventGridTriggerFunction&code=<code> \ -H 'Content-Type: application/json'\ -H 'aeg-event-type: Notification' --data-binary @event-grid.json
For more details on submitting this request, see Azure Event Grid trigger for Azure Functions.
The amount of time the cloud function takes to run depends heavily on the data flow and how many extensions it needs. Because the Cloud function may have to download some extensions from Nexus and perform initialization, a cold start may take many seconds. Even 20-30 seconds is not uncommon for a data flow with several extensions, while other data flows may complete in 10 seconds. After you run the function successfully using the Test button, it may be helpful to run several additional iterations in order to understand how the function will perform when a cold start is not necessary. Again, this depends heavily upon the configured data flow and other services that it may interact with. A simple data flow may complete in as little as 5 milliseconds for HTTP triggers or 100 milliseconds for Event Grid triggers, while a data flow that must perform complex transformations and reach out to one or more web services or databases for enrichment may take several seconds to complete.
It is important to adjust the function's timeout configuration if the cold start takes longer than the amount of time allocated.