Cloudera Docs

DataFlow Deployments

DataFlow Deployments provides a cloud-native runtime to run your Apache NiFi flows through auto- scaling Kubernetes clusters. It also provides a centralized monitoring and alerting capability that results in improved Software Development Life Cycle (SDLC) for developers.

Quickstart

Get started with Cloudera DataFlow quickly by walking through a few simple steps. Ensure that administrators have access to CDP, enable DataFlow for an environment, give users access to CDF, and then add to the Catalog and deploy the Hello World ReadyFlow.

Cloud Infrastructure

Learn about the infrastructure requirements of CDP and Cloudera DataFlow with supported cloud providers.

Environments

You can enable the DataFlow service for any supported environment you have registered with CDP. Once DataFlow has been enabled for an environment, you can start deploying flow definitions to it.

ReadyFlows

ReadyFlows are predefined, out-of-the-box data flows that can be immediately deployed by providing a set of required parameters.

Flow Development Using NiFi

Before you can deploy a flow definition in Cloudera DataFlow (CDF), you need to develop your data flow logic in a development environment using Apache NiFi. Download the NiFi flow as a flow definition, import it to Cloudera DataFlow and finally, deploy the flow. The flow definition acts as a configuration logic for your flow deployments. It enables you to deploy your data flow without the need to maintain cluster infrastructure. You can deploy the same flow to multiple environments in Cloudera DataFlow.

Catalog / Managing Flow Definitions

Your flow definitions are stored in the Catalog. This is where you manage the CDF flow definition lifecycle from import through versioning to deletion.

Flow Deployments

A flow deployment represents a NiFi cluster running on Kubernetes and executing a specific flow definition.

Inbound Connections

Inbound Connection Endpoints allow you to stream data from an external source application to a DataFlow deployment.

Security

Learn about DataFlow security features and tasks, including authorization.

Troubleshooting

Learn about identifying and addressing errors throughout the lifecycle of a DataFlow deployment.