What's new in Flow Management with NiFi 2 [Technical Preview]

Learn about the new features of Flow Management using NiFi 2.0 in Cloudera DataFlow for Data Hub 7.3.1.

Flow Management Data Hub clusters are compatible with both NiFi 1 and NiFi 2. The following sections provide details about Flow Management Data Hub based on Apache NiFi 2.0, with information on the most important new features, improvements, and fixes included in this release.

NiFi 2 introduces numerous changes from NiFi 1, including several breaking changes. See Behavioral changes for more information about the differences. Additionally, expect further breaking changes in future releases, particularly with components being removed in favor of more efficient alternatives.

Rebase on NiFi 2.0.0 M2

This upgrade offers access to the newest NiFi features and enhancements on the 2.x branch.

Python processors

One of the key features introduced in Apache NiFi 2 is native support for Python processors. This capability allows you to create custom processors using Python, enabling seamless integration of Python scripts into your dataflows. With each milestone release of NiFi 2, Python integration continues to evolve, providing developers with enhanced functionality, greater flexibility, and more powerful tools for building robust dataflows.

The below list shows the Python processors that are available in Flow Management 7.3.1 clusters using NiFi 2.

Bedrock
ChunkData
ChunkDocument
EmbedData
InsertToMilvus
LexicalQueryMilvus
ParseDocument
PartitionCsv
PartitionDocx
PartitionHtml
PartitionPdf
PartitionText
PromptChatGPT
PutChroma
PutOpenSearchVector
PutPinecone
PutQdrant
QueryChroma
QueryOpenSearchVector
QueryPinecone
QueryQdrant
VectorQueryMilvus

For a comprehensive list of supported NiFi components in Flow Management 7.3.1 Data Hub clusters, see Supported NiFi extensions.

For more information about the latest updates in Flow Management Data Hub clusters using NiFi 1, see What's new in Flow Management with NiFi 1.