Cloudera Docs

What's new in Flow Management

Learn about the new Flow Management features in Cloudera DataFlow for Data Hub 7.2.17.

Flow Management in CDP Public Cloud 7.2.17 is based on Apache NiFi 1.21.0 and includes significant improvements and fixes. Here are the most important new features and improvements:

Schema Registry
For new clusters, a Schema Registry instance is provisioned on the management node next to Cloudera Manager and NiFi Registry. This allows you to leverage the Schema Registry capabilities without the need for a Streams Messaging Data Hub cluster.
Cloudera Schema Registry Controller Service
As the Hortonworks Schema Registry Controller Service will be removed in NiFi 2.0, a new Cloudera Schema Registry Controller Service is now available and you are invited to switch to the new controller service to prepare for the NiFi 2.0 release. For new clusters, Cloudera provisions default instances of the controller service that are pre-configured with the Schema Registry instance deployed in the cluster.
Java version
Starting with CDP 7.2.17, Java 8 and 11 are installed on the nodes by default. While Java 8 remains the default, it is possible to switch to another Java version for existing clusters, or after an upgrade. For more information, see Changing the Java version of Flow Management Data Hub clusters.

When you create a new cluster through the UI, Java 8 is used as the default JDK. The ability to deploy a new Flow Management cluster with Java 11 is available only using the CLI. For more information, see Creating your cluster.

New components:
  • ConvertProtobuf to convert Protobuf data into JSON
  • Decrypt and Encode Content capabilities
  • Modify Compression processor
  • GCP Vision components for image processing
  • AWS Polly, AWS Textract, AWS Transcribe, AWS Translate
  • Asana source processor
  • Jira source processor
  • JSLT processor for easier transformations of JSON data
  • gRPC components with custom schema handling
  • Box, Dropbox, Google Drive sink processors
  • Support for Apache IoTDB
  • Salesforce sink processor
  • Redis sink processor (Technical Preview)
  • DeltaLake sink processor (Technical Preview)
  • Trigger Hive Metastore Event processor to update the HMS after pushing data into the object store
  • Validate JSON
  • Verify Content MAC
  • Generate Record processor – to better simulate data feeds
  • PostgreSQL controller service – to connect to PostgreSQL databases without the need to deal with the JDBC driver