What's new in this release?

Previously, access to NiFi component documentation was limited to using the Apache NiFi website or the NiFi UI in a running instance. However, with this release, Cloudera makes the documentation of all CFM components (general and Cloudera-specific) available as part of Cloudera Flow Management documentation.

It is important to note that Cloudera Flow Management incorporates over 100 components that are not available in Apache NiFi. To access this comprehensive documentation, see the Cloudera Apache NiFi Components Reference Guide.

Cloudera introduces a new Registry Client implementation, designed to facilitate access to the exclusive Cloudera Flow Library. This library contains predefined flow configurations to expedite the deployment of use cases. While the Flow Library will see ongoing enhancements, it already offers access to all ReadyFlows included in the Cloudera DataFlow for Public Cloud data service.

For additional information and instructions on how to configure the registry client, see Using Flow Library Registry Client .

NiFi's integration with Cloudera Manager has been improved by introducing a set of new features:

• Automatic Heap Dump Capture: In the event of an Out Of Memory error, NiFi now automatically captures a heap dump.

• Schema Registry Integration: You can include Schema Registry as a dependency during NiFi installation. Additionally, you can configure a Cloudera Schema Registry controller service to seamlessly connect with the Schema Registry instance of CDP.

• Thread Pool Adjustment: The default size of NiFi's Thread Pool has been updated to align with the available CPU cores on the NiFi host.

• Enhanced Health Checks: This release incorporates additional health checks related to NiFi's memory utilization and configuration.

New processors:

• ExtractRecordSchema

  You can use this processor to define a Record Reader and have the schema of your data inferred and added as a FlowFile attribute. If schema inference is required, Cloudera recommends to use this processor over the unsupported and deprecated InferAvroSchema processor.

• GenerateRecord
• GetAwsPollyJobStatus
• GetAwsTextractJobStatus
• GetAwsTranscribeJobStatus
• GetAwsTranslateJobStatus
• GetAzureQueueStorage_v12
• ListenNetFlow

  This processor supports receiving NetFlow Export Packets over UDP for the NetFlow versions 1, 5, and 9.

• ModifyCompression
• PutAzureQueueStorage_v12
• PutIcebergCDC [Technical Preview]
  You can use this processor to apply CDC events into Iceberg formatted tables.
• QueryIoTDBRecord
• RemoveRecordField
• StartAwsPollyJob
• StartAwsTextractJob
• StartAwsTranscribeJob
• StartAwsTranslateJob
• ValidateJson
• VerifyContentMAC

New controller services:

• ADLSCredentialsControllerServiceLookup
• AmazonGlueSchemaRegistry
• AzureServiceBusJMSConnectionFactoryProvider
• AzureStorageCredentialsControllerServiceLookup_v12
• ClouderaSchemaRegistry

  You can use this controller service as a replacement of the HortonworksSchemaRegistry controller service, as it has been deprecated and marked for removal in anticipation of NiFi 2.0 release in the Apache NiFi project.

• CMLLookupService [Technical Preview]

  You can use this controller service in combination with the LookupRecord processor to enrich data by calling Cloudera Machine Learning API to do ML scoring on streams of data.

• EBCDICRecordReader [Technical Preview]

  You can use this reader with Cobol copybook files to read Mainframe data with EBCDIC encoding and convert the data into another structured format such as JSON, Avro, and so on.

• ExcelReader
• PostgreSQLConnectionPool
• RedshiftConnectionPool
• StandardFileResourceService

New parameter provider:

• CyberArkConjurParameterProvider

  You can use this parameter provider to source the values of your parameters from a CyberArk instance.