Cloudera Docs

Behavioral changes

Learn about behavioral changes in Cloudera Flow Management 4.0.0.

NiFi 2.0 introduces a lot of significant changes and enhancements, including some breaking changes. It is important to familiarize yourself with the following points before migrating your existing flows.

Java 21

Java 21 is the minimum Java version required with NiFi 2.0, so make sure you have Java 21 installed on your NiFi nodes before upgrading.

Templates and XML flow definitions

The concept of templates in NiFi has been deprecated, and the XML templates are stored in memory in NiFi as well as in the persisted flow definition.

Additionally, flow.xml.gz no longer exists, only flow.json.gz can be used in NiFi clusters for defining flows in the canvas.

If you have templates, export those templates as JSON definitions or version the templates into a NiFi Registry instance. The best practice is to use a NiFi Registry in combination with NiFi when it comes to version control and share / reuse flow definitions.

Custom components / NARs

Although not certain, it is very likely that a custom NAR designed for NiFi 1 will not be successfully loaded into NiFi 2. If your NiFi setup includes custom components or NARs, it is a requirement to update your dependencies to align with NiFi 2. This entails making the necessary adjustments and rebuilding your NARs using Java 21.

Variables replaced by parameters

Variables and the variable registry have been removed from NiFi. Only Parameter contexts and parameters are available for use going forward. In future Cloudera Flow Management releases, tooling will be provided to help with the conversion of variables to parameters. In the meantime, this conversion should be done manually when migrating flows to NiFi 2. Any variables left will simply be ignored when loading the flow definition.

Event driven thread pool no longer exists

The event driven scheduling strategy was an option available on some processors. This was an experimental feature in NiFi and did not prove to bring any significant performance improvements. The event driven thread pool has been removed, leaving only the time driven thread pool available. Any components previously configured using the event driven scheduling strategy should be switched to the time driven scheduling strategy.

Removed languages in scripted components

In NiFi 2.0, support for certain languages in scripted components has been removed. The affected languages are: ECMAScript, Lua, Ruby, and Python. It is recommended to switch to Groovy or to leverage the new Python API feature for developing processors.

Removed components and replacement options

The following list contains the list of the components that have been removed between clusters based on NiFi 1.26 and clusters based on NiFi 2.0, along with the recommended alternatives where available.

  • Processors
    • Base64EncodeContent => EncodeContent
    • CompareFuzzyHash => no replacement
    • ConsumeEWS => no replacement
    • ConsumeKafka_1_0 => ConsumeKafka_2_6
    • ConsumeKafka_2_0 => ConsumeKafka_2_6
    • ConsumeKafkaRecord_1_0 => ConsumeKafkaRecord_2_6
    • ConsumeKafkaRecord_2_0 => ConsumeKafkaRecord_2_6
    • ConvertAvroSchema => ConvertRecord
    • ConvertAvroToORC => no replacement
    • ConvertCSVToAvro => ConvertRecord
    • ConvertExcelToCSVProcessor => ConvertRecord with ExcelReader
    • ConvertJSONToAvro => ConvertRecord
    • CryptographicHashAttribute => UpdateAttribute
    • DeleteAzureBlobStorage => DeleteAzureBlobStorage_v12
    • DeleteRethinkDB => no replacement
    • EncryptContent => EncryptContentAge or EncryptContentPGP
    • ExecuteInfluxDBQuery => use Influx Data NARs for NiFi
    • ExtractCCDAAttributes => no replacement
    • FetchAzureBlobStorage => FetchAzureBlobStorage_v12
    • FetchElasticsearchHttp => GetElasticsearch
    • FuzzyHashContent => no replacement
    • GetAzureQueueStorage => GetAzureQueueStorage_v12
    • GetHTMLElement => no replacement
    • GetHTTP => InvokeHTTP
    • GetIgniteCache => no replacement
    • GetJMSQueue => ConsumeJMS
    • GetJMSTopic => ConsumeJMS
    • GetRethinkDB => no replacement
    • GetTCP => no replacement
    • GetTwitter => ConsumeTwitter
    • HashAttribute => CryptographicHashAttribute
    • HashContent => CryptographicHashContent
    • InferAvroSchema => ExtractRecordSchema
    • ListAzureBlobStorage => ListAzureBlobStorage_v12
    • ModifyHTMLElement => no replacement
    • PostHTTP => InvokeHTTP
    • PostSlack => PublishSlack
    • PublishKafka_1_0 => PublishKafka_2_6
    • PublishKafka_2_0 => PublishKafka_2_6
    • PublishKafkaRecord_1_0 => PublishKafkaRecord_2_6
    • PublishKafkaRecord_2_0 => PublishKafkaRecord_2_6
    • PutAzureBlobStorage => PutAzureBlobStorage_v12
    • PutAzureQueueStorage => PutAzureQueueStorage_v12
    • PutBigQueryBatch => PutBigQuery
    • PutBigQueryStreaming => PutBigQuery
    • PutElasticsearchHttp => PutElasticsearchJson
    • PutElasticsearchHttpRecord => PutElasticsearchRecord
    • PutHiveQL => PutClouderaHiveQL
    • PutHiveStreaming => PutClouderaHiveStreaming
    • PutHTMLElement => no replacement
    • PutIgniteCache => no replacement
    • PutInfluxDB => use Influx Data NARs for NiFi
    • PutJMS => PublishJMS
    • PutRethinkDB => no replacement
    • PutRiemann => no replacement
    • PutSlack => PublishSlack
    • QueryElasticsearchHttp => PaginatedJsonQueryElasticsearch
    • ScrollElasticsearchHttp => SearchElasticsearch
    • SelectHiveQL => SelectClouderaHiveQL
    • SpringContextProcessor => no replacement
    • StoreInKiteDataset => no replacement
    • UpdateHiveTable => UpdateClouderaHiveTable
  • Controller services
    • ActionHandlerLookup => no replacement
    • AlertHandler => no replacement
    • AzureStorageCredentialsControllerService => AzureStorageCredentialsControllerService_v12
    • AzureStorageCredentialsControllerServiceLookup => AzureStorageCredentialsControllerServiceLookup_v12
    • AzureStorageEmulatorCredentialsControllerService => no replacement
    • EasyRulesEngineProvider => no replacement
    • EasyRulesEngineService => no replacement
    • ExpressionHandler => no replacement
    • GraphiteMetricReporterService => no replacement
    • GremlinClientService => no replacement
    • HBase_1_1_2_ClientMapCacheService => HBase_2_ClientMapCacheService
    • HBase_1_1_2_ClientService => HBase_2_ClientService
    • HBase_1_1_2_ListLookupService => no replacement
    • HBase_1_1_2_RecordLookupService => HBase_2_RecordLookupService
    • HiveConnectionPool => ClouderaHiveConnectionPool
    • HortonworksSchemaRegistry => ClouderaSchemaRegistry
    • KafkaRecordSink_1_0 => KafkaRecordSink_2_6
    • KafkaRecordSink_2_0 => KafkaRecordSink_2_6
    • LogHandler => no replacement
    • OAuth2TokenProviderImpl => StandardOauth2AccessTokenProvider
    • OpenCypherClientService => no replacement
    • RecordSinkHandler => no replacement
    • ScriptedActionHandler => no replacement
    • ScriptedRulesEngine => no replacement
  • Reporting tasks
    • AmbariReportingTask => no replacement
    • MetricsEventReportingTask => no replacement
    • MetricsReportingTask => no replacement
  • Components with new coordinates
    • InvokeGRPC => moved into nifi-cdf-grpc-nar
    • ListenGRPC => moved into nifi-cdf-grpc-nar
    • KerberosKeytabUserService => moved into nifi-kerberos-user-service-nar
    • KerberosPasswordUserService => moved into nifi-kerberos-user-service-nar
    • KerberosTicketCacheUserService => moved into nifi-kerberos-user-service-nar
    Tooling will be provided in upcoming Cloudera Flow Management releases to automatically handle these changes. Currently, two options are available:
    • Manually edit the flow.json.gz file to update the coordinates of the impacted components.
    • Make the changes after the flow is imported in NiFi 2.0 by replacing the ghost components with the new implementations for each instance of the components listed above.
  • Pulsar components
    All Pulsar components have been temporarily removed. They will be reintroduced in an upcoming release. In the meantime, you can download the NARs from a public Maven repository and deploy them as custom NARs.