What's new in Cloudera Flow Management
Learn about the new features and improvements in Cloudera Flow Management and see how they can benefit your workflows.
Cloudera Flow Management 2.1.7.1000
On October 2, 2024, Cloudera Flow Management 2.1.7.1000 (Service Pack 1) was released, introducing the following new features and key updates.
The new NiFi pre-upgrade check functionality helps you to proactively assess your NiFi environment before upgrading to NiFi 2, which will be included in an upcoming Cloudera Flow Management 4.x release. The pre-upgrade check compiles a detailed list of potential breaking changes that may occur during an upgrade, allowing you to address these issues proactively. While the script identifies potential problems, it is your responsibility to take necessary actions to mitigate these issues as a preparation for a future upgrade. For more information, see Pre-upgrade check for NiFi.
For detailed information about the issues resolved in Cloudera Flow Management 2.1.7.1000, see Fixed issues and Fixed CVEs.
Cloudera Flow Management 2.1.7.1000 is based on Apache NiFi 1.26.0 and it also incorporates a lot of Cloudera exclusive features and improvements. If you are currently using Cloudera Flow Management 2.1.7, upgrade to Cloudera Flow Management 2.1.7.1000 to take advantage of the latest version. The new download links are available in Download locations.
Cloudera Flow Management 2.1.7
Discover the new functionalities and improvements in Cloudera Flow Management 2.1.7. and learn how these new features can benefit you.
The Cloudera Flow Management 2.1.7 release is based on Apache NiFi 1.26.0 and it also incorporates a lot of Cloudera exclusive features and improvements.
Here is an overview of what is new in this release:
New processors
- CalculateParquetOffsets
- CalculateParquetRowGroupOffsets
-
These processors can be used in combination with ConvertRecord and Parquet Reader to significantly reduce the time required to convert very large Parquet files into another format.
- CaptureChangeDebeziumDB2
- CaptureChangeDebeziumMySQL
- CaptureChangeDebeziumOracle
- CaptureChangeDebeziumPostgreSQL
- CaptureChangeDebeziumSQLServer
-
Currently in Technical Preview, these processors leverage the Debezium project to ingest Change Data Capture (CDC) events from external databases.
- DecryptContentAge
- EncryptContentAge
-
These new-generation processors are designed for data encryption and decryption. For more information, see Modernizing Streaming Encryption with age in Apache NiFi.
- ListenOTLP
-
This processor enables NiFi to act as a destination for OpenTelemetry Protocol (OTLP) agents to receive OpenTelemetry data from external applications. For more information about this new processor, see Building OpenTelemetry Collection in Apache NiFi with Netty.
- PutClouderaHiveQL
- PutClouderaHiveStreaming
- PutClouderaORC
- SelectClouderaHiveQL
- UpdateClouderaHiveTable
- TriggerClouderaHiveMetaStoreEvent
-
These Cloudera exclusive components are designed to interact with Hive-based components in the Cloudera Data Platform. Since Hive components will no longer be part of Apache NiFi starting with NiFi 2.0, it is highly recommended to switch to these components as soon as possible to make the upgrade/migration to NiFi 2.0 easier.
- PutJiraIssue
-
This processor allows you to to create new issues in Jira using the Jira REST API.
- PutZendeskTicket
-
This processor allows you to create Zendesk tickets using the Zendesk API.
- ConsumeElasticsearch
-
This processor repeatedly runs a paginated query against a field using a Range query to consume new documents from an Elasticsearch index/query.
- FilterAttribute
-
Thi processor filters the attributes of a FlowFile by retaining specified attributes and removing the rest or by removing specified attributes and retaining the rest.
- PackageFlowFile
-
This processor packages FlowFile attributes and content into an output FlowFile that can be exported from NiFi and imported back into NiFi, preserving the original attributes and content.
- PublishSlack
-
This processor allows you to post a message to the specified Slack channel.
New controller services
- ActiveMQJMSConnectionFactoryProvider
-
This controller service allows you to interact with ActiveMQ without the need to deploy the JMS client on all of the NiFi nodes.
- ClouderaHiveConnectionPool
-
This controller service allows you to interact with Hive without the need to deploy the required dependencies on all of the NiFi nodes.
- DatabaseTableSchemaRegistry
-
This controller service enables you to retrieve the schema associated with a table from an external database. This allows you to validate the data going through NiFi against that schema before pushing the data into this table.
- ImpalaConnectionPool
-
This controller service allows you to interact with Impala without the need to deploy the required dependencies on all NiFi nodes.
- RabbitMQJMSConnectionFactoryProvider
-
This controller service allows you to interact with RabbitMQ without the need to deploy the JMS client on all NiFi nodes.
- ProtobufReader
-
This record reader allows you to read Protobuf data with the record based components.
- YamlTreeReader
-
This record reader allows you to read YAML data with the record based components.
