Cloudera Docs

What's New in This Release?

Cloudera Flow Management (CFM) 1.1.0 includes the Apache NiFi features and improvements.

Apache NiFi Updates

NiFi has been upgraded from 1.9.0 to 1.11.4.

New features and improvements in Apache NiFi include:
  • Parameters. The parameters are a new concept in NiFi that allows users to define Parameter Contexts attached to process groups so that any property (including sensitive properties and properties not supporting expression language) can be parameterized. This is particularly useful in CI/CD pipelines when deploying workflows across multiple environments. Parameters should be used as a replacement of the variables.
  • Predictive monitoring. NiFi has now an internal analytics framework which can be enabled to predict back pressure occurrence, given the configured settings for threshold on a queue. It uses recent observations from a queue (either number of objects or content size over time) and calculates predictions based on a model to anticipate backpressure.

    For more details on analytics properties, see Analytics Properties.

  • Rules engine (Technical Preview). It is now possible to define a Rules Engine controller service to support execution of a centralized set of rules (stored as files or provided within the service configuration) against a provided set of data called facts. Upon execution, the rules engine will determine what rules have been met and return a list of actions that should be executed based on the conditions defined within the rules. This is particularly useful for workflow monitoring.
  • Content and Flow File repositories encryption (Technical Preview). In addition to the Provenance repository it is now possible to encrypt both Content and Flow File repositories on disks to comply with strict security requirements. This feature is in technical preview.

    For more details on flow file repository properties, see Encrypted Write Ahead FlowFile Repository Properties.

    For more details on content repository properties, see Encrypted File System Content Repository Properties.

  • SQL reporting task. The QueryNiFiReportingTask allows users to execute SQL queries against tables containing information on Connection Status, Processor Status, Bulletins, Process Group Status, JVM Metrics, Provenance and Connection Status Predictions. In combination with Site to Site, it is particularly useful to define fine-grained monitoring capabilities on top of the running workflows.