Cloudera Docs

Known issues in Flow Management

Learn about the known issues in Flow Management clusters, the impact or changes to the functionality, and the workaround.

Learn about the known issues and limitations in Flow Management in this release:
NiFi UI becomes unavailable when a bulletin is generated at Flow Controller level
When a bulletin is generated at the Flow Controller level, such as for reporting tasks or a disconnected node, the NiFi UI becomes unavailable and gets stuck with a loading spinner. The UI remains inaccessible until the bulletin at the Flow Controller level is cleared (a bulletin persists for 5 minutes after being generated).

The issue is a result of NIFI-11433 and is fixed in NIFI-11560. The fix is not included in this release.

To address the issue until CDP 7.2.17.100 becomes available, upgrade your Data Hub cluster using the CDP CLI and utilize an image containing the fix. For instructions on how to use the upgrade-cluster CLI command, see https://cloudera.github.io/cdp-dev-docs/cli-docs/datahub/upgrade-cluster.html.

The image IDs are:
  • AWS: 11be770b-27d2-4818-b0a5-6ac79e77e971
  • Azure: eff93217-445b-48f6-8f36-01cdb3df01ce
  • GCP: 56200b06-b5d6-4eee-bba7-17bd5d4b9121
Incomplete Ranger policy for NiFi metrics in Cloudera Manager
To have Cloudera Manager properly reflect the NiFi metrics for the NiFi service, the Flow NiFi access policies in Ranger needs to be updated to include the "nifi" group.
KafkaRecordSink puts multiple records in one message
All the records are sent as a single Kafka message containing an array of records.

For more information, see NIFI-8326.

There is no workaround for this issue.
NiFi Atlas reporting task does not work after data lake upgrade from light to medium
After you upgrade your data lake from light to medium scale, the data lake machine hostname and IP address will change. As the Atlas reporting task uses Atlas and Kafka server hostnames, after the upgrade the wrong hostnames will prevent NiFi to report into Atlas.

Update the configuration of the ReportLineageToAtlas reporting task:

  1. Open the Global menu on the NiFi UI.
  2. Click Controller settings.
  3. Select the Reporting tasks tab in the dialog box.
  4. Stop the ReportLineageToAtlas reporting task and update the configuration:
    • Replace the hostname value in the Atlas Urls configuration with the new Atlas hostname.
    • Replace the hostnames value in the Kafka Bootstrap servers configuration with the new Kafka bootstrap server hostnames.
  5. Start the ReportLineageToAtlas reporting task.
InferAvroSchema may fail when inferring schema for JSON data
In Apache NiFi 1.17, the dependency on Apache Avro has been upgraded to 1.11.0. However, the InferAvroSchema processor depends on the hadoop-libraries NAR from which the Avro version comes from, causing a NoSuchMethodError exception. Having well defined schemas ensures consistent behavior, allows for proper schema versioning and prevents downstream systems to generate errors because of unexpected schema changes. Besides, schema inference may not always be 100% accurate and can be an expensive operation in terms of performances.
Use the ConvertRecord processor and have the Record Writer write the schema as a FlowFile attribute.