Cloudera Docs

Behavioral changes

Learn about behavioral changes in Cloudera Flow Management 2.1.7.2000.

Change in Impala integration for NiFi

Summary
Starting with Cloudera Flow Management 2.1.7, NiFi includes a new controller service, ImpalaConnectionPool, which provides improved integration with Impala. This new service replaces the previous use of DBCPConnectionPool for Impala connections.
Recommendation
  1. Stop using DBCPConnectionPool for Impala connections.
  2. Update your NiFi flows to use the new ImpalaConnectionPool controller service for all Impala connectivity.

If you are upgrading from an earlier version of Cloudera Flow Management where DBCPConnectionPool was used to connect to Impala, you may encounter the NoClassDefFoundError error message in your data flows. This error is caused by a race condition when loading the ImpalaJDBC driver.

If you encounter the NoClassDefFoundError after upgrading:
  1. Review your NiFi flows and replace any DBCPConnectionPool services used for Impala with ImpalaConnectionPool.
  2. Restart the affected NiFi processors after making this change.

Secure communication between NiFi and ZooKeeper configured by default

If both ZooKeeper and NiFi services are secured, NiFi communication with ZooKeeper will be automatically configured as secured (TLS) using a new port, 2182. If you enforce TCP communication through a firewall and explicitly allow certain ports, you need to open them for port 2182.

If you do not want to use secure communication between ZooKeeper and NiFi, follow these steps to configure unsecured communication on port 2181:

  1. Update the ZooKeeper connection string:

    1. In Cloudera Manager, navigate to NiFi > Configuration.
    2. Set nifi.zookeeper.connect.string by replacing ${ZK_QUORUM} with the unsecure ZK QUORUM string, which has port 2181.
    To find your ZooKeeper quorum string from a NiFi node, run the following command as root:
    NIFI_PROC=$(ls -td /var/run/cloudera-scm-agent/process/NIFI/ | head -1); grep "Connect String" $NIFI_PROC/state-management.xml | cut -d\> -f2 | cut -d\< -f1; unset NIFI_PROC
    This command will provide your connect string. For example: 
    host1:2181,host2:2181,host3:2181

  2. Add a safety valve for staging/state-management.xml in Cloudera Manager with the following property:

    • Name: xml.state-management.cluster-provider.zk-provider.property.Connect String
    • Value: <YOUR ZOOKEEPER CONNECT STRING>

  3. After upgrading to version 2.1.7, uncheck the nifi.zookeeper.client.secure option in Cloudera Manager.

ScriptedTransformRecord processor requires proper schema name attribute for record writer

NIFI-11523 introduced a fix that ensures the ScriptedTransformRecord processor uses the correct schema defined for the record writer. Previously, if the schema name attribute was set in the writer but not in the flow, it was ignored, defaulting to the reader schema. This behavior has been corrected, which may cause the processor to fail after upgrading if the schema name attribute is not set in the flow.

The failure is typically logged as:
org.apache.nifi.schema.access.SchemaNotFoundexception: ${schema.name} did not provide appropriate Schema Name

To prevent failures, ensure that the schema name attribute is properly configured in the flow or match it to the schema defined for the record reader for identical behavior.