Fixed Issues in Flow Management

Review the list of Flow Management issues that are resolved in Cloudera DataFlow for Data Hub 7.2.16.

CFM 2.2.6 is rebased against Apache NiFi 1.18.0. It includes all fixed issues of this Apache NiFi release as well as the below list:

NIFI-10660: When converting Avro GenericRecord to a java Map, check GenericRecord's schema to see if field exists before calling Record.get().
NIFI-10937 NIFI-10938 NIFI-10939: When restoring flow from XML, properly parse XML for registry clients, looking for 'id' tag instead of 'identifier' tag. Fixed typo in error message about Reporting Task for Registry Clients. Updated StandardFlowComparator so that we compare sub-process groups' contents only if the Flow Coordinates are different by more than their version
NIFI-NIFI-10918: When fetching a flow from a Flow Registry, if it references any 'internal versioned flows' instead of requiring that we have a client configured for the appropriate URL, attempt to fetch the flow from each client. We will start with the clients that do report that they can handle the URL but will try others as well. As soon as we successfully fetch the flow, we stop.
NIFI-NIFI-10914: Adjusting input check for loading nested versioned flows
NIFI-10901: PublishKafka headers not sent in ProducerRecord
NIFI-10874: Fixed issue with multiple levels of versioned flows
NIFI-10787: Cannot commit flows to NiFi Registry after updating our NiFi release to 1.18.0. NifiRegistryFlowRegistryClient defines the PropertyDescriptor PROPERTY_URL name as small case "url". The map bases on the name property of the PropertyDescriptor object. Here searching with uppercase value of "URL" causes the map lookup to fail and cause a NPE later on. Therefore, it is changed as "url"
NIFI-10817: Moved the calls in StandardProcessSession to 'resetState()' and 'acknowledgeRecords()' from the outer commit(boolean) to the inner commit(Checkpoint, boolean). By moving the call here, the logic of StandardProcessSession is unaffected. But the StatelessProcessSession that inherits from it now has the benefit of having the state cleaned up when calling super.commit(Checkpoint, boolean).
NIFI-10814: Display description for selected flow
NIFI-10389: Added mime.type to GetShopify and GetHubSpot
NIFI-10828: Adding new REST API endpoint for exposing versioned flow details
NIFI-10793: Comment is not populated when process group is created via API
NIFI-9822: ConsumeKafkaRecord allows writing out Kafka record key
NIFI-10762: Ranger Authorizer accepts multiple Ranger Admin Identity property values
NIFI-10801: Fix hive metastore based tests in Iceberg processors
NIFI-10737: Corrected ListenBeats buffer handling
NIFI-4798: Allow empty value for UpdateAttribute property
NIFI-10130: AzureGraphUserGroupProvider handles group with transitive members
NIFI-10228: Parse json flow using instanceIdentifier as the UUID
NIFI-10287: ExecuteScript - Allow python scripts to use external modules
NIFI-10637: Simplified Locale validation in ParseCEF
NIFI-10675: Corrected SSL support in Neo4j V4 Cypher Service
NIFI-10764: Fixed handling BIT STRING in nifi-asn1-services
NIFI-10767: When an empty line is encountered with Syslog Readers, just skip over the empty line instead of throwing an Exception
NIFI-10768: Filter empty messages in ListenSyslog
NIFI-10771: Add SINGLE_BATCH_PER_NODE to allowable values for flow file concurrency in ProcessGroupDTO
NIFI-10783: Switched to LinkedHashMap for CompareFuzzyHash
NIFI-10788: Ensure proposed service config is applied when component synchronizer adds a new service
NIFI-10702: Clear server list on connection error in SMB processors
NIFI-10680: Corrected Bouncy Castle references in optional profiles
NIFI-10614: Upgraded Bouncy Castle from 1.70 to 1.71
NIFI-10698: Add filtering options to summary table
NIFI-10709: Showed log in link on logout complete page
NIFI-10317: Taking care of NullPointerException if AMQP header value is null

Refactoring to remove repeated conversions to string from char

Correctly handle null values: null vs "null"

Adding test

Updated ConsumeAMQPTest to test for null and empty header values

Updated ConsumeAMQPTest to use hard coded string values when testing

Made convertMapToString() private
NIFI-9878: Added timeout handling for Cache Client handshaking
NIFI-10303: Route GetElasticsearch to failure if _id is blank after attribute evaluation
NIFI-10594: Render empty lines in Labels
NIFI-10631: Changed HashMap to LinkedHashMap in TestHBase_2_ClientService
NIFI-10635: Fix handling of enums in PutDatabaseRecord
NIFI-10641: Resolving typo on ParameterProvider warning message
NIFI-10643: Paging secrets in AwsSecretsManagerParameterProvider
NIFI-10668: Persist proxy password in remote process group
NIFI-10674: Block evaluateELString from reading Parameters
NIFI-10682: When determining effective property values, ensure that we always use the up-to-date version of Property Descriptors
NIFI-10687: Set Elasticsearch document _id to null if ID attribute evaluated to blank String for PutElasticsearchRecord or PutElasticsearchJson; use @timestamp default value if @timestamp record path evaluates to blank String in PutElasticsearchRecord
NIFI-10689: Fixed Parameter Provider Name encoding in context dialog
NIFI-10700: Only restart ports if their scheduledState is Running. - Adding unit tests. - Fixing unit test to properly verify Port isn't restarted.
NIFI-10693: Remove proxy configuration properties from PutBigQuery
NIFI-10673: When a component is added to a ProcessGroup, we had a method called ensureUniqueVersionControlId but the method only took into accounts in that Process Group. Updated methods to now consider components in any Process Group that is part of the same Versioned Flow. Also added system test to verify the problem and the fix. Added Toolkit clients and additional methods as necessary in order to implement system tests
NIFI-10644: Add Message Demarcator-style processing in PublishMQTT
NIFI-10628: Added Default Schedule Time and Run Duration to processors
NIFI-10650: Fix demarcator is appended to the end of the FlowFile's content by ConsumeMQTT
NIFI-10567: Corrected Sensitive Dynamic Property handling for flow.xml (#6524)
NIFI-10648: Upgraded Apache Commons Text to 1.10.0
NIFI-10442: Create PutIceberg processor
NIFI-10602: Fix GetHubSpot and GetShopify state handling on NiFi restart
NIFI-10577: Fix Sensitive Checkbox rendering for Fetch Parameters
NIFI-10586: Prioritized ssh-rsa algorithm in SFTP Processors
NIFI-10597: Change 'App Config File' property to external resource in Box processors
NIFI-10273: Supported file entries larger than 8.5GB for TAR in MergeContent
NIFI-10543: Support broker failover in MQTT processors
CFM-2713: Resource Fetcher over HTTPS with Basic Authentication
NIFI-9955: Upgraded ZooKeeper 3.4.14 to 3.5.9 in Kite bundle
NIFI-9952: Adjusted Kite bundle to centrally managed jackson dependencies
NIFI-9933: Upgraded Apache Ant to 1.10.12
NIFI-10096: Corrected nested inherited Parameter Context loading