Ambari Managed HDF Upgrade
Also available as:
PDF

Known Issues

Summary of known issues for this release.

Table 1. Known Issues

Hortonworks Bug ID

Apache JIRA

Apache Component

Summary
N/A N/A NiFi

Description of the problem

JDK 8u242 and newer versions cause Site-to-Site failure

When you use JDK 8u242 or a newer version, the RAW transport protocol in Site-to-Site fails with the following exception:
2018-12-19 16:19:26,811 ERROR [Site-to-Site Listener] org.apache.nifi.NiFi
 java.nio.channels.IllegalBlockingModeException: null
 at java.base/sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:121)
 at org.apache.nifi.remote.SocketRemoteSiteListener$1.run(SocketRemoteSiteListener.java:125)
 at java.base/java.lang.Thread.run(Thread.java:834)
Workarounds: Choose one of the following options before upgrading to JDK 8u242 or a newer JDK version:
  • Upgrade to HDF 3.51 or a newer version. Cloudera recommends this option.
  • Change all the Remote Process Groups and Site-to-Site based reporting tasks to use HTTP transport protocol instead of RAW transport protocol.

If neither of these options work for you, contact Cloudera Support for other options.

DOCS-8292 JDK-8245417 and JDK-8256818 NiFi

Description of the problem:

JDK 8u271, JDK 8u281, and JDK 8u291 may cause socket leak issue in NiFi due to JDK-8245417 and JDK-8256818. Pay attention to the build version of your JDK because some later builds are fixed as described in JDK-8256818.

Workaround:

Consider using a more recent version of the JDK like 8u282, or builds of the JDK where the issue is fixed.

DOCS-6263 KAFKA-7697 Kafka

Description of the problem:

You may experience a performance deterioration after upgrade to HDF 3.4.x with Kafka version 2.1.0.

Workaround:

To work around this issue, upgrade to HDF 3.5.x or see your support representative for an available hotfix.

BUG-120051 NIFI-6285 NiFi Description of the problem:

When you configure a NiFi Connection by changing the Load Balance Strategy, NiFi automatically redistributes the data as needed. If this occurs while data is swapped out from the NiFi process, however, this can result in the data not being swapped back in. When this occurs, the data remains in the queue and not be processed.

Associated error message:

You may see an error message in the logs, similar to: Encountered unknown Swap File ./flowfile_repository/swap/1557762343026-9e6b1ea4-016a-1000-ffff-ffff9a287a94-0ef48376-b919-4aae-9b72-0e7acfd1bfb9.rebalance.swap; will ignore this Swap File. This file should be cleaned up manually.

To avoid this issue:

To avoid this, ensure that you stop the Processor upstream of any connection and allow the queue to empty before changing the Load Balance Strategy.

To work around this issue:

To work around this issue if you do encounter it, shut down NiFi and rename the files in the FlowFile Repository's "swap" directory such that it matches the pattern <timestamp>-<connection identifier>-<uuid>.local.swap instead of <timestamp>-<connection identifier>-<uuid>.<other-partition-name>.swap and then restart NiFi. Be sure only to rename those Swap Files whose name contains the relevant connection's identifier.

BUG-118594 N/A N/A

Description of the problem:

SAM app with storm-druid sink throws exception.

BUG-108848

N/A Kafka

Description of the problem:

Kafka 1.1.1 producer cannot produce messages with snappy compression on IBM Power.

N/A Kafka-7697 Kafka

Description of the problem: You may experience a performance deterioration after upgrade to HDF 3.4.x with Kafka version 2.1.0.

Workaround: To work around this issue, upgrade to HDF 3.5.x or see your support representative for an available hotfix.

N/A N/A Kafka

HDF 3.4.1.1 contains Apache Kafka version 2.1.0, while HDP 3.1.0 contains Kafka 2.0.0. If you are using HDF with any HDP components, be aware of this interoperability issue. Review the Notable changes in Apache Kafka 2.1.0 to understand any interoperability issues that may arise.

Table 2. Technical Service Bulletin
Technical Service Bulletin Apache JIRA Apache component Summary
TSB-497 N/A Solr CVE-2021-27905: Apache Solr SSRF vulnerability with the Replication handler

The Apache Solr ReplicationHandler (normally registered at "/replication" under a Solr core) has a "masterUrl" (also "leaderUrl" alias) parameter. The “masterUrl” parameter is used to designate another ReplicationHandler on another Solr core to replicate index data into the local core. To help prevent the CVE-2021-27905 SSRF vulnerability, Solr should check these parameters against a similar configuration used for the "shards" parameter.

For the latest update on this issue, see the corresponding Knowledge article: TSB 2021-497: CVE-2021-27905: Apache Solr SSRF vulnerability with the Replication handler