Cloudera Docs » 3.1.1 » Release Notes
Release Notes
Also available as:
PDF

​Chapter 1. Hortonworks DataFlow 3.1.1 Release Notes

This document provides you with the latest information about the HDF 3.1.1 release and its product documentation.

​Component Support

HDF 3.1.1 includes the following components:

  • Apache Ambari 2.6.1

  • Apache Kafka 1.0.0

  • Apache NiFi 1.5.0

  • NiFi Registry 0.1.0

  • Apache Ranger 0.7.0

  • Apache Storm 1.1.1

  • Apache ZooKeeper 3.4.6

  • Apache MiNiFi Java Agent 0.4.0

  • Apache MiNiFi C++ 0.4.0

  • Hortonworks Schema Registry 0.5.0

  • Hortonworks Streaming Analytics Manager 0.6.0

​Component Availability in HDF

Previous HDF releases shipped with the following components versions.

 NiFiStormKafkaZooKeeperAmbariRangerMiNiFi Java AgentMiNiFi C++Streaming Analytics ManagerSchema RegistryNiFi Registry
HDF 3.1.11.5.01.1.11.0.03.4.62.6.10.7.00.4.00.4.00.6.0.5.00.1.0
HDF 3.1.01.5.01.1.11.0.03.4.62.6.10.7.00.4.00.4.00.6.00.5.00.1.0
HDF 3.0.3 (IBM Power System only)1.2.01.1.00.10.2.13.4.62.6.00.7.00.2.0TP0.5.00.3.0N/A
HDF 3.0.21.2.01.1.0`0.10.2.13.4.62.6.00.7.00.2.0TP0.5.00.3.0N/A
HDF 3.0.11.2.01.1.00.10.2.1 3.4.62.5.10.7.00.2.0TP0.5.00.3.0N/A
HDF 3.0.01.2.01.1.00.10.2.1 3.4.62.5.10.7.00.2.0TP0.5.00.3.0N/A
HDF 2.1.41.1.01.0.21.10.13.4.62.4.2.00.6.20.1.0TPN/AN/AN/A
HDF 2.1.21.1.01.0.20.10.13.4.62.4.2.00.6.20.1.0TPN/AN/AN/A
HDF 2.1.11.1.01.0.20.10.13.4.62.4.2.00.6.20.1.0TPN/AN/AN/A
HDF 2.1.01.1.01.0.20.10.13.4.62.4.2.00.6.20.1.0TPN/AN/AN/A
HDF 2.0.21.0.01.0.10.10.0.13.4.62.4.1.00.6.00.0.1TPN/AN/AN/A
HDF 2.0.11.0.01.0.10.10.0.13.4.62.4.1.00.6.00.0.1TPN/AN/AN/A
HDF 2.0.01.0.01.0.10.10.0.13.4.62.4.0.10.6.00.0.1TPN/AN/AN/A
HDF 1.2.10.6.10.10.00.9.0.13.4.6N/AN/AN/AN/AN/AN/AN/A
HDF 1.2.0.10.6.10.10.00.9.0.13.4.6N/AN/AN/AN/AN/AN/AN/A
HDF 1.2.00.6.00.10.00.9.0.13.4.6N/AN/AN/AN/AN/AN/AN/A
HDF 1.1.00.4.0N/AN/AN/AN/AN/AN/AN/AN/AN/AN/A
HDF 1.00.3.0N/AN/AN/AN/AN/AN/AN/AN/AN/AN/A

​What's New in HDF 3.1.1

HDF 3.1.1 is a maintenance release that includes bug fixes and certifies HDF for use on IBM Power Systems.

For information about the HDF 3.1.0 release, see the HDF 3.1.0 Release Notes.

​Unsupported Features

Some features exist within HDF 3.1.1, but Hortonworks does not currently support these capabilities.

​Technical Preview Features

The following features are available within HDF 3.1.1 but are not ready for production deployment. Hortonworks encourages you to explore these technical preview features in non-production environments and provide feedback on your experiences through the Hortonworks Community Forums.

​

Table 1.1. Technical Previews

ComponentFeature

MiNiFi

Android/iOS MiNiFi libraries for mobile integration


​Community Driven Features

The following features are developed and tested by the Hortonworks community but are not officially supported by Hortonworks. These features are excluded for a variety of reasons, including insufficient reliability or incomplete test case coverage, declaration of non-production readiness by the community at large, and feature deviation from Hortonworks best practices. Do not use these features in your production environments.

Community Driven Kafka features

  • Kafka Connect

  • Kafka Streams

Community Driven NiFi Tools and Services

  • Embedded ZooKeeper

  • Sensitive key migration toolkit

Community Driven NiFi Processors

  • AttributeRollingWindow

  • AWSCredentialsProviderControllerService

  • CompareFuzzyHash

  • ConsumeAzureEventHub

  • ConsumeEWS

  • ConsumeIMAP

  • ConsumeKafka_0_11

  • ConsumeKafkaRecord_0_11

  • ConsumePOP3

  • ConvertExcelToCSVProcessor

  • CountText

  • DebugFlow

  • DeleteDynamoDB

  • DeleteGCSObject

  • DeleteHDFS

  • DeleteMongo

  • DeleteRethinkDB

  • ExecuteFlumeSink

  • ExecuteFlumeSource

  • ExecuteSparkInteractive

  • ExtractCCDAAttributes

  • ExtractEmailAttachments

  • ExtractEmailHeaders

  • ExtractMediaMetadata

  • ExtractTNEFAttachments

  • FetchAzureBlobStorage

  • FetchGCSObject

  • FuzzyHashContent

  • GetDynamoDB

  • GetHDFSEvents

  • GetRethinkDB

  • GetSNMP

  • InvokeGRPC

  • ISPEnrichIP

  • InferAvroSchema

  • ListenBeats

  • ListenGRPC

  • ListenLumberjack

  • ListenSMTP

  • ListAzureBlobStorage

  • ListGCSBucket

  • ListS3

  • LogMessage

  • ModifyBytes

  • MoveHDFS

  • PublishKafka_0_11

  • PublishKafkaRecord_0_11

  • PutKudu

  • PutMongoRecord

  • PutRethinkDB

  • OrcFormatConversion

  • PutAzureBlobStorage

  • PutDynamoDB

  • PutGCSObject

  • PutIgniteCache

  • PutKinesisFirehose

  • PutKinesisStream

  • PutLambda

  • PutSlack

  • PutTCP

  • PutUDP

  • QueryDNS

  • SetSNMP

  • SpringContextProcessor

  • StoreInKiteDataset

[Note]Note

HDF 3.1.x does not support Hive 2. As a result, NiFi Processors working with Hive (PutHiveQL, PutHiveStreaming, SelectHiveQL) and the NiFi Controller Service HiveConnectionPool may not support Hive 2.

Community Driven NiFi Controller Services

  • AWSCredentialsProviderControllerService

  • ConfluentSchemaRegistry

  • GCPCredentialsControllerService

  • GraphiteMetricReporterService

  • IPLookupService

  • JettyWebSocketClient

  • JettyWebSocketServer

  • LivySessionController

  • MongoDBControllerService

  • MongoDBLookupService

  • PropertiesFileLookupService

  • RedisConnectionPoolService

  • RedisDistributedMapCacheClientService

  • SimpleCsvFileLookupService

  • SimpleKeyValueLookupService

  • XMLFileLookupService

Community Driven NiFi Reporting Tasks

  • DataDogReportingTask

  • MetricsReportingTask

  • SiteToSiteBulletinReportingTask

  • SiteToSiteStatusReportingTask

  • StandardGangliaReporter

​Unsupported Customizations

Hortonworks cannot guarantee that default NiFi processors are compatible with proprietary protocol implementations or proprietary interface extensions. For example, we support interfaces like JMS and JDBC that are built around standards, specifications, or open protocols. But we do not support customizations of those interfaces, or proprietary extensions built on top of those interfaces.

​Deprecated Technologies

This section points out any technology from previous releases that has been deprecated or removed from this release (operating systems, Java versions, databases, product features). Use this section as a guide for your implementation plans.

Deprecated

Technology that Hortonworks is removing in a future release. Deprecated items are supported until they are removed; deprecation gives you time to plan for removal.

Removed

Technology that Hortonworks has removed from production and is no longer supported.

​

Table 1.2. Deprecated Operating Systems

Operating System

Release DeprecatedRelease Removed
Ubuntu 12HDF 3.0.0HDF 3.0.0
Debian 6HDF 2.1.2HDF 3.0.0

​

Table 1.3. Deprecated NiFi Processors

Processor

Release Deprecated
ConvertCSVToAvroHDF 3.0.0
ConvertJSONToAvroHDF 3.0.0
GetKafka HDF 2.0.0
PutKafkaHDF 2.0.0
EvaluateRegularExpressionHDF 1.0.0

​

Table 1.4. Deprecated Kafka APIs

APIRelease DeprecatedUse Instead
kafka.producer.ProducerHDF 3.1.0org.apache.kafka.clients.producer.KafkaProducer
kafka.consumer.SimpleConsumerHDF 3.1.0org.apache.kafka.clients.consumer.KafkaConsumer
SecurityProtocol.PLAINTEXTSASLHDF 3.1.0SASL_PLAINTEXT

​HDF Repository Locations

Use the following table to identify the HDF 3.1.1 repository location for your operating system and operational objectives. HDF 3.1.1 supports the following operating systems:

​

Table 1.5. RHEL/Oracle Linux/CentOS 6 HDF repository & additional download locations

OS FormatDownload location
RHEL/Oracle Linux/CentOS 6 (64-bit):HDF Build number3.1.1.0-35
HDF Base URL http://public-repo-1.hortonworks.com/HDF/centos6/3.x/updates/3.1.1.0
HDF Repohttp://public-repo-1.hortonworks.com/HDF/centos6/3.x/updates/3.1.1.0/hdf.repo
RPM tarballhttp://public-repo-1.hortonworks.com/HDF/centos6/3.x/updates/3.1.1.0/HDF-3.1.1.0-centos6-rpm.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/centos6/3.x/updates/3.1.1.0/HDF-3.1.1.0-centos6-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/centos6/3.x/updates/3.1.1.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.1.0-35.tar.gz
MiNiFi C++http://public-repo-1.hortonworks.com/HDF/centos6/3.x/updates/3.1.1.0/tars/nifi-minifi-cpp/nifi-minifi-cpp-0.4.0-bin.tar.gz
HDP and Ambari Repositories
Ambarihttp://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.6.1.0/ambari.repo
HDP http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.4.0/hdp.repo
HDP-UTILS Base URLhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos6
OS Agnostic Downloads
NiFi only
NiFi Toolkit
Docker Hub

https://hub.docker.com/r/hortonworks/nifi/

Tags are latest and 3.1.1.0

NiFi Registry
MiNiFi Java Agent
MiNiFi Toolkit
iOS/Android Libraries

​

Table 1.6. RHEL/Oracle Linux/CentOS 7 HDF repository & additional download locations

OS FormatDownload location
RHEL/Oracle Linux/CentOS 7 (64-bit):HDF Build Number3.1.1.0-35
HDF Base URLhttp://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.1.1.0
HDF Repohttp://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.1.1.0/hdf.repo
RPM tarballhttp://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.1.1.0/HDF-3.1.1.0-centos7-rpm.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.1.1.0/HDF-3.1.1.0-centos7-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.1.1.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.1.0-35.tar.gz
MiNiFi C++ Agenthttp://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.1.1.0/tars/nifi-minifi-cpp/nifi-minifi-cpp-0.4.0-bin.tar.gz
HDP and Ambari Repositories
Ambari http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.1.0/ambari.repo
HDP http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.4.0/hdp.repo
HDP-UTILS Base URLhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7
OS Agnostic Downloads
NiFi only
NiFi Toolkit
Docker Hub

https://hub.docker.com/r/hortonworks/nifi/

Tags are latest and 3.1.1.0

NiFi Registry
MiNiFi Java Agent
MiNiFi Toolkit
iOS/Android Libraries

​

Table 1.7. SLES 11 SP3/SP4 HDF repository & additional download locations

OS FormatDownload location
SUSE Enterprise Linux 11 SP3, SP4 HDF Build Number3.1.1.0-35
HDF Base URLhttp://public-repo-1.hortonworks.com/HDF/suse11sp3/3.x/updates/3.1.1.0
Repohttp://public-repo-1.hortonworks.com/HDF/suse11sp3/3.x/updates/3.1.1.0/hdf.repo
RPM tarballhttp://public-repo-1.hortonworks.com/HDF/suse11sp3/3.x/updates/3.1.1.0/HDF-3.1.1.0-suse11sp3-rpm.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/suse11sp3/3.x/updates/3.1.1.0/HDF-3.1.1.0-suse11sp3-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/suse11sp3/3.x/updates/3.1.1.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.1.0-35.tar.gz
HDP and Ambari Repositories
Ambari http://public-repo-1.hortonworks.com/ambari/suse11/2.x/updates/2.6.1.0/ambari.repo
HDP http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/updates/2.6.4.0/hdp.repo
HDP-UTILS Base URLhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/suse11sp3
OS Agnostic Downloads
NiFi only
NiFi Toolkit
Docker Hub

https://hub.docker.com/r/hortonworks/nifi/

Tags are latest and 3.1.1.0

NiFi Registry
MiNiFi Java Agent
MiNiFi Toolkit
iOS/Android Libraries

​

Table 1.8. SLES 12 HDF repository & additional download locations

OS FormatDownload location
SUSE Linux Enterprise Server (SLES) v12 SP1HDF Build Number3.1.1.0-35
HDF Base URLhttp://public-repo-1.hortonworks.com/HDF/sles12/3.x/updates/3.1.1.0
Repohttp://public-repo-1.hortonworks.com/HDF/sles12/3.x/updates/3.1.1.0/hdf.repo
RPM tarballhttp://public-repo-1.hortonworks.com/HDF/sles12/3.x/updates/3.1.1.0/HDF-3.1.1.0-sles12-rpm.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/sles12/3.x/updates/3.1.1.0/HDF-3.1.1.0-sles12-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/sles12/3.x/updates/3.1.1.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.1.0-35.tar.gz
HDP and Ambari Repositories
Ambari http://public-repo-1.hortonworks.com/ambari/sles12/2.x/updates/2.6.1.0/ambari.repo
HDP http://public-repo-1.hortonworks.com/HDP/sles12/2.x/updates/2.6.4.0/hdp.repo
HDP-UTILS Base URLhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/sles12
OS Agnostic Downloads
NiFi only
NiFi Toolkit
Docker Hub

https://hub.docker.com/r/hortonworks/nifi/

Tags are latest and 3.1.1.0

NiFi Registry
MiNiFi Java Agent
MiNiFi Toolkit
iOS/Android Libraries

​

Table 1.9. Ubuntu 14 HDF repository & additional download locations

OS FormatDownload location
Ubuntu Trusty (14.04) (64-bit)HDF Build Number3.1.1.0-35
HDF Base URLhttp://public-repo-1.hortonworks.com/HDF/ubuntu14/3.x/updates/3.1.1.0
Repohttp://public-repo-1.hortonworks.com/HDF/ubuntu14/3.x/updates/3.1.1.0/hdf.list
Deb tarballhttp://public-repo-1.hortonworks.com/HDF/ubuntu14/3.x/updates/3.1.1.0/HDF-3.1.1.0-ubuntu14-deb.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/ubuntu14/3.x/updates/3.1.1.0/HDF-3.1.1.0-ubuntu14-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/ubuntu14/3.x/updates/3.1.1.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.1.0-35.tar.gz
HDP and Ambari Repositories
Ambarihttp://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.6.1.0/ambari.list
HDP http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.6.4.0/hdp.list
HDP-UTILS Base URLhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu14
OS Agnostic Downloads
NiFi only
NiFi Toolkit
Docker Hub

https://hub.docker.com/r/hortonworks/nifi/

Tags are latest and 3.1.1.0

NiFi Registry
MiNiFi Java Agent
MiNiFi Toolkit
iOS/Android Libraries

​

Table 1.10. Ubuntu 16 HDF repository & additional download locations

OS FormatDownload location
Ubuntu Trusty 16 (64-bit)HDF Build Number3.1.1.0-35
HDF Base URLhttp://public-repo-1.hortonworks.com/HDF/ubuntu16/3.x/updates/3.1.1.0
Repohttp://public-repo-1.hortonworks.com/HDF/ubuntu16/3.x/updates/3.1.1.0/hdf.list
Deb tarballhttp://public-repo-1.hortonworks.com/HDF/ubuntu16/3.x/updates/3.1.1.0/HDF-3.1.1.0-ubuntu16-deb.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/ubuntu16/3.x/updates/3.1.1.0/HDF-3.1.1.0-ubuntu16-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/ubuntu16/3.x/updates/3.1.1.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.1.0-35.tar.gz
MiNiFi C++ Agenthttp://public-repo-1.hortonworks.com/HDF/ubuntu16/3.x/updates/3.1.1.0/tars/nifi-minifi-cpp/nifi-minifi-cpp-0.4.0-bin.tar.gz
HDP and Ambari Repositories
Ambarihttp://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.6.1.0/ambari.list
HDP http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0/hdp.list
HDP-UTILS Base URLhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu16
OS Agnostic Downloads
NiFi only
NiFi Toolkit
Docker Hub

https://hub.docker.com/r/hortonworks/nifi/

Tags are latest and 3.1.1.0

NiFi Registry
MiNiFi Java Agent
MiNiFi Toolkit
iOS/Android Libraries

​

Table 1.11. Debian 7 HDF repository & additional download locations

OS FormatDownload location
Debian 7HDF Build Number3.1.1.0-35
HDF Base URLhttp://public-repo-1.hortonworks.com/HDF/debian7/3.x/updates/3.1.1.0
Repohttp://public-repo-1.hortonworks.com/HDF/debian7/3.x/updates/3.1.1.0/hdf.list
Deb tarballhttp://public-repo-1.hortonworks.com/HDF/debian7/3.x/updates/3.1.1.0/HDF-3.1.1.0-debian7-deb.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/debian7/3.x/updates/3.1.1.0/HDF-3.1.1.0-debian7-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/debian7/3.x/updates/3.1.1.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.1.0-35.tar.gz
HDP and Ambari Repositories
Ambari http://public-repo-1.hortonworks.com/ambari/debian7/2.x/updates/2.6.1.0/ambari.list
HDPhttp://public-repo-1.hortonworks.com/HDP/debian7/2.x/updates/2.6.4.0/hdp.list
HDP-UTILS Base URLhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/debian7
OS Agnostic Downloads
NiFi only
NiFi Toolkit
Docker Hub

https://hub.docker.com/r/hortonworks/nifi/

Tags are latest and 3.1.1.0

NiFi Registry
MiNiFi Java Agent
MiNiFi Toolkit
iOS/Android Libraries

​

Table 1.12. NiFi and MiNiFi MSI files

File Location
NiFi MSI http://public-repo-1.hortonworks.com/HDF/windows/3.x/updates/3.1.1.0/nifi-3.1.1.0-35.msi
MiNiFi MSIhttp://public-repo-1.hortonworks.com/HDF/windows/3.x/updates/3.1.1.0/minifi-3.1.1.0-35.msi

​HDF Repository Location for IBM Power Systems

​

Table 1.13. RHEL 7 HDF repository & additional download locations

OS FormatDownload location
RHEL 7 (64-bit):HDF Build Number3.1.1.0-35
HDF Base URLhttp://public-repo-1.hortonworks.com/HDF/centos7-ppc/3.x/updates/3.1.1.0
HDF Repohttp://public-repo-1.hortonworks.com/HDF/centos7-ppc/3.x/updates/3.1.1.0/hdf.repo
RPM tarballhttp://public-repo-1.hortonworks.com/HDF/centos7-ppc/3.x/updates/3.1.1.0/HDF-3.1.1.0-centos7-ppc-rpm.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/centos7-ppc/3.x/updates/3.1.1.0/HDF-3.1.1.0-centos7-ppc-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/centos7-ppc/3.x/updates/3.1.1.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.1.0-35.tar.gz
HDP and Ambari Repositories
Ambari http://public-repo-1.hortonworks.com/ambari/centos7-ppc/2.x/updates/2.6.1.0/ambari.repo
HDP

http://public-repo-1.hortonworks.com/HDP/centos7-ppc/2.x/updates/2.6.4.0/hdp.repo

HDP-UTILShttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7-ppc/hdp-util.repo

​Common Vulnerabilities and Exposures

The following CVEs have been fixed in HDF 3.1.x.

​CVE-2017-12632

Summary: Apache NiFi host header poisoning issue
Severity: Medium
Versions Affected: Apache NiFi 0.1.0 - 1.4.0, HDF 1.x, 2.x, 3.0.x
Description: A malicious host header in an incoming HTTP request could cause NiFi to load resources from an external server.
Mitigation: The fix to sanitize host headers and compare to a controlled whitelist was applied on the Apache NiFi 1.5.0 release. HDF users should upgrade to HDF 3.1.0.

​CVE-2017-15697

Summary: Apache NiFi XSS issue in context path handling
Severity: Moderate
Versions Affected: Apache NiFi 1.0.0 - 1.4.0, HDF 2.0.0 - 3.0.x
Description: A malicious X-ProxyContextPath or X-Forwarded-Context header containing external resources or embedded code could cause remote code execution.
Mitigation: The fix to properly handle these headers was applied on the Apache NiFi 1.5.0 release. HDF users should upgrade to HDF 3.1.0.

​CVE-2017-12623

Summary: Apache NiFi XXE issue in template XML upload
Severity: Important
Versions Affected: Apache NiFi 1.0.0 - 1.3.0; HDF 2.x, 3.0.0 - 3.0.1.1
Impact: Any authenticated user could upload a template which contained malicious code and accessed sensitive files via an XML External Entity (XXE) attack.
Mitigation: The fix to properly handle XML External Entities was applied on the Apache NiFi 1.4.0 release. Users running a prior 1.x release should upgrade to the appropriate release. HDF users should upgrade to HDF 3.1.0.

​CVE-2017-15703

Summary: Apache NiFi Java deserialization issue in template XML upload
Severity: Moderate
Versions Affected: Apache NiFi 1.0.0 - 1.3.0; HDF 2.x, 3.0.0 - 3.0.1.1
Description: Any authenticated user (valid client certificate but without ACL permissions) could upload a template which contained malicious code and caused a denial of service via Java deserialization attack.
Mitigation: The fix to properly handle Java deserialization was applied on the Apache NiFi 1.4.0 release. HDF users should upgrade to HDF 3.1.0.

​Known Issues

Hortonworks Bug ID

Apache JIRA

Component

Summary

    
BUG-97319 Ambari/NiFi

Issue: After you have upgraded HDF services on an HDP cluster, you may experience issues starting NiFi.

Problem: After upgrade, NiFi starts but Ambari reports that it is stopped. The command line NiFi service is running on the host.

Workaround: To work around this issue, click the Ambari Restart All option for NiFi. NiFi starts successfully.

BUG-94090 SAM

Issue: You may be unable to import some topologies from HDF 3.0.x

Problem: When you have a new HDF 3.1.0 installation, you cannot import topologies created in HDF 3.0.x.

Workaround: To work around this issue, you can upgrade your existing version of HDF to HDF 3.1.0.

  NiFi

Issue: When you have upgraded HDF services on your HDP cluster, the NiFi version number displayed in Ambari does not change from NiFi 1.2.0 to NiFi 1.5.0.

Workaround: After performing the upgrade scenario for HDF services on an HDP cluster as documented in the Ambari Managed HDF Upgrade documentation, disregard the NiFi version displayed in Ambari. NiFi has been successfully upgraded.

  NiFi

Issue: Unable to load native library.

Associated error message: You may encounter an error message similar to one of the following:

  • Failed to write to HDFS of libhadoop was built without snappy support... Native snappy library not available: this version of libhadoop was built without snappy support.

  • Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: Native Library /path/to/hadoop/lib/native/libhadoop.so.1.0.0 already loaded in another classloader

Problem: Native libraries can only be loaded into the Java runtime by a single ClassLoader. Once a ClassLoader loads a native library, subsequent attempts to load that same native library by a different ClassLoader will fail. If multiple components in NiFi attempt to load a native library, it will fail to do so and the components may fail to execute if it attempts to use those native libraries. Whether those components fail ultimately depend on the order they are loaded and executed which is not guaranteed by NiFi. This will occur most frequently with components that leverage the native Hadoop library. These components include but are not limited to:

  • NiFi Ranger Authorizer

  • Any HDFS Processors

  • Any HBase Processors

  • Any Hive Processors

Workaround: This issue is most frequently encountered using an HDFS Processor with a Compression Codec that Hadoop leverages native libraries to implement. In this scenario, the dataflow can be updated to utilize a CompressContent Processor before/after the HDFS Processor to apply the desired compression. The CompressContent does not leverage native libraries and will not be susceptible to this known issue.

BUG-94989 Storm/Ambari

Issue: When performing an Ambari managed rolling upgrade, you may encounter an inaccurate warning message indicating that Storm topologies need to be stopped.

Associated error message: Storm topologies need to stop.

Workaround: You may safely ignore this error message. Storm topologies do not need to be stopped before performing a rolling upgrade.

 KNOX-1108NiFi/Knox

Issue: NiFi/Knox integration does not support HA.

In NiFiHaDispatch, executeRequest is overridden and does not have the try/catch block in DefaultHaDispatch's executeRequest method which is used to catch exceptions and begin the fail over process.

Workaround: There is no workaround for this issue.

BUG-90903N/ANiFi/Knox

Issue: Knox HA failover for NiFi is not supported.

Workaround: There is no workaround for this issue.

BUG-63132N/AStorm

Summary: Solr bolt does not run in a Kerberos environment.

Associated error message: The following is an example:

[ERROR] Request to collection hadoop_logs failed due to (401) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http:[...] Error 401 Authentication required

Workaround: None at this time.

​Third-Party Licenses

HDF 3.1.1 deploys numerous third-party licenses and dependencies, all of which are compatible with the Apache software license. For complete third-party license information, see the licenses and notice files contained within the distribution.

© 2012–2020, Cloudera, Inc.
Document licensed under the Creative Commons Attribution ShareAlike 4.0 License.
Cloudera.com | Documentation | Support | Community