Descriptions of New Features

Hortonworks Data Platform (HDP) 3.1.4 provides you with several new capabilities. You must learn about the new features and their benefits to use the new features in HDP 3.1.4.


Apache component	Feature areas, benefits, and links to further information
Atlas	Many fixes on top of Apache Atlas 1.1.0, including these new features: Support for process execution entities that represent separate execution instances for a single process. Hive Metastore hook for collecting metadata. Bulk API for retrieving entities by unique attributes.
Hive	Core Capabilities Automatic partition management synchronizes changes in the metadata and on the file system. You can now configure how long you can retain partition data and metadata. Hive Warehouse Connector now validates mapping of columns against those in Hive to alert the user to input errors. Writing a DataFrame to Hive supports specifying partitions. A new MergeBuilder interface for HiveWarehouseSession API operations supports merging tables. Upgrade HiveStrictManagedMigration has two new options: Specify the number of threads for processing tables in parallel. Default: The number of CPU cores. Specify the the type of tables to process. For example, MANAGED_TABLE, EXTERNAL_TABLE. Default: All tables.
Kakfa	Core Capabilities New Kafka Streams configuration parameter upgrade allows rolling bounce upgrade from an older version. Updated ProcessorStateManager APIs in Kafka Streams for registering state stores to the processor topology. For more details please read the Streams Upgrade Guide. Dynamic broker configuration options can be stored in ZooKeeper using kafka-configs.sh before brokers are started. This option can be used to avoid storing clear passwords in server.properties as all password configs may be stored encrypted in ZooKeeper. ZooKeeper hosts are now re-resolved if the connection attempt fails. But if your ZooKeeper host names resolve to multiple addresses, and some of them are not reachable, you may need to increase the connection timeout `zookeeper.connection.timeout.ms` property. Ability to define ACLs on prefixed resources, for example, any topic starting with 'foo'.
Ranger	Core Capabilities Support for time-bound classification or business catalog mapping. Ranger policy engine recognizes the start and end times for tags and enforces the policies based on the period of validity of the tags. Support for Safenet KeySecure crypto platform for storing master keys of Ranger KMS. Support to allow users to define policy level conditions in Ranger.
Spark	Integration Spark Kafka source integration with schema registry.
Zeppelin	Security Support for SPNEGO based user authentication for zeppelin. With SPNEGO based authentication, LDAP/Active Directory based user search for notebook ACLs will continue to work. Integration Unbundle Zeppelin Notebook from Zeppelin installation.