HDP-2.3.4 Release Notes
Also available as:

Behavioral Changes


Hortonworks strongly recommends that all users running HDP 2.3.4 upgrade to HDP

Behavioral changes denote a marked change in behavior from the previously released version to this version of software. In HDP 2.3.4, behavioral changes affect the following Hadoop components.

Table 1.4. Behavioral Changes

Hortonworks Bug IDComponentApache JIRASummaryDetails
BUG-45582Hue All NULL values of Hive query results in Hue Beeswax are blank values in the exported CSV and XLS files.

Previous Behavior: Previously when exporting results data in Hue Beeswax, all NULL values were presented as blank values in the exported text files.

New Behavior: In this release any NULL value in Hive query results is translated to "NULL" string in resulted CSV and XLS files.



HDFS When HDFS metadata is replaced, namenode enters safe mode.

Scenario: A user loads an old Fsimage and starts Namenode. This Fsimage is missing the latest transactions that recently occurred on the cluster.

Previous Behavior: When a user starts the Namenode with this old image, the Namenode will come out of safemode as long as the amount of blocks reported reaches the set threshold (0.999 is default for dfs.namenode.safemode.threshold-pct). When the Namenode learns about a block without a filename, it instructs the Datanode to delete those blocks, resulting in lost data.

New Behavior: The new behavior is that when the Namenode detects such a scenario where there are blocks in the cluster which are tagged with future generation stamps, the Namenode will remain in the safemode state to prevent the blocks from being deleted. If the user manually executes hdfs dfsadmin –safemode leave, the Namenode will refuse to leave safemode. At this point, the user has an opportunity to shut down the Namenode and load the most current FSimage. If the user for some reason is not able to find the latest Fsimage, the last resort to get the cluster up and running with minimal data loss is to force the Namenode out of safemode by running the command hdfs dfsadmin –safemode forceExit after the Namenode has been started.

BUG-45879Ambari, SparkAMBARI-13899Spark configuration now auto-detects hdp.version info

Previous Behavior: Users were required to set hdp.version manually in the configuration file for Ambari, so that Ambari could expose the right version of HDP to configurations. This caused some problems during rolling upgrade; because configurations should not be changed during rolling upgrade, the version number would be incorrect after upgrading.

New Behavior: This change adds a self-detection mechanism for Spark to determine the correct hdp.version, without hdp.version being set through Ambari. It is also backwards compatible with the method described in Previous Behavior.

If this self-detection mechanism fails, you must expose the HDP_VERSION environment variable in spark-env.sh manually.

BUG-46334HDFSHDFS-9184The feature introduces a new setting hadoop.caller.context.enabled. When set to additional fields are written into namenode audit log records to help identify the job or query that introduced each NameNode operation. This feature is enabled by default starting with this release of HDP.

New Behavior: This feature brings a new key-value pair at the end of each audit log record. The newly added key at is callerContext, value context:signature. The overall format would be callerContext=context:signature. If the signature is null or empty, the value will be context only, in the format of callerContext=context. If the hadoop.caller.context.enabled config key is false, the key-value pair will not be showing. The audit log format is not changed in this case. It is also possible to limit the maximum length of context and signature. Consider the hadoop.caller.context.max.size config key (default 128 bytes) and hadoop.caller.context.signature.max.size (default 40 bytes) config key respectively.

There is a chance that the new information in the audit log may break existing scripts/automation that was being used to analyze the audit log. In this case the scripts may need to be fixed. We do not recommend disabling this feature as it can be a useful troubleshooting aid.

BUG-46954Ambari, SparkAMBARI-13584SPARK_CONF_DIR change

Previous Behavior: Previous releases of Spark ignored SPARK_CONF_DIR and always used the system configurations.

New Behavior: The previous behavior has been fixed. If the user has SPARK_CONF_DIR set, that value will be used to locate configuration files for Spark.

BUG-49030Hue No error message is displayed in Hue File Browser when user with no permissions uploads a file.

Previous Behavior: Hue File Browser not displaying error messages when Ranger policy denies access to a file.

New Behavior: In this release, the error message is displayed and restricted the length of these messages up to 1500 characters to be shown in pop-up dialogs.