Customizing Critical Hive Configurations

You receive property configuration guidelines, including which properties you need to reconfigure after upgrading. You understand which the upgrade process carries over from your old cluster to your new cluster.

The CDP upgrade process tries to preserve your Hive configuration property overrides. These overrides are the custom values you set to configure Hive in the old CDH or HDP cluster. The upgrade process does not perserve all overrides. For example, a custom value you set for hive.exec.max.dynamic.partitions.pernode is preserved. In the case of other properties, for example hive.cbo.enable, the upgrade ignores any override and just sets the CDP-recommended valued. Hive Configuration Requirements and Recommendations (link below) indicates which overrides the upgrade process preserves or disregards (Safety Valve Overrides column).

The upgrade process does not preserve overrides to the configuration values of the following properties that you likely need to reconfigure to meet your needs:

hive.conf.hidden.list
hive.conf.restricted.list
hive.exec.post.hooks
hive.script.operator.env.blacklist
hive.security.authorization.sqlstd.confwhitelist
hive.security.command.whitelist

The Apache Hive Wiki (link below) describes these properties. The values of these properties are lists.

The upgrade process ignores your old list and sets a new generic list. For example, the hive.security.command.whitelist value is a list of security commands on the whitelist. Any whitelist overrides you set in the old cluster are not preserved. If you did not override whitelist values in the old cluster, you still need to check and change the new default values in CDP. The new default is probably a shorter (more restrictive) list than the original default you were using in the old cluster.

Check and change each property listed above after upgrading as described in the next topic.

Consider reconfiguring more property values than the six listed above. Even if you didn't override the default value in the old cluster, the CDP default might have changed in a way that impacts your work. Hive Configuration Changes (link below) lists the old CDH/HDP and new CDP defaults.