Cloudera Docs

Oozie and client configurations

Learn about how Oozie handles the client configuration files for different actions that rely on Hadoop, Hive, and so on, clients.

In certain cases, it is necessary to have the required client configurations (site.xml files) available for action executions in YARN. Previously, you manually copied some site.xml files to either your workflow lib folder or the corresponding Oozie ShareLib. This section summarizes how Oozie utilizes the client configuration files.

Hive2 action

The hive-site.xml is not automatically propagated to YARN. The Hive2 action executes the beeline command, which uses the hive-site.xml file from /etc/hive/conf.

Oozie automatically propagates the hbase-site.xml to YARN if the conditions described in the Conditions section are met.

MapReduce and Shell actions

The hbase-site.xml, hive-site.xml, and sqoop-site.xml are propagated to the YARN container if the corresponding conditions mentioned in the Conditions section are met.

Spark and Spark3 actions

Oozie automatically propagates the sqoop-site.xml to YARN if the conditions described in the Conditions section are met.

Oozie automatically propagates the hive-site.xml to YARN if the Hive service dependency is enabled for Oozie in Cloudera Manager.

Oozie automatically propagates the hbase-site.xml to YARN if the HBase service dependency is enabled for Oozie in Cloudera Manager.

Sqoop action

Oozie automatically propagates the hive-site.xml to YARN if the conditions described in the Conditions section are met.

Oozie automatically propagates the sqoop-site.xml to YARN if the Sqoop client dependency is enabled for Oozie in Cloudera Manager.

Oozie automatically propagates the hbase-site.xml to YARN if the HBase service dependency is enabled for Oozie in Cloudera Manager.

Conditions

All the client configurations are only automatically saved to the YARN container if no existing file is already present. This allows you to manually make a client configuration file available in the Workflow's lib folder or the corresponding Oozie ShareLib.

The following applies to the client configurations mentioned in the sections above, where the Conditions section is referenced:

To enable Oozie to automatically copy the client configuration files, the following dependencies must be enabled in Cloudera Manager for Oozie:
  • hbase-site.xml

    HBase service dependency.

  • hive-site.xml

    Hive service dependency.

  • sqoop-site.xml

    Sqoop client dependency.

Automatic propagation of these configurations can be disabled by setting the following settings to true:
  • hbase-site.xml: oozie.action.propagate.hbase-site.xml.disabled
  • hive-site.xml: oozie.action.propagate.hive-site.xml.disabled
  • sqoop-site.xml: oozie.action.propagate.sqoop-site.xml.disabled
You can configure these settings globally using a safety-valve, or in the Workflow's global configuration, or in the job.properties file, or at the action-level through the action's configuration in the workflow definition. The order of precedence is as follows:
  1. Action-level configuration
  2. Workflow global configuration
  3. job.properties configuration
  4. Global safety-valve configuration