The Oozie Service
Cloudera Manager installs the Oozie service as part of the CDH installation.
You can elect to have the service created and started as part of the first run installation wizard. If you elect not to create the service using the installation wizard, you can use the Add Service wizard to perform the installation. The wizard will automatically configure and start the dependent services and the Oozie service. See Adding a Service for instructions.
For information on configuring Oozie for high availability, see Oozie High Availability.
Redeploying the Oozie ShareLib
Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)
Some Oozie actions – specifically DistCp, Streaming, Pig, Sqoop, and Hive – require external JAR files in order to run. Instead of having to keep these JAR files in each workflow's lib folder, or forcing you to manually manage them via the oozie.libpath property on every workflow using one of these actions, Oozie provides the ShareLib. The ShareLib behaves very similarly to oozie.libpath, except that it’s specific to the aforementioned actions and their required JARs.
When you upgrade CDH or switch between MapReduce and YARN computation frameworks, redeploy the Oozie ShareLib as follows:
- Go to the Oozie service.
- Select .
- Select .
- Select .
Adding Schema to Oozie
Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)
- In the Cloudera Manager Admin Console, go to the Oozie service.
- Click the Configuration tab.
- Click Oozie Server Default Group.
- Select the Oozie SchemaService Workflow Extension Schemas property.
- Enter the desired schema from Oozie Schema - CDH 5, appending .xsd to each entry.
- Click Save Changes to commit the changes.
- Restart the Oozie service.
CDH 5.2.0 | CDH 5.1.0 | CDH 5.0.0 | |
---|---|---|---|
distcp |
distcp-action-0.1 distcp-action-0.2 |
distcp-action-0.1 distcp-action-0.2 |
distcp-action-0.1 distcp-action-0.2 |
|
email-action-0.1 |
email-action-0.1 email-action-0.2 |
email-action-0.1 |
hive |
hive-action-0.2 hive-action-0.3 hive-action-0.4 hive-action-0.5 |
hive-action-0.2 hive-action-0.3 hive-action-0.4 hive-action-0.5 |
hive-action-0.2 hive-action-0.3 hive-action-0.4 hive-action-0.5 |
HiveServer2 |
hive2-action-0.1 |
||
oozie-bundle |
oozie-bundle-0.1 oozie-bundle-0.2 |
oozie-bundle-0.1 oozie-bundle-0.2 |
oozie-bundle-0.1 oozie-bundle-0.2 |
oozie-coordinator |
oozie-coordinator-0.1 oozie-coordinator-0.2 oozie-coordinator-0.3 oozie-coordinator-0.4 |
oozie-coordinator-0.1 oozie-coordinator-0.2 oozie-coordinator-0.3 oozie-coordinator-0.4 |
oozie-coordinator-0.1 oozie-coordinator-0.2 oozie-coordinator-0.3 oozie-coordinator-0.4 |
oozie-sla |
oozie-sla-0.1 oozie-sla-0.2 |
oozie-sla-0.1 oozie-sla-0.2 |
oozie-sla-0.1 oozie-sla-0.2 |
oozie-workflow |
oozie-workflow-0.1 oozie-workflow-0.2 oozie-workflow-0.2.5 oozie-workflow-0.3 oozie-workflow-0.4 oozie-workflow-0.4.5 oozie-workflow-0.5 |
oozie-workflow-0.1 oozie-workflow-0.2 oozie-workflow-0.2.5 oozie-workflow-0.3 oozie-workflow-0.4 oozie-workflow-0.4.5 oozie-workflow-0.5 |
oozie-workflow-0.1 oozie-workflow-0.2 oozie-workflow-0.2.5 oozie-workflow-0.3 oozie-workflow-0.4 oozie-workflow-0.4.5 oozie-workflow-0.5 |
shell |
shell-action-0.1 shell-action-0.2 shell-action-0.3 |
shell-action-0.1 shell-action-0.2 shell-action-0.3 |
shell-action-0.1 shell-action-0.2 shell-action-0.3 |
sqoop |
sqoop-action-0.2 sqoop-action-0.3 sqoop-action-0.4 |
sqoop-action-0.2 sqoop-action-0.3 sqoop-action-0.4 |
sqoop-action-0.2 sqoop-action-0.3 sqoop-action-0.4 |
ssh |
ssh-action-0.1 ssh-action-0.2 |
ssh-action-0.1 ssh-action-0.2 |
ssh-action-0.1 ssh-action-0.2 |
CDH 4.6.0-4.3.0 | CDH 4.2.0 | CDH 4.1.0 | CDH 4.0.0 | |
---|---|---|---|---|
distcp |
distcp-action-0.1 distcp-action-0.2 |
distcp-action-0.1 distcp-action-0.2 |
distcp-action-0.1 |
distcp-action-0.1 |
|
email-action-0.1 |
email-action-0.1 |
email-action-0.1 |
email-action-0.1 |
hive |
hive-action-0.2 hive-action-0.3 hive-action-0.4 hive-action-0.5 |
hive-action-0.2 hive-action-0.3 hive-action-0.4 |
hive-action-0.2 hive-action-0.3 hive-action-0.4 |
hive-action-0.2 |
oozie-bundle |
oozie-bundle-0.1 oozie-bundle-0.2 |
oozie-bundle-0.1 oozie-bundle-0.2 |
oozie-bundle-0.1 oozie-bundle-0.2 |
oozie-bundle-0.1 |
oozie-coordinator |
oozie-coordinator-0.1 oozie-coordinator-0.2 oozie-coordinator-0.3 oozie-coordinator-0.4 |
oozie-coordinator-0.1 oozie-coordinator-0.2 oozie-coordinator-0.3 oozie-coordinator-0.4 |
oozie-coordinator-0.1 oozie-coordinator-0.2 oozie-coordinator-0.3 oozie-coordinator-0.4 |
oozie-coordinator-0.1 oozie-coordinator-0.2 oozie-coordinator-0.3 |
oozie-sla |
oozie-sla-0.1 |
oozie-sla-0.1 |
oozie-sla-0.1 |
oozie-sla-0.1 |
oozie-workflow |
oozie-workflow-0.1 oozie-workflow-0.2 oozie-workflow-0.2.5 oozie-workflow-0.3 oozie-workflow-0.4 oozie-workflow-0.4.5 |
oozie-workflow-0.1 oozie-workflow-0.2 oozie-workflow-0.2.5 oozie-workflow-0.3 oozie-workflow-0.4 |
oozie-workflow-0.1 oozie-workflow-0.2 oozie-workflow-0.2.5 oozie-workflow-0.3 oozie-workflow-0.4 |
oozie-workflow-0.1 oozie-workflow-0.2 oozie-workflow-0.2.5 oozie-workflow-0.3 |
shell |
shell-action-0.1 shell-action-0.2 shell-action-0.3 |
shell-action-0.1 shell-action-0.2 shell-action-0.3 |
shell-action-0.1 shell-action-0.2 shell-action-0.3 |
shell-action-0.1 |
sqoop |
sqoop-action-0.2 sqoop-action-0.3 sqoop-action-0.4 |
sqoop-action-0.2 sqoop-action-0.3 sqoop-action-0.4 |
sqoop-action-0.2 sqoop-action-0.3 sqoop-action-0.4 |
sqoop-action-0.2 |
ssh |
ssh-action-0.1 ssh-action-0.2 |
ssh-action-0.1 |
ssh-action-0.1 |
ssh-action-0.1 |
Enabling the Oozie Web Console
Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)
- Download ext-2.2. Extract the contents of the file to /var/lib/oozie/ on the same host as the Oozie Server.
- In the Cloudera Manager Admin Console, go to the Oozie service.
- Click the Configuration tab.
- Check Enable Oozie server web console.
- Click Save Changes to commit the changes.
- Restart the Oozie service.
Using an External Database for Oozie
- Derby runs in embedded mode and it is not possible to monitor its health.
- It is not clear how to implement a live backup strategy for the embedded Derby database, though it may be possible.
- Under load, Cloudera has observed locks and rollbacks with the embedded Derby database which don't happen with server-based databases.
The databases that Oozie supports are listed at Requirements and Supported Versions.
Setting up an External Database for Oozie
See PostgreSQL, MySQL, or Oracle for the procedure for setting up an Oozie database.
PostgreSQL
Use the procedure that follows to configure Oozie to use PostgreSQL instead of Apache Derby.
Create the Oozie user and Oozie database.
For example, using the PostgreSQL psql command-line tool:
$ psql -U postgres Password for user postgres: ***** postgres=# CREATE ROLE oozie LOGIN ENCRYPTED PASSWORD 'oozie' NOSUPERUSER INHERIT CREATEDB NOCREATEROLE; CREATE ROLE postgres=# CREATE DATABASE "oozie" WITH OWNER = oozie ENCODING = 'UTF8' TABLESPACE = pg_default LC_COLLATE = 'en_US.UTF8' LC_CTYPE = 'en_US.UTF8' CONNECTION LIMIT = -1; CREATE DATABASE postgres=# \q
Configure PostgreSQL to accept network connections for the oozie user.
- Edit the postgresql.conf file and set the listen_addresses property to *, to make sure that the PostgreSQL server starts listening on all your network interfaces. Also make sure that the standard_conforming_strings property is set to off.
- Edit the PostgreSQL data/pg_hba.conf file as follows:
host oozie oozie 0.0.0.0/0 md5
MySQL
Use the procedure that follows to configure Oozie to use MySQL instead of Apache Derby.
Create the Oozie database and Oozie MySQL user.
For example, using the MySQL mysql command-line tool:
$ mysql -u root -p Enter password: mysql> create database oozie default character set utf8; Query OK, 1 row affected (0.00 sec) mysql> grant all privileges on oozie.* to 'oozie'@'localhost' identified by 'oozie'; Query OK, 0 rows affected (0.00 sec) mysql> grant all privileges on oozie.* to 'oozie'@'%' identified by 'oozie'; Query OK, 0 rows affected (0.00 sec) mysql> exit Bye
Oracle
Use the procedure that follows to configure Oozie to use Oracle 11g instead of Apache Derby.
Create the Oozie Oracle user and grant privileges.
The following example uses the Oracle sqlplus command-line tool, and shows the privileges Cloudera recommends.
$ sqlplus system@localhost Enter password: ****** SQL> create user oozie identified by oozie default tablespace users temporary tablespace temp; User created. SQL> grant alter any index to oozie; grant alter any table to oozie; grant alter database link to oozie; grant create any index to oozie; grant create any sequence to oozie; grant create database link to oozie; grant create session to oozie; grant create table to oozie; grant drop any sequence to oozie; grant select any dictionary to oozie; grant drop any table to oozie; grant create procedure to oozie; grant create trigger to oozie; SQL> exit $
Creating the Oozie Database Schema
The Oozie database tool works in 2 modes: it can create the database, or it can produce an SQL script that a database administrator can run to create the database manually. If you use the tool to create the database schema, you must have the permissions needed to execute DDL operations.
To run the Oozie database tool against the database
$ sudo -u oozie /usr/lib/oozie/bin/ooziedb.sh create -run
You should see output such as the following (the output of the script may differ slightly depending on the database vendor) :
Validate DB Connection. DONE Check DB schema does not exist DONE Check OOZIE_SYS table does not exist DONE Create SQL schema DONE DONE Create OOZIE_SYS table DONE Oozie DB has been created for Oozie version '4.0.0-cdh5.0.0' The SQL commands have been written to: /tmp/ooziedb-5737263881793872034.sql
To create the upgrade script
Run /usr/lib/oozie/bin/ooziedb.sh create -sqlfile SCRIPT. For example:
$ sudo -u oozie /usr/lib/oozie/bin/ooziedb.sh create -sqlfile oozie-create.sql
You should see output such as the following (the output of the script may differ slightly depending on the database vendor) :
Validate DB Connection. DONE Check DB schema does not exist DONE Check OOZIE_SYS table does not exist DONE Create SQL schema DONE DONE Create OOZIE_SYS table DONE Oozie DB has been created for Oozie version '4.0.0-cdh5.0.0' The SQL commands have been written to: oozie-create.sql WARN: The SQL commands have NOT been executed, you must use the '-run' option
Configuring Oozie to Use an External Database
Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)
- In the Cloudera Manager Admin Console, go to the Oozie service.
- Click the Configuration tab.
- Expand Oozie Server Default Group and click Database.
- Specify the settings for Oozie Server database type, Oozie Server database name, Oozie Server database host, Oozie Server database user, and Oozie Server database password.
- Click Save Changes to commit the changes.
- Restart the Oozie service.