Configure and Start Apache Oozie

Note

The duration of the Oozie upgrade is dependent on the amount of job history stored in ooziedb. This history must be backed up and restored during the upgrade process. Best practice when planning for upgrade is to backup ooziedb from your production oozie server and restore it to a test or development oozie server. This can help you estimate the time that is be required to upgrade Oozie during your production upgrade.

	Note
	The `su` commands in this section use "hdfs" to represent the HDFS Service user and "oozie" to represent the Oozie Service user. If you are using another name for your HDFS Service user or your Oozie Service user, you need to substitute your Service user names for "hdfs" or "oozie" in each of the `su` commands.

Upgrading Apache Oozie is a complex process. Although the instructions are straightforward, set aside a dedicated block of time to upgrade oozie clients and servers.

Perform the following preparation steps on each oozie server host:

You must restore oozie-site.xml from your backup to the conf directory on each oozie server and client.
Copy the JDBC jar to libext-customer:
1. Create the /usr/hdp/current/oozie/libext-customer directory.
  cd /usr/hdp/current/oozie-server
  mkdir libext-customer
2. Grant read/write/execute access to all users for the libext-customer directory.
  chmod -R 777 /usr/hdp/current/oozie-server/libext-customer
Copy these files to the libext-customer directory
cp /usr/hdp/current/hadoop-client/lib/hadoop*lzo*.jar /usr/hdp/current/oozie-server/libext-customer
cp /usr/share/HDP-oozie/ext.zip /usr/hdp/current/oozie-server/libext-customer/
Also, copy Oozie db jar in libext-customer.
If Falcon was also installed and configured before upgrade in HDP 2.4.x, then after upgrade you might also need to do the following:
cp /usr/hdp/current/falcon-server/oozie/ext/falcon-oozie-el-extension-"jar /usr/hdp/current/oozie-server/libext-customer
Extract share-lib.
/usr/hdp/current/oozie/bin/oozie-setup.sh sharelib create -fs hdfs://<namenode>:8020
To verify that the sharelibs extracted correctly, run the following command:
oozie admin -oozie http://<oozie server host address>:11000/oozie -shareliblist
There should be:
- Available ShareLib
- oozie
- hive
- distcp
- hcatalog
- sqoop
- mapreduce-streaming
- pig
Change the ownershop and permissions of the oozie directory:
su -l hdfs -c "hdfs dfs -chown oozie:hadoop /user/oozie"
su -l hdfs -c "hdfs dfs -chmod -R 755 /user/oozie"
Add the two zip files from spark-client phone to spark sharelib:
```
hdfs dfs -put /usr/hdp/current/spark-client/python/lib/py4j-0.9-src.zip /user/oozie/share/lib/lib_<timestamp>/spark
```
```
hdfs dfs -put /usr/hdp/current/spark–client/python/lib/pyspark.zip /user/oozie/share/lib/lib_<timestamp>/spark
```
If a previous version of Oozie was created using auto schema creation, run the following SQL query:
```
insert into oozie_sys (name, data) values ('db.version', '2.5');
```
As the Oozie user (not root), run the upgrade.
su - oozie -c "/usr/hdp/current/oozie-server/bin/ooziedb.sh upgrade -run"
As root, prepare the Oozie WAR file.
chown oozie:oozie /usr/hdp/current/oozie-server/oozie-server/conf/server.xml
su - oozie -c "/usr/hdp/current/oozie-server/bin/oozie-setup.sh prepare-war -d /usr/hdp/current/oozie-server/libext-customer"
Look for console output to indicate success. For example, if you are using MySQL you should see something similar to:
```
INFO: Adding extension: libext-customer/mysql-connector-java.jar
New Oozie WAR file with added 'JARs' at /var/lib/oozie/oozie-server/webapps/oozie.war 
```
Make sure that following property is added in oozie-log4j.properties:
log4j.appender.oozie.layout.ConversionPattern=%d{ISO8601} %5p %c{1}:%L - SERVER[${oozie.instance.id}] %m%n
where ${oozie.instance.id} is determined by oozie, automatically.

If you have custom Oozie actions, you must define them in oozie-site.xml. Edit the /etc/oozie/conf/oozie-site.xml file and add the following properties:

<property>
     <name>oozie.service.SchemaService.wf.ext.schemas</name>
     <value>[Comma seperated list of custom actions]</value>
</property>
<property>
     <name>oozie.service.SparkConfigurationService.spark.configuration</name> 
     <value>*=/etc/spark/conf/</value>
</property>

For example, if you have added Spark Action, enter the following:

<property>
     <name>oozie.service.SchemaService.wf.ext.schemas</name>
     <value>spark-action-0.1.xsd</value>
</property>

Configure HTTPS for the Oozie server.
1. Create a self signed certificate or get certificate from a trusted CA for the Oozie Server
2. Import the certificate to the client JDK trust store on all client nodes.
3. In the Ambari Oozie configuration, set the following environment variables in oozie-env.sh, adding them if it does not exist:
```
export OOZIE_HTTPS_PORT=11443
export OOZIE_HTTPS_KEYSTORE_FILE=/home/oozie/.keystore
export OOZIE_HTTPS_KEYSTORE_PASS=password
            
```
4. Change OOZIE_HTTP_PORT={{oozie_server_port}} to OOZIE_HTTP_PORT=11000.
5. Set the oozie.base.url to the HTTPS address.
6. Save the configuration, and restart the Oozie components.
Use the /usr/hdp/current/oozie-server/bin/oozied.sh script with the start parameter to start the Oozie server.
Check processes.
ps -ef | grep -i oozie
When needed, the /usr/hdp/current/oozie-server/bin/oozied.sh script with the stop parameter stops the Oozie server.

​Configure and Start Apache Oozie