7. Upgrade Apache Oozie - Hortonworks Data Platform

Before you upgrade, stop any job in a RUNNING or SUSPENDED state before you upgrade.

	Note
	Proceeding with an Apache Oozie upgrade when you have running or suspended jobs might cause the upgrade to fail or result in data corruption.

Check for running and suspended jobs using the following commands

oozie jobs -oozie http://localhost:11000/oozie -filter status=SUSPENDED
oozie jobs -oozie http://localhost:11000/oozie -filter status=RUNNING

If both commands return results showing that there are no jobs matching the criteria, you can proceed with the upgrade:
```
oozie jobs -oozie http://localhost:11000/oozie -filter status=SUSPENDED
No Jobs match your criteria!
                    
```

If either of the commands returns a list of jobs similar to the following example, you should terminate each job before proceeding:

oozie jobs -oozie http://localhost:11000/oozie -filter status=SUSPENDED
Job ID App Name Status User Group Started Ended 
------------------------------------------------------------------------------------------------------------------------------------
0000005-150710151956050-oozie-oozi-W Spark SUSPENDED oozie - 2015-09-16 16:47 GMT 2015-09-16 16:47 GMT 
------------------------------------------------------------------------------------------------------------------------------------
0000004-150710151956050-oozie-oozi-W Pig1 SUSPENDED oozie - 2015-09-16 16:46 GMT 2015-09-16 16:46 GMT 
------------------------------------------------------------------------------------------------------------------------------------
0000003-150710151956050-oozie-oozi-W Pig1 SUSPENDED oozie - 2015-09-11 00:02 GMT 2015-09-11 00:02 GMT 
------------------------------------------------------------------------------------------------------------------------------------

oozie jobs -oozie http://localhost:11000/oozie -filter status=RUNNING
Job ID App Name Status User Group Started Ended 
------------------------------------------------------------------------------------------------------------------------------------
0000007-150710151956050-oozie-oozi-W Spark RUNNING oozie - 2015-09-16 16:52 GMT 2015-09-16 16:52 GMT 
------------------------------------------------------------------------------------------------------------------------------------
0000006-150710151956050-oozie-oozi-W Fork_Example RUNNING oozie - 2015-09-16 16:52 GMT 2015-09-16 16:52 GMT 
------------------------------------------------------------------------------------------------------------------------------------

Terminate each running or suspended job using the following command:

oozie job -oozie http://localhost:11000/oozie -kill 0000007-150710151956050-oozie-oozi-W

Execute the following command on the Oozie server and client machines:

For RHEL/CentOS:
```
yum upgrade oozie  
```
For SLES:
Run the following commands:
```
zypper up oozie
```
For Ubuntu or Debian:
```
apt-get install oozie
```

You must replace your configuration after upgrading. Copy /etc/oozie/conf from the template to the conf directory on each oozie server and client.

Change the JDBC config to match your Oozie database. The entries to edit are:

oozie.service.JPAService.jdbc.driver
oozie.service.JPAService.jdbc.url

For example, for MySQL, use:

oozie.service.JPAService.jdbc.driver = com.mysql.jdbc.Driver
oozie.service.JPAService.jdbc.url = jdbc:mysql://$my_server:my_port/oozie?createDatabaseIfNotExist=true

Copy the JDBC jar to libext-customer.

Create the /usr/lib/oozie/libext-customer directory.
```
cd /usr/lib/oozie
mkdir libext-customer
```
Grant read/write/execute access to all users for the libext-customer directory.
```
chmod -R 777 /usr/lib/oozie/libext-customer
```

Copy the JDBC jar of your Oozie database to the libext-customer directory. For example, if you are using MySQL, copy your mysql-connector-java.jar to /usr/lib/oozie/libext-customer and /usr/lib/oozie/libtools.

	Note
	HDP 2.1 does not provide the MySQL jar. You can download one from MySQL, when you download mysql-connector-java here.

Copy these files to the libext-customer directory

cp /usr/lib/hadoop/lib/hadoop-lzo*.jar /usr/lib/oozie/libext-customer
cp /usr/share/HDP-oozie/ext-2.2.zip /usr/lib/oozie/libext-customer/

Extract share-lib.

cd /usr/lib/oozie
tar xzvf /usr/lib/oozie//oozie-sharelib.tar.gz
su -l hdfs -c "hdfs dfs -mkdir -p /user/oozie"
su -l hdfs -c "hdfs dfs -copyFromLocal /usr/lib/oozie/share /user/oozie/."

You may see complaints that some files exist. This is an expected behavior. Delete any exisitng /oozie/share and replace it with the newly extracted files.

su -l hdfs -c "hdfs dfs -chown oozie:hadoop /user/oozie"
su -l hdfs -c "hdfs dfs -chmod -R 755 /user/oozie"

Run upgrade as the Oozie user. Do not run as the root user to execute this.

su $OOZIE_USER
/usr/lib/oozie/bin/ooziedb.sh upgrade -run

Prepare the Oozie WAR file. Run as root:

sudo su -l oozie -c "/usr/lib/oozie/bin/oozie-setup.sh prepare-war -d /usr/lib/oozie/libext-customer"

Look for console output to indicate success. For example, if you are using MySQL you should see something similar to:

INFO: Adding extension: libext-customer/mysql-connector-java.jar
New Oozie WAR file with added 'JARs' at /var/lib/oozie/oozie-server/webapps/oozie.war

Replace the content of /user/oozie/share in HDFS. On the Oozie server host:

Verify that all the Oozie jobs have been successfully stupped by running the following commands:
```
oozie jobs -oozie http://localhost:11000/oozie -filter status=SUSPENDED
oozie jobs -oozie http://localhost:11000/oozie -filter status=RUNNING
                    
```
If either of these commands indicates suspended or running jobs, you should repeat step 1 before proceeding.

Extract the Oozie sharelib into a tmp folder.

mkdir -p /tmp/oozie_tmp
cp /usr/lib/oozie/oozie-sharelib.tar.gz /tmp/oozie_tmp
cd /tmp/oozie_tmp
tar xzvf oozie-sharelib.tar.gz

Back up the/user/oozie/share folder in HDFS and then delete it. If you have any custom files in this folder back them up separately and then add them back after the share folder is updated.
```
su -l hdfs -c "hdfs dfs -copyToLocal /user/oozie/share /tmp/oozie_tmp/oozie_share_backup"
su -l hdfs -c "hdfs dfs -rm -r /user/oozie/share"
```

Add the latest share libs that you extracted in step 1. After you have added the files, modify ownership and ACL.

su -l hdfs -c "hdfs dfs -copyFromLocal /tmp/oozie_tmp/share /user/oozie/."
su -l hdfs -c "hdfs dfs -chown -R oozie:hadoop /user/oozie"
su -l hdfs -c "hdfs dfs -chmod -R 755 /user/oozie"

Set the oozie.service.WorkflowAppService.system.libpath in oozie-site.xml to the right path of sharelib in hdfs.

<property>
  <name>oozie.service.WorkflowAppService.system.libpath</name>
  <value>/user/$
  {user.name}
  /share/lib</value>
  <description>
  System library path to use for workflow applications.
  This path is added to workflow application if their job properties sets the property 'oozie.use.system.libpath' to true.
  </description>
</property>

Start Oozie. Run as root.

sudo su -l oozie -c "cd /grid/0/var/log/oozie; /usr/lib/oozie/bin/oozie-start.sh"

Check processes.

ps -ef | grep -i oozie