Chapter 6. Configuring for High Availability

To enable high availability you must manually configure the Falcon server. When the primary Falcon server is down, the system administrator must manually start the backup Falcon server. Then the backup Falcon server continues where the primary server stopped.

	Note
	Due to changes in Hive, for the Oozie HCAT URI (which is used for Hive table feeds) Falcon supports URIs with only one metastore. This applies even if you have multiple metastores configured.

Configuring Properties and Setting Up Directory Structure for High Availability

The Falcon server stores its data in the startup.properties file that is located in the <falcon_home>/conf directory. You should configure the startup properties as follows to achieve high availability:

*.config.store.uri: This location should be a directory on HDFS.

*.retry.recorder.path: This location should be an NFS-mounted directory that is owned by Falcon and has permissions set to 755 (rwx/r-x/r-x).

*.falcon.graph.storage.directory: This location should also be an NFS-mounted directory that is owned by Falcon, and with permissions set to 755.

Falcon conf directory: The default location of this directory is <falcon_home>/conf, which is symbolically linked to /etc/falcon/conf.

This directory value must point to an NFS-mounted directory to ensure that the changes made on the primary Falcon server are populated to the backup server.

To set up an NFS-mounted directory, follow these steps:

The example input uses 240.0.0.10 for the NFS server, 240.0.0.12 for the primary Falcon server, and 240.0.0.13 for the backup Falcon server.

On the server that hosts the NFS mount directory, log in as root and perform the following steps.
1. Install and start NFS:
```
yum install nfs-utils nfs-utils-lib
chkconfig nfs on
service rpcbind start
service nfs start
```
2. Create a directory that holds the Falcon data:
```
mkdir -p /hadoop/falcon/data
```
3. Add the following lines to the file /etc/exports to share the data directories:
```
/hadoop/falcon/data 240.0.0.12(rw,sync,no_root_squash,no_subtree_check)
/hadoop/falcon/data 240.0.0.13(rw,sync,no_root_squash,no_subtree_check)
```
4. Export the shared data directories:
```
exportfs -a
```
Logged in as root, install the nfs-utils package and its library on each of the Falcon servers:
```
yum install nfs-utils nfs-utils-lib
```

Create the NFS mount directory, and then mount the data directories:

mkdir -p /hadoop/falcon/data
mount 240.0.0.10:/hadoop/falcon/data/hadoop/falcon/data

Preparing the Falcon Servers

To prepare the Falcon servers for high availability:

Logged in as root on each of the Falcon servers, ensure that the properties *.retry.recorder.path and *.falcon.graph.storage.directory point to a directory within the NFS-mounted directory: for example, the /hadoop/falcon/data directory shown in the previous example.
Logged in as the falcon user, start the primary Falcon server, not the backup server:
```
<falcon_home>/bin/falcon-start
```

Manually Failing Over the Falcon Servers

When the primary Falcon server fails, you must manually fail over to the backup server:

Logged in as the falcon user, ensure that the Falcon process is not running on the backup server:
```
<falcon-home>/bin/falcon-stop
```
Logged in as root, update the client.properties files on all of the Falcon client nodes and set the property falcon.url to the fully qualified domain name of the backup server.
If Transport Layer Security (TLS) is disabled, use port 15000:
```
falcon.url=http://<back-up-server>:15000/ ### if TLS is disabled
```
If TLS is enabled, use port 15443:
```
falcon.url=https://<back-up-server>:15443/ ### if TLS is enabled
```
Logged in as the falcon user, start the backup Falcon server:
```
<falcon-home>/bin/falcon-start
```