Configure Knox for HA
Knox provides basic failover and retry functionality for REST API calls made to a service when service HA has been configured and enabled.
To enable HA functionality in Knox the following configuration must be added to the topology file:
| Service | Parameter |
|---|---|
| WebHDFS |
<param>
<name>WEBHDFS</name>
<value>maxFailoverAttempts=3;failoverSleep=1000;maxRetryAttempts=300;retrySleep=1000;enabled=true</value>
</param>
|
| HBase |
<param>
<name>HBASE</name>
<value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
</param>
|
| Hive |
<param>
<name>HIVE</name>
<value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true;zookeeperEnsemble=
|
| Oozie |
<param>
<name>OOZIE</name>
<value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
</param>
|
| WebHCat |
<param>
<name>WEBHCAT</name>
<value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
</param>
|
The various configuration parameters are described below:
maxFailoverAttempts-- The maximum number of times a failover will be attempted. The current failover strategy is very simplistic in that the next URL in the list of URLs provided for the service is used, and the one that failed is put at the bottom of the list. If the list is exhausted and the maximum number of attempts has not been reached, the first URL that failed will be tried again (the list will start again from the original top entry).failoverSleep-- The amount of time in milliseconds that the process will wait or sleep before attempting to failover.maxRetryAttempts-- The maximum number of times that a retry request will be attempted. Unlike failover, the retry is done on the same URL that failed. This is a special case in HDFS when the node is in safe mode. The expectation is that the node will come out of safe mode, so a retry is desirable here as opposed to a failover.retrySleep-- The amount of time in milliseconds that the process will wait or sleep before a retry is issued.enabled- Flag to turn the particular service on or off for HA.
The additional configuration parameters for Hive are described below:
zookeeperEnsemble-- A comma separated list of host names (or IP addresses) of the zookeeper hosts that consist of the ensemble that the Hive servers register their information with. This value can be obtained from Hive’s config file hive-site.xml as the value for the parameter ‘hive.zookeeper.quorum’.zookeeperNamespace-- This is the namespace under which HiveServer2 information is registered in the ZooKeeper ensemble. This value can be obtained from Hive’s config file hive-site.xml as the value for the parameter ‘hive.server2.zookeeper.namespace’.
For the service configuration itself, the additional URLs for standby nodes should be added to the list. The active URL (at the time of configuration) should ideally be added at the top of the list. Example for HBase, Oozie, WebHCat, and WebHDFS:
<service>
<role>{COMPONENT}</role>
<url>http://{host1}:50070/{component}</url>
<url>http://{host2}:50070/{component}</url>
</service>Example for Hive:
<service>
<role>HIVE</role>
</service>Please note that there is no <url> tag specified here as the URLs for the Hive servers are obtained from ZooKeeper.