Livy high availability support
Livy supports high availability. If there is more than one Livy server in the CDP cluster, high availability is automatically enabled by Cloudera Manager by setting the livy.server.recovery.mode property to ha and by including the list of Zookeeper servers in the Livy configuration.
- JDBC connection: The JDBC connections are not redirected. More details about this limitation are available below.
- Load balancing: The active-passive high availability
notprovide additional parallelism or capacity. Interactive sessions are only handled by the active server, adding more servers does not distribute load across servers.
Using Livy without Knox gateway
If you are not using Livy through Knox gateway, clients must follow HTTP redirects and resend authentication. The logic the clients can use is to make a list of the Livy servers by obtaining them from the Cloudera Manager API and if any of the servers do not respond, the clients should retry sending the request to another server in the list. For example, in the case of cURL, the --location-trusted flag has to be specified to follow redirects and resend authentication.
Livy high availability and JDBC connection
Livy provides high availability by active-passive setup. Only the active Livy server node can accept JDBC connections, and the passive nodes reject connection attempts. Therefore, if a client wants to connect using JDBC, it has to iterate through all servers and check which one accepts connections. If the active server goes down, the connection is broken and another node takes over the active role. This behavior is the same for both HTTP and binary mode connections.