Prerequisites for Setting up Cloudera Manager High Availability

  • A multi-homed TCP load balancer, or two TCP load balancers, capable of proxying requests on specific ports to one server from a set of backing servers.
    • The load balancer does not need to support termination of TLS/SSL connections.
    • This load balancer can be hardware or software based, but should be capable of proxying multiple ports. HTTP/HTTPS-based load balancers are insufficient because Cloudera Manager uses several non-HTTP-based protocols internally.
    • This document uses HAProxy, a small, open-source, TCP-capable load balancer, to demonstrate a workable configuration.
  • A networked storage device that you can configure to be highly available. Typically this is an NFS store, a SAN device, or a storage array that satisfies the read/write throughput requirements of the Cloudera Management Service. This document assumes the use of NFS due to the simplicity of its configuration and because it is an easy, vendor-neutral illustration.
  • The procedures in this document require ssh access to all the hosts in the cluster where you are enabling high availability for Cloudera Manager.

The Heartbeat Daemon and Virtual IP Addresses

You may have configured Cloudera Manager high availability by configuring virtual IP addresses and using the Heartbeat daemon (http://www.linux-ha.org/wiki/Heartbeat). The original Heartbeat package is deprecated; however, support and maintenance releases are still available through LinBit ( https://www.linbit.com/en/linbit-takes-over-heartbeat-maintenance/).

Cloudera recommends using Corosync and Pacemaker (both currently maintained through ClusterLabs). Corosync is an open-source high-availability tool commonly used in the open-source community.

Editions of this document released for Cloudera Manager4 and CDH 4 also used virtual IP addresses that move as a resource from one host to another on failure. Using virtual IP addresses has several drawbacks:
  • Questionable reliance on outdated Address Resolution Protocol (ARP) behavior to ensure that the IP-to-MAC translation works correctly to resolve to the new MAC address on failure.
  • Split-brain scenarios that lead to problems with routing.
  • A requirement that the virtual IP address subnet be shared between the primary and the secondary hosts, which can be onerous if you deploy your secondaries off site.

Therefore, Cloudera no longer recommend the use of virtual IP addresses, and instead recommends using a dedicated load balancer.

Single-User Mode, TLS, and Kerberos

High availability, as described in this document, supports the following: