Cloudera Docs

Enabling High Availability and automatic failover

You can use to configure your cluster for HDFS HA and automatic failover. In , HA is implemented using Quorum-based storage. Quorum-based storage relies upon a set of JournalNodes, each of which maintains a local edits directory that logs the modifications to the namespace metadata. Enabling HA enables automatic failover as part of the same command.

  • Cluster Administrator (also provided by Full Administrator)
  • The Enable High Availability workflow leads you through adding a second (standby) NameNode and configuring JournalNodes.
  1. Go to the HDFS service.
  2. Select Actions > Enable High Availability.
    A screen showing the hosts that are eligible to run a standby NameNode and the JournalNodes displays.
    1. Specify a name for the nameservice and click Continue.
    2. In the NameNode Hosts field, click Select a host.
      The host selection dialog box displays.
    3. Select the checkbox next to the hosts where you want the standby NameNode to be set up and click OK.
      The standby NameNode cannot be on the same host as the active NameNode, and the host that is chosen should have the same hardware configuration (RAM, disk space, number of cores, and so on) as the active NameNode.
    4. In the JournalNode Hosts field, click Select hosts.
      The host selection dialog box displays.
    5. Check the checkboxes next to an odd number of hosts (a minimum of three) to act as JournalNodes and click OK.
      JournalNodes should be hosted on hosts with similar hardware specification as the NameNodes. recommends that you put a JournalNode each on the same hosts as the active and standby NameNodes, and the third JournalNode on similar hardware, such as the JobTracker.
    6. Click Continue.
    7. In the JournalNode Edits Directory property, enter a directory location for the JournalNode edits directory into the fields for each JournalNode host.
      • You may enter only one directory for each JournalNode. The paths do not need to be the same on every JournalNode.
      • The directories you specify should be empty.
      • The directory owner should be hdfs:hadoop and must have read, write, and r permission (drwx------).
    8. Extra Options: Decide whether should clear existing data in ZooKeeper, standby NameNode, and JournalNodes.
      If the directories are not empty (for example, you are re-enabling a previous HA configuration), will not automatically delete the contents—you can select to delete the contents by keeping the default checkbox selection. The recommended default is to clear the directories. If you choose not to do so, the data should be in sync across the edits directories of the JournalNodes and should have the same version data as the NameNodes.
    9. Click Continue.

      executes a set of commands that stop the dependent services, delete, create, and configure roles and directories as required, create a nameservice and failover controller, restart the dependent services, and deploy the new client configuration.

  3. Restart Ranger KMS, if configured for your cluster.
  4. Configure HDFS HA for other services, if required.
    Configuring other components to use HDFS HA