org.apache.hadoop.hbase.master

Class TestMasterFailover

java.lang.Object

org.apache.hadoop.hbase.master.TestMasterFailover

public class TestMasterFailover
extends Object

Constructor Summary

Constructors

Constructor and Description
TestMasterFailover()

Method Summary

Modifier and Type	Method and Description
void	testMasterFailoverWithMockedRIT() Complex test of master failover that tests as many permutations of the different possible states that regions in transition could be in within ZK.
void	testMasterFailoverWithMockedRITOnDeadRS() Complex test of master failover that tests as many permutations of the different possible states that regions in transition could be in within ZK pointing to an RS that has died while no master is around to process it.
void	testMetaInTransitionWhenMasterFailover() Test meta in transition when master failover
void	testOfflineRegionReAssginedAfterMasterRestart() This tests a RIT in offline state will get re-assigned after a master restart
void	testPendingOpenOrCloseWhenMasterFailover() Test region in pending_open/close and failed_open/close when master failover
void	testShouldCheckMasterFailOverWhenMETAIsInOpenedState()
void	testSimpleMasterFailover() Simple test of master failover.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TestMasterFailover

public TestMasterFailover()

Method Detail

testMasterFailoverWithMockedRIT

public void testMasterFailoverWithMockedRIT()
                                     throws Exception

Complex test of master failover that tests as many permutations of the different possible states that regions in transition could be in within ZK.

This tests the proper handling of these states by the failed-over master and includes a thorough testing of the timeout code as well.

Starts with a single master and three regionservers.

Creates two tables, enabledTable and disabledTable, each containing 5 regions. The disabledTable is then disabled.

After reaching steady-state, the master is killed. We then mock several states in ZK.

After mocking them, we will startup a new master which should become the active master and also detect that it is a failover. The primary test passing condition will be that all regions of the enabled table are assigned and all the regions of the disabled table are not assigned.

The different scenarios to be tested are below:

ZK State: OFFLINE

A node can get into OFFLINE state if

• An RS fails to open a region, so it reverts the state back to OFFLINE
• The Master is assigning the region to a RS before it sends RPC

We will mock the scenarios

• Master has assigned an enabled region but RS failed so a region is not assigned anywhere and is sitting in ZK as OFFLINE
• This seems to cover both cases?

ZK State: CLOSING

A node can get into CLOSING state if

• An RS has begun to close a region

We will mock the scenarios

• Region of enabled table was being closed but did not complete
• Region of disabled table was being closed but did not complete

ZK State: CLOSED

A node can get into CLOSED state if

• An RS has completed closing a region but not acknowledged by master yet

We will mock the scenarios

• Region of a table that should be enabled was closed on an RS
• Region of a table that should be disabled was closed on an RS

ZK State: OPENING

A node can get into OPENING state if

• An RS has begun to open a region

We will mock the scenarios

• RS was opening a region of enabled table but never finishes

ZK State: OPENED

A node can get into OPENED state if

• An RS has finished opening a region but not acknowledged by master yet

We will mock the scenarios

• Region of a table that should be enabled was opened on an RS
• Region of a table that should be disabled was opened on an RS

Throws:

Exception

testMasterFailoverWithMockedRITOnDeadRS

public void testMasterFailoverWithMockedRITOnDeadRS()
                                             throws Exception

Complex test of master failover that tests as many permutations of the different possible states that regions in transition could be in within ZK pointing to an RS that has died while no master is around to process it.

This tests the proper handling of these states by the failed-over master and includes a thorough testing of the timeout code as well.

Starts with a single master and two regionservers.

Creates two tables, enabledTable and disabledTable, each containing 5 regions. The disabledTable is then disabled.

After reaching steady-state, the master is killed. We then mock several states in ZK. And one of the RS will be killed.

After mocking them and killing an RS, we will startup a new master which should become the active master and also detect that it is a failover. The primary test passing condition will be that all regions of the enabled table are assigned and all the regions of the disabled table are not assigned.

The different scenarios to be tested are below:

ZK State: CLOSING

A node can get into CLOSING state if

• An RS has begun to close a region

We will mock the scenarios

• Region was being closed but the RS died before finishing the close

ZK State: OPENED

A node can get into OPENED state if

• An RS has finished opening a region but not acknowledged by master yet

We will mock the scenarios

• Region of a table that should be enabled was opened by a now-dead RS
• Region of a table that should be disabled was opened by a now-dead RS

ZK State: NONE

A region could not have a transition node if

• The server hosting the region died and no master processed it

We will mock the scenarios

• Region of enabled table was on a dead RS that was not yet processed
• Region of disabled table was on a dead RS that was not yet processed

Throws:

Exception

testShouldCheckMasterFailOverWhenMETAIsInOpenedState

public void testShouldCheckMasterFailOverWhenMETAIsInOpenedState()
                                                          throws Exception

Throws:

Exception

testOfflineRegionReAssginedAfterMasterRestart

public void testOfflineRegionReAssginedAfterMasterRestart()
                                                   throws Exception

This tests a RIT in offline state will get re-assigned after a master restart

Throws:

Exception

testSimpleMasterFailover

public void testSimpleMasterFailover()
                              throws Exception

Simple test of master failover.

Starts with three masters. Kills a backup master. Then kills the active master. Ensures the final master becomes active and we can still contact the cluster.

Throws:

Exception

testPendingOpenOrCloseWhenMasterFailover

public void testPendingOpenOrCloseWhenMasterFailover()
                                              throws Exception

Test region in pending_open/close and failed_open/close when master failover

Throws:

Exception

testMetaInTransitionWhenMasterFailover

public void testMetaInTransitionWhenMasterFailover()
                                            throws Exception

Test meta in transition when master failover

Throws:

Exception