org.apache.hadoop.hbase

Class IntegrationTestRegionReplicaReplication

java.lang.Object

org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.IntegrationTestBase

org.apache.hadoop.hbase.IntegrationTestIngest

org.apache.hadoop.hbase.IntegrationTestRegionReplicaReplication

public class IntegrationTestRegionReplicaReplication
extends IntegrationTestIngest

Integration test for testing async wal replication to secondary region replicas. Sets up a table with given region replication (default 2), and uses LoadTestTool client writer, updater and reader threads for writes and reads and verification. It uses a delay queue with a given delay ("read_delay_ms", default 5000ms) between the writer/updater and reader threads to make the written items available to readers. This means that a reader will only start reading from a row written by the writer / updater after 5secs has passed. The reader thread performs the reads from the given region replica id (default 1) to perform the reads. Async wal replication has to finish with the replication of the edits before read_delay_ms to the given region replica id so that the read and verify will not fail. The job will run for at least given runtime (default 10min) by running a concurrent writer and reader workload followed by a concurrent updater and reader workload for num_keys_per_server.

Example usage:

 hbase org.apache.hadoop.hbase.IntegrationTestRegionReplicaReplication
 -DIntegrationTestRegionReplicaReplication.num_keys_per_server=10000
 -Dhbase.IntegrationTestRegionReplicaReplication.runtime=600000
 -DIntegrationTestRegionReplicaReplication.read_delay_ms=5000
 -DIntegrationTestRegionReplicaReplication.region_replication=3
 -DIntegrationTestRegionReplicaReplication.region_replica_id=2
 -DIntegrationTestRegionReplicaReplication.num_read_threads=100
 -DIntegrationTestRegionReplicaReplication.num_write_threads=100
 

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	IntegrationTestRegionReplicaReplication.DelayingMultiThreadedUpdater This extends MultiThreadedWriter to add a configurable delay to the keys written by the writer threads to become available to the MultiThradedReader threads.
static class	IntegrationTestRegionReplicaReplication.DelayingMultiThreadedWriter This extends MultiThreadedWriter to add a configurable delay to the keys written by the writer threads to become available to the MultiThradedReader threads.

Field Summary

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestIngest

cluster, DEFAULT_NUM_KEYS_PER_SERVER, DEFAULT_NUM_READ_THREADS, DEFAULT_NUM_WRITE_THREADS, DEFAULT_RUN_TIME, HIPHEN, JUNIT_RUN_TIME, LOAD_TEST_TOOL_INIT_ARGS, loadTool, LOG, NUM_KEYS_PER_SERVER_KEY, NUM_READ_THREADS_KEY, NUM_WRITE_THREADS_KEY, RUN_TIME_KEY, util

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestBase

CHAOS_MONKEY_PROPS, monkey, MONKEY_LONG_OPT, monkeyProps, monkeyToUse, NO_CLUSTER_CLEANUP_LONG_OPT, noClusterCleanUp

Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

cmdLineArgs, conf, EXIT_FAILURE, EXIT_SUCCESS

Constructor Summary

Constructors

Constructor and Description
IntegrationTestRegionReplicaReplication()

Method Summary

Modifier and Type	Method and Description
protected int	getMinServerCount()
static void	main(java.lang.String[] args)
protected void	runIngestTest(long defaultRunTime, long keysPerServerPerIter, int colsPerKey, int recordSize, int writeThreads, int readThreads)
void	setConf(Configuration conf)
void	testIngest()

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestIngest

getArgsForLoadTestTool, getArgsForLoadTestToolInitTable, getColumnFamilies, getNumKeys, getTablename, initTable, internalRunIngestTest, runTestFromCommandLine, setUpCluster

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestBase

addOptions, cleanUp, cleanUpCluster, cleanUpMonkey, cleanUpMonkey, doWork, getConf, getDefaultMonkeyFactory, getTestingUtil, processBaseOptions, processOptions, setUp, setUpMonkey, startMonkey

Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, parseArgs, parseInt, parseLong, printUsage, printUsage, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

IntegrationTestRegionReplicaReplication

public IntegrationTestRegionReplicaReplication()

Method Detail

getMinServerCount

protected int getMinServerCount()

Overrides:

getMinServerCount in class IntegrationTestIngest

setConf

public void setConf(Configuration conf)

Overrides:

setConf in class AbstractHBaseTool

testIngest

public void testIngest()
                throws java.lang.Exception

Overrides:

testIngest in class IntegrationTestIngest

Throws:

java.lang.Exception

runIngestTest

protected void runIngestTest(long defaultRunTime,
                             long keysPerServerPerIter,
                             int colsPerKey,
                             int recordSize,
                             int writeThreads,
                             int readThreads)
                      throws java.lang.Exception

Overrides:

runIngestTest in class IntegrationTestIngest

Throws:

java.lang.Exception

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Throws:

java.lang.Exception