org.apache.hadoop.hbase.test

Class IntegrationTestTimeBoundedRequestsWithRegionReplicas

java.lang.Object

org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.IntegrationTestBase

org.apache.hadoop.hbase.IntegrationTestIngest

org.apache.hadoop.hbase.test.IntegrationTestTimeBoundedRequestsWithRegionReplicas

Direct Known Subclasses:

IntegrationTestTimeBoundedMultiGetRequestsWithRegionReplicas

public class IntegrationTestTimeBoundedRequestsWithRegionReplicas
extends IntegrationTestIngest

An IntegrationTest for doing reads with a timeout, to a read-only table with region replicas. ChaosMonkey is run which kills the region servers and master, but ensures that meta region server is not killed, and at most 2 region servers are dead at any point in time. The expected behavior is that all reads with stale mode true will return before the timeout (5 sec by default). The test fails if the read requests does not finish in time.

This test uses LoadTestTool to read and write the data from a single client but multiple threads. The data is written first, then we allow the region replicas to catch up. Then we start the reader threads doing get requests with stale mode true. Chaos Monkey is started after some delay (20 sec by default) after the reader threads are started so that there is enough time to fully cache meta. These parameters (and some other parameters from LoadTestTool) can be used to control behavior, given values are default:

 -Dhbase.DIntegrationTestTimeBoundedRequestsWithRegionReplicas.runtime=600000
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_regions_per_server=5
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.get_timeout_ms=5000
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_keys_per_server=2500
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.region_replication=3
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_read_threads=20
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_write_threads=20
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_regions_per_server=5
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.chaos_monkey_delay=20000
 

Use this test with "serverKilling" ChaosMonkey. Sample usage:

 hbase org.apache.hadoop.hbase.test.IntegrationTestTimeBoundedRequestsWithRegionReplicas
 -Dhbase.IntegrationTestTimeBoundedRequestsWithRegionReplicas.runtime=600000
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_write_threads=40
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_read_threads=40
 -Dhbase.ipc.client.allowsInterrupt=true --monkey serverKilling
 

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	IntegrationTestTimeBoundedRequestsWithRegionReplicas.TimeBoundedMultiThreadedReader

Field Summary

Fields

Modifier and Type	Field and Description
protected static java.lang.String	CHAOS_MONKEY_DELAY_KEY
protected static long	DEFAUL_CHAOS_MONKEY_DELAY
protected static long	DEFAULT_GET_TIMEOUT
protected static int	DEFAULT_REGION_REPLICATION
protected static java.lang.String	GET_TIMEOUT_KEY

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestIngest

cluster, DEFAULT_NUM_KEYS_PER_SERVER, DEFAULT_NUM_READ_THREADS, DEFAULT_NUM_WRITE_THREADS, DEFAULT_RUN_TIME, HIPHEN, JUNIT_RUN_TIME, LOAD_TEST_TOOL_INIT_ARGS, loadTool, NUM_KEYS_PER_SERVER_KEY, NUM_READ_THREADS_KEY, NUM_WRITE_THREADS_KEY, RUN_TIME_KEY, util

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestBase

CHAOS_MONKEY_PROPS, monkey, MONKEY_LONG_OPT, monkeyProps, monkeyToUse, NO_CLUSTER_CLEANUP_LONG_OPT, noClusterCleanUp

Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

cmdLineArgs, conf, EXIT_FAILURE, EXIT_SUCCESS

Constructor Summary

Constructors

Constructor and Description
IntegrationTestTimeBoundedRequestsWithRegionReplicas()

Method Summary

Modifier and Type	Method and Description
protected java.lang.String[]	getArgsForLoadTestTool(java.lang.String mode, java.lang.String modeSpecificArg, long startKey, long numKeys)
protected MonkeyFactory	getDefaultMonkeyFactory()
static void	main(java.lang.String[] args)
protected void	runIngestTest(long defaultRunTime, long keysPerServerPerIter, int colsPerKey, int recordSize, int writeThreads, int readThreads)
void	setConf(Configuration conf)
protected void	startMonkey()
protected void	writeData(int colsPerKey, int recordSize, int writeThreads, long startKey, long numKeys)

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestIngest

getArgsForLoadTestToolInitTable, getColumnFamilies, getMinServerCount, getNumKeys, getTablename, initTable, internalRunIngestTest, runTestFromCommandLine, setUpCluster, testIngest

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestBase

addOptions, cleanUp, cleanUpCluster, cleanUpMonkey, cleanUpMonkey, doWork, getConf, getTestingUtil, processBaseOptions, processOptions, setUp, setUpMonkey

Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, parseArgs, parseInt, parseLong, printUsage, printUsage, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_GET_TIMEOUT

protected static final long DEFAULT_GET_TIMEOUT

See Also:

Constant Field Values

GET_TIMEOUT_KEY

protected static final java.lang.String GET_TIMEOUT_KEY

See Also:

Constant Field Values

DEFAUL_CHAOS_MONKEY_DELAY

protected static final long DEFAUL_CHAOS_MONKEY_DELAY

See Also:

Constant Field Values

CHAOS_MONKEY_DELAY_KEY

protected static final java.lang.String CHAOS_MONKEY_DELAY_KEY

See Also:

Constant Field Values

DEFAULT_REGION_REPLICATION

protected static final int DEFAULT_REGION_REPLICATION

See Also:

Constant Field Values

Constructor Detail

IntegrationTestTimeBoundedRequestsWithRegionReplicas

public IntegrationTestTimeBoundedRequestsWithRegionReplicas()

Method Detail

startMonkey

protected void startMonkey()
                    throws java.lang.Exception

Overrides:

startMonkey in class IntegrationTestBase

Throws:

java.lang.Exception

getDefaultMonkeyFactory

protected MonkeyFactory getDefaultMonkeyFactory()

Overrides:

getDefaultMonkeyFactory in class IntegrationTestBase

setConf

public void setConf(Configuration conf)

Overrides:

setConf in class AbstractHBaseTool

writeData

protected void writeData(int colsPerKey,
                         int recordSize,
                         int writeThreads,
                         long startKey,
                         long numKeys)
                  throws java.io.IOException

Throws:

java.io.IOException

runIngestTest

protected void runIngestTest(long defaultRunTime,
                             long keysPerServerPerIter,
                             int colsPerKey,
                             int recordSize,
                             int writeThreads,
                             int readThreads)
                      throws java.lang.Exception

Overrides:

runIngestTest in class IntegrationTestIngest

Throws:

java.lang.Exception

getArgsForLoadTestTool

protected java.lang.String[] getArgsForLoadTestTool(java.lang.String mode,
                                                    java.lang.String modeSpecificArg,
                                                    long startKey,
                                                    long numKeys)

Overrides:

getArgsForLoadTestTool in class IntegrationTestIngest

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Throws:

java.lang.Exception