org.apache.hadoop.hbase

Class TestPartialResultsFromClientSide

java.lang.Object

org.apache.hadoop.hbase.TestPartialResultsFromClientSide

public class TestPartialResultsFromClientSide
extends java.lang.Object

These tests are focused on testing how partial results appear to a client. Partial results are Results that contain only a portion of a row's complete list of cells. Partial results are formed when the server breaches its maximum result size when trying to service a client's RPC request. It is the responsibility of the scanner on the client side to recognize when partial results have been returned and to take action to form the complete results.

Unless the flag Scan#setAllowPartialResults(boolean) has been set to true, the caller of ResultScanner#next() should never see partial results.

Constructor Summary

Constructors

Constructor and Description
TestPartialResultsFromClientSide()

Method Summary

Modifier and Type	Method and Description
static void	setUpBeforeClass()
static void	tearDownAfterClass()
void	testAllowPartialResults() Ensure that we only see Results marked as partial when the allowPartial flag is set
void	testBatchingResultWhenRegionMove()
void	testCompleteResultWhenRegionMove()
void	testEquivalenceOfScanResults() Ensure that the results returned from a scanner that retrieves all results in a single RPC call matches the results that are returned from a scanner that must incrementally combine partial results into complete results.
void	testEquivalenceOfScanResults(Table table, Scan scan1, Scan scan2)
void	testExceptionThrownOnMismatchedPartialResults() When reconstructing the complete result from its partials we ensure that the row of each partial result is the same.
void	testExpectedNumberOfCellsPerPartialResult() Setting the max result size allows us to control how many cells we expect to see on each call to next on the scanner.
void	testExpectedNumberOfCellsPerPartialResult(Scan baseScan)
void	testExpectedNumberOfCellsPerPartialResult(Scan baseScan, int expectedNumberOfCells)
void	testExpectedValuesOfPartialResults() Ensure that the expected key values appear in a result returned from a scanner that is combining partial results into complete results
void	testExpectedValuesOfPartialResults(boolean reversed)
void	testNoPartialResultsWhenRowFilterPresent() When a scan has a filter where org.apache.hadoop.hbase.filter.Filter#hasFilterRow() is true, the scanner should not return partial results.
void	testOrderingOfCellsInPartialResults() Order of cells in partial results matches the ordering of cells from complete results
void	testOrderingOfCellsInPartialResults(Scan basePartialScan)
void	testPartialResultsAndBatch() Test various combinations of batching and partial results for correctness
void	testPartialResultsAndBatch(int batch, int cellsPerPartialResult)
void	testPartialResultsAndCaching() Examine the interaction between the maxResultSize and caching.
void	testPartialResultsAndCaching(int resultSizeRowLimit, int cachingRowLimit)
void	testPartialResultsReassembly() Test the method Result#createCompleteResult(List)
void	testPartialResultsReassembly(Scan scanBase)
void	testPartialResultsWithColumnFilter() Test partial Result re-assembly in the presence of different filters.
void	testPartialResultsWithColumnFilter(Filter filter)
void	testPartialResultWhenRegionMove()
void	testReadPointAndPartialResults()
void	testReversedCompleteResultWhenRegionMove()
void	testReversedPartialResultWhenRegionMove()
void	testSmallScansDoNotAllowPartials() Small scans should not return partial results because it would prevent small scans from retrieving all of the necessary results in a single RPC request which is what makese small scans useful.
void	testSmallScansDoNotAllowPartials(Scan baseScan)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TestPartialResultsFromClientSide

public TestPartialResultsFromClientSide()

Method Detail

setUpBeforeClass

public static void setUpBeforeClass()
                             throws java.lang.Exception

Throws:

java.lang.Exception

tearDownAfterClass

public static void tearDownAfterClass()
                               throws java.lang.Exception

Throws:

java.lang.Exception

testExpectedValuesOfPartialResults

public void testExpectedValuesOfPartialResults()
                                        throws java.lang.Exception

Ensure that the expected key values appear in a result returned from a scanner that is combining partial results into complete results

Throws:

java.lang.Exception

testExpectedValuesOfPartialResults

public void testExpectedValuesOfPartialResults(boolean reversed)
                                        throws java.lang.Exception

Throws:

java.lang.Exception

testAllowPartialResults

public void testAllowPartialResults()
                             throws java.lang.Exception

Ensure that we only see Results marked as partial when the allowPartial flag is set

Throws:

java.lang.Exception

testEquivalenceOfScanResults

public void testEquivalenceOfScanResults()
                                  throws java.lang.Exception

Ensure that the results returned from a scanner that retrieves all results in a single RPC call matches the results that are returned from a scanner that must incrementally combine partial results into complete results. A variety of scan configurations can be tested

Throws:

java.lang.Exception

testEquivalenceOfScanResults

public void testEquivalenceOfScanResults(Table table,
                                         Scan scan1,
                                         Scan scan2)
                                  throws java.lang.Exception

Throws:

java.lang.Exception

testOrderingOfCellsInPartialResults

public void testOrderingOfCellsInPartialResults()
                                         throws java.lang.Exception

Order of cells in partial results matches the ordering of cells from complete results

Throws:

java.lang.Exception

testOrderingOfCellsInPartialResults

public void testOrderingOfCellsInPartialResults(Scan basePartialScan)
                                         throws java.lang.Exception

Throws:

java.lang.Exception

testExpectedNumberOfCellsPerPartialResult

public void testExpectedNumberOfCellsPerPartialResult()
                                               throws java.lang.Exception

Setting the max result size allows us to control how many cells we expect to see on each call to next on the scanner. Test a variety of different sizes for correctness

Throws:

java.lang.Exception

testExpectedNumberOfCellsPerPartialResult

public void testExpectedNumberOfCellsPerPartialResult(Scan baseScan)
                                               throws java.lang.Exception

Throws:

java.lang.Exception

testExpectedNumberOfCellsPerPartialResult

public void testExpectedNumberOfCellsPerPartialResult(Scan baseScan,
                                                      int expectedNumberOfCells)
                                               throws java.lang.Exception

Throws:

java.lang.Exception

testPartialResultsAndBatch

public void testPartialResultsAndBatch()
                                throws java.lang.Exception

Test various combinations of batching and partial results for correctness

Throws:

java.lang.Exception

testPartialResultsAndBatch

public void testPartialResultsAndBatch(int batch,
                                       int cellsPerPartialResult)
                                throws java.lang.Exception

Throws:

java.lang.Exception

testPartialResultsReassembly

public void testPartialResultsReassembly()
                                  throws java.lang.Exception

Test the method Result#createCompleteResult(List)

Throws:

java.lang.Exception

testPartialResultsReassembly

public void testPartialResultsReassembly(Scan scanBase)
                                  throws java.lang.Exception

Throws:

java.lang.Exception

testExceptionThrownOnMismatchedPartialResults

public void testExceptionThrownOnMismatchedPartialResults()
                                                   throws java.io.IOException

When reconstructing the complete result from its partials we ensure that the row of each partial result is the same. If one of the rows differs, an exception is thrown.

Throws:

java.io.IOException

testNoPartialResultsWhenRowFilterPresent

public void testNoPartialResultsWhenRowFilterPresent()
                                              throws java.lang.Exception

When a scan has a filter where org.apache.hadoop.hbase.filter.Filter#hasFilterRow() is true, the scanner should not return partial results. The scanner cannot return partial results because the entire row needs to be read for the include/exclude decision to be made

Throws:

java.lang.Exception

testPartialResultsAndCaching

public void testPartialResultsAndCaching()
                                  throws java.lang.Exception

Examine the interaction between the maxResultSize and caching. If the caching limit is reached before the maxResultSize limit, we should not see partial results. On the other hand, if the maxResultSize limit is reached before the caching limit, it is likely that partial results will be seen.

Throws:

java.lang.Exception

testPartialResultsAndCaching

public void testPartialResultsAndCaching(int resultSizeRowLimit,
                                         int cachingRowLimit)
                                  throws java.lang.Exception

Parameters:

resultSizeRowLimit - The row limit that will be enforced through maxResultSize

cachingRowLimit - The row limit that will be enforced through caching

Throws:

java.lang.Exception

testSmallScansDoNotAllowPartials

public void testSmallScansDoNotAllowPartials()
                                      throws java.lang.Exception

Small scans should not return partial results because it would prevent small scans from retrieving all of the necessary results in a single RPC request which is what makese small scans useful. Thus, ensure that even when Scan#getAllowPartialResults() is true, small scans do not return partial results

Throws:

java.lang.Exception

testSmallScansDoNotAllowPartials

public void testSmallScansDoNotAllowPartials(Scan baseScan)
                                      throws java.lang.Exception

Throws:

java.lang.Exception

testReadPointAndPartialResults

public void testReadPointAndPartialResults()
                                    throws java.lang.Exception

Throws:

java.lang.Exception

testPartialResultsWithColumnFilter

public void testPartialResultsWithColumnFilter()
                                        throws java.lang.Exception

Test partial Result re-assembly in the presence of different filters. The Results from the partial scanner should match the Results returned from a scanner that receives all of the results in one RPC to the server. The partial scanner is tested with a variety of different result sizes (all of which are less than the size necessary to fetch an entire row)

Throws:

java.lang.Exception

testPartialResultsWithColumnFilter

public void testPartialResultsWithColumnFilter(Filter filter)
                                        throws java.lang.Exception

Throws:

java.lang.Exception

testPartialResultWhenRegionMove

public void testPartialResultWhenRegionMove()
                                     throws java.io.IOException

Throws:

java.io.IOException

testReversedPartialResultWhenRegionMove

public void testReversedPartialResultWhenRegionMove()
                                             throws java.io.IOException

Throws:

java.io.IOException

testCompleteResultWhenRegionMove

public void testCompleteResultWhenRegionMove()
                                      throws java.io.IOException

Throws:

java.io.IOException

testReversedCompleteResultWhenRegionMove

public void testReversedCompleteResultWhenRegionMove()
                                              throws java.io.IOException

Throws:

java.io.IOException

testBatchingResultWhenRegionMove

public void testBatchingResultWhenRegionMove()
                                      throws java.io.IOException

Throws:

java.io.IOException