org.apache.hadoop.hbase.mapreduce

Class TestHFileOutputFormat

java.lang.Object

org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

public class TestHFileOutputFormat
extends java.lang.Object

Simple test for KeyValueSortReducer and HFileOutputFormat. Sets up and runs a mapreduce job that writes hfile output. Creates a few inner classes to implement splits and an inputformat that emits keys and values like those of PerformanceEvaluation.

Constructor Summary

Constructors

Constructor and Description
TestHFileOutputFormat()

Method Summary

Modifier and Type	Method and Description
static void	main(java.lang.String[] args)
void	manualTest(java.lang.String[] args)
void	test_LATEST_TIMESTAMP_isReplaced() Test that HFileOutputFormat RecordWriter amends timestamps if passed a keyvalue whose timestamp is HConstants.LATEST_TIMESTAMP.
void	test_TIMERANGE()
void	testColumnFamilySettings() Test that HFileOutputFormat RecordWriter uses compression and bloom filter settings from the column family descriptor
void	testExcludeAllFromMinorCompaction() This test is to test the scenario happened in HBASE-6901.
void	testExcludeMinorCompaction()
void	testJobConfiguration()
void	testMRIncrementalLoad()
void	testMRIncrementalLoadWithSplit()
void	testSerializeDeserializeFamilyBlockSizeMap() Test for HFileOutputFormat.configureBlockSize(org.apache.hadoop.hbase.client.Table, Configuration) and (Configuration).
void	testSerializeDeserializeFamilyBloomTypeMap() Test for HFileOutputFormat.configureBloomType(org.apache.hadoop.hbase.client.Table, Configuration) and (Configuration).
void	testSerializeDeserializeFamilyCompressionMap() Test for HFileOutputFormat.configureCompression(org.apache.hadoop.hbase.client.Table, Configuration) and (Configuration).
void	testSerializeDeserializeFamilyDataBlockEncodingMap() Test for HFileOutputFormat.configureDataBlockEncoding(org.apache.hadoop.hbase.client.Table, Configuration) and (Configuration).
void	testWritingPEData() Run small MR job.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TestHFileOutputFormat

public TestHFileOutputFormat()

Method Detail

test_LATEST_TIMESTAMP_isReplaced

public void test_LATEST_TIMESTAMP_isReplaced()
                                      throws java.lang.Exception

Test that HFileOutputFormat RecordWriter amends timestamps if passed a keyvalue whose timestamp is HConstants.LATEST_TIMESTAMP.

Throws:

java.lang.Exception

See Also:

HBASE-2615

test_TIMERANGE

public void test_TIMERANGE()
                    throws java.lang.Exception

Throws:

java.lang.Exception

testWritingPEData

public void testWritingPEData()
                       throws java.lang.Exception

Run small MR job.

Throws:

java.lang.Exception

testJobConfiguration

public void testJobConfiguration()
                          throws java.lang.Exception

Throws:

java.lang.Exception

testMRIncrementalLoad

public void testMRIncrementalLoad()
                           throws java.lang.Exception

Throws:

java.lang.Exception

testMRIncrementalLoadWithSplit

public void testMRIncrementalLoadWithSplit()
                                    throws java.lang.Exception

Throws:

java.lang.Exception

testSerializeDeserializeFamilyCompressionMap

public void testSerializeDeserializeFamilyCompressionMap()
                                                  throws java.io.IOException

Test for HFileOutputFormat.configureCompression(org.apache.hadoop.hbase.client.Table, Configuration) and (Configuration). Tests that the compression map is correctly serialized into and deserialized from configuration

Throws:

java.io.IOException

testSerializeDeserializeFamilyBloomTypeMap

public void testSerializeDeserializeFamilyBloomTypeMap()
                                                throws java.io.IOException

Test for HFileOutputFormat.configureBloomType(org.apache.hadoop.hbase.client.Table, Configuration) and (Configuration). Tests that the compression map is correctly serialized into and deserialized from configuration

Throws:

java.io.IOException

testSerializeDeserializeFamilyBlockSizeMap

public void testSerializeDeserializeFamilyBlockSizeMap()
                                                throws java.io.IOException

Test for HFileOutputFormat.configureBlockSize(org.apache.hadoop.hbase.client.Table, Configuration) and (Configuration). Tests that the compression map is correctly serialized into and deserialized from configuration

Throws:

java.io.IOException

testSerializeDeserializeFamilyDataBlockEncodingMap

public void testSerializeDeserializeFamilyDataBlockEncodingMap()
                                                        throws java.io.IOException

Test for HFileOutputFormat.configureDataBlockEncoding(org.apache.hadoop.hbase.client.Table, Configuration) and (Configuration). Tests that the compression map is correctly serialized into and deserialized from configuration

Throws:

java.io.IOException

testColumnFamilySettings

public void testColumnFamilySettings()
                              throws java.lang.Exception

Test that HFileOutputFormat RecordWriter uses compression and bloom filter settings from the column family descriptor

Throws:

java.lang.Exception

testExcludeAllFromMinorCompaction

public void testExcludeAllFromMinorCompaction()
                                       throws java.lang.Exception

This test is to test the scenario happened in HBASE-6901. All files are bulk loaded and excluded from minor compaction. Without the fix of HBASE-6901, an ArrayIndexOutOfBoundsException will be thrown.

Throws:

java.lang.Exception

testExcludeMinorCompaction

public void testExcludeMinorCompaction()
                                throws java.lang.Exception

Throws:

java.lang.Exception

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Throws:

java.lang.Exception

manualTest

public void manualTest(java.lang.String[] args)
                throws java.lang.Exception

Throws:

java.lang.Exception