org.apache.hadoop.hbase.regionserver

Class DataBlockEncodingTool

java.lang.Object

org.apache.hadoop.hbase.regionserver.DataBlockEncodingTool

public class DataBlockEncodingTool
extends Object

Tests various algorithms for key compression on an existing HFile. Useful for testing, debugging and benchmarking.

Constructor Summary

Constructors

Constructor and Description
DataBlockEncodingTool(String compressionAlgorithmName)

Method Summary

Modifier and Type	Method and Description
void	benchmarkAlgorithm(Compression.Algorithm algorithm, String name, byte[] buffer, int offset, int length) Check decompress performance of a given algorithm and print it.
void	benchmarkCodecs() Benchmark codec's speed.
void	checkStatistics(KeyValueScanner scanner, int kvLimit) Check statistics for given HFile for different data block encoders.
void	displayStatistics() Display statistics of different compression algorithms.
static void	main(String[] args) A command line interface to benchmarks.
static void	testCodecs(Configuration conf, int kvLimit, String hfilePath, String compressionName, boolean doBenchmark, boolean doVerify) Test a data block encoder on the given HFile.
boolean	verifyCodecs(KeyValueScanner scanner, int kvLimit) Verify if all data block encoders are working properly.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DataBlockEncodingTool

public DataBlockEncodingTool(String compressionAlgorithmName)

Parameters:

compressionAlgorithmName - What kind of algorithm should be used as baseline for comparison (e.g. lzo, gz).

Method Detail

checkStatistics

public void checkStatistics(KeyValueScanner scanner,
                            int kvLimit)
                     throws IOException

Check statistics for given HFile for different data block encoders.

Parameters:

scanner - Of file which will be compressed.

kvLimit - Maximal count of KeyValue which will be processed.

Throws:

IOException - thrown if scanner is invalid

verifyCodecs

public boolean verifyCodecs(KeyValueScanner scanner,
                            int kvLimit)
                     throws IOException

Verify if all data block encoders are working properly.

Parameters:

scanner - Of file which was compressed.

kvLimit - Maximal count of KeyValue which will be processed.

Returns:

true if all data block encoders compressed/decompressed correctly.

Throws:

IOException - thrown if scanner is invalid

benchmarkCodecs

public void benchmarkCodecs()
                     throws IOException

Benchmark codec's speed.

Throws:

IOException

benchmarkAlgorithm

public void benchmarkAlgorithm(Compression.Algorithm algorithm,
                               String name,
                               byte[] buffer,
                               int offset,
                               int length)
                        throws IOException

Check decompress performance of a given algorithm and print it.

Parameters:

algorithm - Compression algorithm.

name - Name of algorithm.

buffer - Buffer to be compressed.

offset - Position of the beginning of the data.

length - Length of data in buffer.

Throws:

IOException

displayStatistics

public void displayStatistics()
                       throws IOException

Display statistics of different compression algorithms.

Throws:

IOException

testCodecs

public static void testCodecs(Configuration conf,
                              int kvLimit,
                              String hfilePath,
                              String compressionName,
                              boolean doBenchmark,
                              boolean doVerify)
                       throws IOException

Test a data block encoder on the given HFile. Output results to console.

Parameters:

kvLimit - The limit of KeyValue which will be analyzed.

hfilePath - an HFile path on the file system.

compressionName - Compression algorithm used for comparison.

doBenchmark - Run performance benchmarks.

doVerify - Verify correctness.

Throws:

IOException - When pathName is incorrect.

main

public static void main(String[] args)
                 throws IOException

A command line interface to benchmarks. Parses command-line arguments and runs the appropriate benchmarks.

Parameters:

args - Should have length at least 1 and holds the file path to HFile.

Throws:

IOException - If you specified the wrong file.