org.apache.hadoop.hbase.regionserver

Class HMobStore

java.lang.Object

org.apache.hadoop.hbase.regionserver.HStore

org.apache.hadoop.hbase.regionserver.HMobStore

All Implemented Interfaces:

ConfigurationObserver, PropagatingConfigurationObserver, HeapSize, Store, StoreConfigInformation

@InterfaceAudience.Private
public class HMobStore
extends HStore

The store implementation to save MOBs (medium objects), it extends the HStore. When a descriptor of a column family has the value "IS_MOB", it means this column family is a mob one. When a HRegion instantiate a store for this column family, the HMobStore is created. HMobStore is almost the same with the HStore except using different types of scanners. In the method of getScanner, the MobStoreScanner and MobReversedStoreScanner are returned. In these scanners, a additional seeks in the mob files should be performed after the seek to HBase is done. The store implements how we save MOBs by extending HStore. When a descriptor of a column family has the value "IS_MOB", it means this column family is a mob one. When a HRegion instantiate a store for this column family, the HMobStore is created. HMobStore is almost the same with the HStore except using different types of scanners. In the method of getScanner, the MobStoreScanner and MobReversedStoreScanner are returned. In these scanners, a additional seeks in the mob files should be performed after the seek in HBase is done.

Field Summary

Fields inherited from class org.apache.hadoop.hbase.regionserver.HStore

BLOCKING_STOREFILES_KEY, blocksize, bytesPerChecksum, cacheConf, checksumType, COMPACTCHECKER_INTERVAL_MULTIPLIER_KEY, conf, cryptoContext, DEEP_OVERHEAD, DEFAULT_BLOCKING_STOREFILE_COUNT, DEFAULT_COMPACTCHECKER_INTERVAL_MULTIPLIER, FIXED_OVERHEAD, memstore, region

Fields inherited from interface org.apache.hadoop.hbase.regionserver.Store

NO_PRIORITY, PRIORITY_USER

Constructor Summary

Constructors

Constructor and Description
HMobStore(HRegion region, HColumnDescriptor family, Configuration confParam)

Method Summary

Modifier and Type	Method and Description
void	commitFile(Path sourceFile, Path targetPath) Commits the mob file.
List<StoreFile>	compact(CompactionContext compaction, CompactionThroughputController throughputController) The compaction in the store of mob.
protected void	createCacheConf(HColumnDescriptor family) Creates the mob cache config.
StoreFile.Writer	createDelFileWriterInTmp(Date date, long maxKeyCount, Compression.Algorithm compression, byte[] startKey) Creates the writer for the del file in temp directory.
protected KeyValueScanner	createScanner(Scan scan, NavigableSet<byte[]> targetCols, long readPt, KeyValueScanner scanner) Gets the MobStoreScanner or MobReversedStoreScanner.
protected StoreEngine<?,?,?,?>	createStoreEngine(Store store, Configuration conf, KeyValue.KVComparator cellComparator) Creates the mob store engine.
StoreFile.Writer	createWriterInTmp(Date date, long maxKeyCount, Compression.Algorithm compression, byte[] startKey) Creates the writer for the mob file in temp directory.
StoreFile.Writer	createWriterInTmp(MobFileName mobFileName, Path basePath, long maxKeyCount, Compression.Algorithm compression) Creates the writer for the mob file in temp directory.
StoreFile.Writer	createWriterInTmp(String date, Path basePath, long maxKeyCount, Compression.Algorithm compression, byte[] startKey) Creates the writer for the mob file in temp directory.
long	getCellsCountCompactedFromMob()
long	getCellsCountCompactedToMob()
long	getCellsSizeCompactedFromMob()
long	getCellsSizeCompactedToMob()
Configuration	getConfiguration() Gets current config.
long	getMobFlushCount()
long	getMobFlushedCellsCount()
long	getMobFlushedCellsSize()
long	getMobScanCellsCount()
long	getMobScanCellsSize()
Path	getPath() Gets the mob file path.
Cell	resolve(Cell reference, boolean cacheBlocks) Reads the cell from the mob file, and the read point does not count.
Cell	resolve(Cell reference, boolean cacheBlocks, long readPt, boolean readEmptyValueOnMobCellMiss) Reads the cell from the mob file.
void	updateCellsCountCompactedFromMob(long count)
void	updateCellsCountCompactedToMob(long count)
void	updateCellsSizeCompactedFromMob(long size)
void	updateCellsSizeCompactedToMob(long size)
void	updateMobFlushCount()
void	updateMobFlushedCellsCount(long count)
void	updateMobFlushedCellsSize(long size)
void	updateMobScanCellsCount(long count)
void	updateMobScanCellsSize(long size)

Methods inherited from class org.apache.hadoop.hbase.regionserver.HStore

add, addChangedReaderObserver, areWritesEnabled, assertBulkLoadHFileOk, bulkLoadHFile, bulkLoadHFile, cancelRequestedCompaction, canSplit, close, compact, compactRecentForTestingAssumingDefaultPolicy, completeCompaction, completeCompaction, createFlushContext, createWriterInTmp, delete, deleteChangedReaderObserver, deregisterChildren, determineTTLFromFamily, flushCache, getAvgStoreFileAge, getBlockingFileCount, getBytesPerChecksum, getCacheConfig, getChecksumType, getCloseCheckInterval, getColumnFamilyName, getCompactedCellsCount, getCompactedCellsSize, getCompactionCheckMultiplier, getCompactionPressure, getCompactionProgress, getCompactPriority, getComparator, getCoprocessorHost, getDataBlockEncoder, getFamily, getFileSystem, getFlushableSize, getFlushedCellsCount, getFlushedCellsSize, getFlushedOutputFileSize, getHRegion, getLastCompactSize, getMajorCompactedCellsCount, getMajorCompactedCellsSize, getMaxMemstoreTS, getMaxSequenceId, getMaxStoreFileAge, getMemstoreFlushSize, getMemStoreSize, getMinStoreFileAge, getNumHFiles, getNumReferenceFiles, getOffPeakHours, getRegionFileSystem, getRegionInfo, getRowKeyAtOrBefore, getScanInfo, getScanner, getScanners, getSize, getSmallestReadPoint, getSnapshotSize, getSplitPoint, getStoreEngine, getStorefiles, getStorefilesCount, getStorefilesIndexSize, getStorefilesSize, getStoreFileTtl, getStoreHomedir, getStoreHomedir, getStoreSizeUncompressed, getTableName, getTotalStaticBloomSize, getTotalStaticIndexSize, hasReferences, hasTooManyStoreFiles, heapSize, isMajorCompaction, isPrimaryReplicaStore, needsCompaction, onConfigurationChange, preBulkLoadHFile, refreshStoreFiles, refreshStoreFiles, registerChildren, replayCompactionMarker, requestCompaction, requestCompaction, requestCompaction, rollback, throttleCompaction, timeOfOldestEdit, toString, triggerMajorCompaction, updateColumnValue, upsert

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

HMobStore

public HMobStore(HRegion region,
                 HColumnDescriptor family,
                 Configuration confParam)
          throws IOException

Throws:

IOException

Method Detail

createCacheConf

protected void createCacheConf(HColumnDescriptor family)

Creates the mob cache config.

Overrides:

createCacheConf in class HStore

Parameters:

family - The current column family.

getConfiguration

public Configuration getConfiguration()

Gets current config.

createScanner

protected KeyValueScanner createScanner(Scan scan,
                                        NavigableSet<byte[]> targetCols,
                                        long readPt,
                                        KeyValueScanner scanner)
                                 throws IOException

Gets the MobStoreScanner or MobReversedStoreScanner. In these scanners, a additional seeks in the mob files should be performed after the seek in HBase is done.

Overrides:

createScanner in class HStore

Throws:

IOException

createStoreEngine

protected StoreEngine<?,?,?,?> createStoreEngine(Store store,
                                                 Configuration conf,
                                                 KeyValue.KVComparator cellComparator)
                                          throws IOException

Creates the mob store engine.

Overrides:

createStoreEngine in class HStore

Parameters:

store - The store. An unfortunate dependency needed due to it being passed to coprocessors via the compactor.

conf - Store configuration.

cellComparator - KVComparator for storeFileManager.

Returns:

StoreEngine to use.

Throws:

IOException

createWriterInTmp

public StoreFile.Writer createWriterInTmp(Date date,
                                          long maxKeyCount,
                                          Compression.Algorithm compression,
                                          byte[] startKey)
                                   throws IOException

Creates the writer for the mob file in temp directory.

Parameters:

date - The latest date of written cells.

maxKeyCount - The key count.

compression - The compression algorithm.

startKey - The start key.

Returns:

The writer for the mob file.

Throws:

IOException

createDelFileWriterInTmp

public StoreFile.Writer createDelFileWriterInTmp(Date date,
                                                 long maxKeyCount,
                                                 Compression.Algorithm compression,
                                                 byte[] startKey)
                                          throws IOException

Creates the writer for the del file in temp directory. The del file keeps tracking the delete markers. Its name has a suffix _del, the format is [0-9a-f]+(_del)?.

Parameters:

date - The latest date of written cells.

maxKeyCount - The key count.

compression - The compression algorithm.

startKey - The start key.

Returns:

The writer for the del file.

Throws:

IOException

createWriterInTmp

public StoreFile.Writer createWriterInTmp(String date,
                                          Path basePath,
                                          long maxKeyCount,
                                          Compression.Algorithm compression,
                                          byte[] startKey)
                                   throws IOException

Creates the writer for the mob file in temp directory.

Parameters:

date - The date string, its format is yyyymmmdd.

basePath - The basic path for a temp directory.

maxKeyCount - The key count.

compression - The compression algorithm.

startKey - The start key.

Returns:

The writer for the mob file.

Throws:

IOException

createWriterInTmp

public StoreFile.Writer createWriterInTmp(MobFileName mobFileName,
                                          Path basePath,
                                          long maxKeyCount,
                                          Compression.Algorithm compression)
                                   throws IOException

Creates the writer for the mob file in temp directory.

Parameters:

mobFileName - The mob file name.

basePath - The basic path for a temp directory.

maxKeyCount - The key count.

compression - The compression algorithm.

Returns:

The writer for the mob file.

Throws:

IOException

commitFile

public void commitFile(Path sourceFile,
                       Path targetPath)
                throws IOException

Commits the mob file.

Parameters:

sourceFile - The source file.

targetPath - The directory path where the source file is renamed to.

Throws:

IOException

resolve

public Cell resolve(Cell reference,
                    boolean cacheBlocks)
             throws IOException

Reads the cell from the mob file, and the read point does not count. This is used for DefaultMobStoreCompactor where we can read empty value for the missing cell.

Parameters:

reference - The cell found in the HBase, its value is a path to a mob file.

cacheBlocks - Whether the scanner should cache blocks.

Returns:

The cell found in the mob file.

Throws:

IOException

resolve

public Cell resolve(Cell reference,
                    boolean cacheBlocks,
                    long readPt,
                    boolean readEmptyValueOnMobCellMiss)
             throws IOException

Reads the cell from the mob file.

Parameters:

reference - The cell found in the HBase, its value is a path to a mob file.

cacheBlocks - Whether the scanner should cache blocks.

readPt - the read point.

readEmptyValueOnMobCellMiss - Whether return null value when the mob file is missing or corrupt.

Returns:

The cell found in the mob file.

Throws:

IOException

getPath

public Path getPath()

Gets the mob file path.

Returns:

The mob file path.

compact

public List<StoreFile> compact(CompactionContext compaction,
                               CompactionThroughputController throughputController)
                        throws IOException

The compaction in the store of mob. The cells in this store contains the path of the mob files. There might be race condition between the major compaction and the sweeping in mob files. In order to avoid this, we need mutually exclude the running of the major compaction and sweeping in mob files. The minor compaction is not affected. The major compaction is marked as retainDeleteMarkers when a sweeping is in progress.

Specified by:

compact in interface Store

Overrides:

compact in class HStore

Parameters:

compaction - compaction details obtained from requestCompaction()

Returns:

Storefile we compacted into or null if we failed or opted out early.

Throws:

IOException

updateCellsCountCompactedToMob

public void updateCellsCountCompactedToMob(long count)

getCellsCountCompactedToMob

public long getCellsCountCompactedToMob()

updateCellsCountCompactedFromMob

public void updateCellsCountCompactedFromMob(long count)

getCellsCountCompactedFromMob

public long getCellsCountCompactedFromMob()

updateCellsSizeCompactedToMob

public void updateCellsSizeCompactedToMob(long size)

getCellsSizeCompactedToMob

public long getCellsSizeCompactedToMob()

updateCellsSizeCompactedFromMob

public void updateCellsSizeCompactedFromMob(long size)

getCellsSizeCompactedFromMob

public long getCellsSizeCompactedFromMob()

updateMobFlushCount

public void updateMobFlushCount()

getMobFlushCount

public long getMobFlushCount()

updateMobFlushedCellsCount

public void updateMobFlushedCellsCount(long count)

getMobFlushedCellsCount

public long getMobFlushedCellsCount()

updateMobFlushedCellsSize

public void updateMobFlushedCellsSize(long size)

getMobFlushedCellsSize

public long getMobFlushedCellsSize()

updateMobScanCellsCount

public void updateMobScanCellsCount(long count)

getMobScanCellsCount

public long getMobScanCellsCount()

updateMobScanCellsSize

public void updateMobScanCellsSize(long size)

getMobScanCellsSize

public long getMobScanCellsSize()