HStore (Hortonworks Data Platform Apache HBase Java API Reference)

java.lang.Object
- org.apache.hadoop.hbase.regionserver.HStore

All Implemented Interfaces:

ConfigurationObserver, PropagatingConfigurationObserver, HeapSize, Store, StoreConfigInformation

Direct Known Subclasses:

HMobStore
```
@InterfaceAudience.Private
public class HStore
extends java.lang.Object
implements Store
```
A Store holds a column family in a Region. Its a memstore and a set of zero or more StoreFiles, which stretch backwards over time.
There's no reason to consider append-logging at this level; all logging and locking is handled at the HRegion level. Store just provides services to manage sets of StoreFiles. One of the most important of those services is compaction services where files are aggregated once they pass a configurable threshold.
The only thing having to do with logs that Store needs to deal with is the reconstructionLog. This is a segment of an HRegion's log that might NOT be present upon startup. If the param is NULL, there's nothing to do. If the param is non-NULL, we need to process the log to reconstruct a TreeMap that might not have been written to disk before the process died.
It's assumed that after this constructor returns, the reconstructionLog file will be deleted (by whoever has instantiated the Store).
Locking and transactions are handled at a higher level. This API should not be called directly but by an HRegion manager.

Field Summary

Fields
Modifier and Type	Field and Description
`static java.lang.String`	`BLOCKING_STOREFILES_KEY`
`protected int`	`blocksize`
`protected int`	`bytesPerChecksum`
`protected CacheConfig`	`cacheConf`
`protected ChecksumType`	`checksumType` Checksum configuration
`static java.lang.String`	`COMPACTCHECKER_INTERVAL_MULTIPLIER_KEY`
`protected Configuration`	`conf`
`protected Encryption.Context`	`cryptoContext`
`static long`	`DEEP_OVERHEAD`
`static int`	`DEFAULT_BLOCKING_STOREFILE_COUNT`
`static int`	`DEFAULT_COMPACTCHECKER_INTERVAL_MULTIPLIER`
`static long`	`FIXED_OVERHEAD`
`protected MemStore`	`memstore`
`protected HRegion`	`region`

Fields inherited from interface org.apache.hadoop.hbase.regionserver.Store
NO_PRIORITY, PRIORITY_USER

Constructor Summary

Constructors
Modifier Constructor and Description

protected HStore(HRegion region, HColumnDescriptor family, Configuration confParam)
Constructor

Constructors
Modifier	Constructor and Description
`protected`	`HStore(HRegion region, HColumnDescriptor family, Configuration confParam)` Constructor

Method Summary

All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`Pair<java.lang.Long,Cell>`	`add(Cell cell)` Adds a value to the memstore
`void`	`addChangedReaderObserver(ChangedReadersObserver o)`
`boolean`	`areWritesEnabled()`
`void`	`assertBulkLoadHFileOk(Path srcPath)` This throws a WrongRegionException if the HFile does not fit in this region, or an InvalidHFileException if the HFile is not valid.
`Path`	`bulkLoadHFile(byte[] family, java.lang.String srcPathStr, Path dstPath)` This method should only be called from Region.
`void`	`bulkLoadHFile(StoreFileInfo fileInfo)`
`void`	`cancelRequestedCompaction(CompactionContext compaction)`
`boolean`	`canSplit()`
`<any>`	`close()` Close all the readers We don't need to worry about subsequent requests because the Region holds a write lock that will prevent any more reads or writes.
`java.util.List<StoreFile>`	`compact(CompactionContext compaction, CompactionThroughputController throughputController)` Compact the StoreFiles.
`java.util.List<StoreFile>`	`compact(CompactionContext compaction, CompactionThroughputController throughputController, User user)`
`void`	`compactRecentForTestingAssumingDefaultPolicy(int N)` This method tries to compact N recent files for testing.
`protected void`	`completeCompaction(java.util.Collection<StoreFile> compactedFiles)` It works by processing a compaction that's been written to disk.
`protected void`	`completeCompaction(java.util.Collection<StoreFile> compactedFiles, boolean removeFiles)` It works by processing a compaction that's been written to disk.
`protected void`	`createCacheConf(HColumnDescriptor family)` Creates the cache config.
`org.apache.hadoop.hbase.regionserver.StoreFlushContext`	`createFlushContext(long cacheFlushId)`
`protected KeyValueScanner`	`createScanner(Scan scan, java.util.NavigableSet<byte[]> targetCols, long readPt, KeyValueScanner scanner)`
`protected StoreEngine<?,?,?,?>`	`createStoreEngine(Store store, Configuration conf, KeyValue.KVComparator kvComparator)` Creates the store engine configured for the given Store.
`StoreFile.Writer`	`createWriterInTmp(long maxKeyCount, Compression.Algorithm compression, boolean isCompaction, boolean includeMVCCReadpoint, boolean includesTag)`
`protected long`	`delete(KeyValue kv)` Adds a value to the memstore
`void`	`deleteChangedReaderObserver(ChangedReadersObserver o)`
`void`	`deregisterChildren(ConfigurationManager manager)` Needs to be called to deregister the children from the manager.
`static long`	`determineTTLFromFamily(HColumnDescriptor family)`
`protected java.util.List<Path>`	`flushCache(long logCacheFlushId, MemStoreSnapshot snapshot, MonitoredTask status)` Write out current snapshot.
`long`	`getAvgStoreFileAge()`
`long`	`getBlockingFileCount()` The number of files required before flushes for this store will be blocked.
`static int`	`getBytesPerChecksum(Configuration conf)` Returns the configured bytesPerChecksum value.
`CacheConfig`	`getCacheConfig()` Used for tests.
`static ChecksumType`	`getChecksumType(Configuration conf)` Returns the configured checksum algorithm.
`static int`	`getCloseCheckInterval()`
`java.lang.String`	`getColumnFamilyName()`
`long`	`getCompactedCellsCount()`
`long`	`getCompactedCellsSize()`
`long`	`getCompactionCheckMultiplier()`
`double`	`getCompactionPressure()` This value can represent the degree of emergency of compaction for this store.
`CompactionProgress`	`getCompactionProgress()` getter for CompactionProgress object
`int`	`getCompactPriority()`
`KeyValue.KVComparator`	`getComparator()`
`RegionCoprocessorHost`	`getCoprocessorHost()`
`HFileDataBlockEncoder`	`getDataBlockEncoder()`
`HColumnDescriptor`	`getFamily()`
`FileSystem`	`getFileSystem()`
`long`	`getFlushableSize()`
`long`	`getFlushedCellsCount()`
`long`	`getFlushedCellsSize()`
`long`	`getFlushedOutputFileSize()`
`HRegion`	`getHRegion()`
`long`	`getLastCompactSize()`
`long`	`getMajorCompactedCellsCount()`
`long`	`getMajorCompactedCellsSize()`
`long`	`getMaxMemstoreTS()`
`long`	`getMaxSequenceId()`
`long`	`getMaxStoreFileAge()`
`long`	`getMemstoreFlushSize()`
`long`	`getMemStoreSize()`
`long`	`getMinStoreFileAge()`
`long`	`getNumHFiles()`
`long`	`getNumReferenceFiles()`
`protected OffPeakHours`	`getOffPeakHours()`
`HRegionFileSystem`	`getRegionFileSystem()`
`HRegionInfo`	`getRegionInfo()`
`Cell`	`getRowKeyAtOrBefore(byte[] row)` Find the key that matches row exactly, or the one that immediately precedes it.
`ScanInfo`	`getScanInfo()`
`KeyValueScanner`	`getScanner(Scan scan, java.util.NavigableSet<byte[]> targetCols, long readPt)` Return a scanner for both the memstore and the HStore files.
`java.util.List<KeyValueScanner>`	`getScanners(boolean cacheBlocks, boolean isGet, boolean usePread, boolean isCompaction, ScanQueryMatcher matcher, byte[] startRow, byte[] stopRow, long readPt)` Get all scanners with no filtering based on TTL (that happens further down the line).
`long`	`getSize()`
`long`	`getSmallestReadPoint()`
`long`	`getSnapshotSize()` Returns the memstore snapshot size
`byte[]`	`getSplitPoint()` Determines if Store should be split
`StoreEngine<?,?,?,?>`	`getStoreEngine()` Returns the StoreEngine that is backing this concrete implementation of Store.
`java.util.Collection<StoreFile>`	`getStorefiles()`
`int`	`getStorefilesCount()`
`long`	`getStorefilesIndexSize()`
`long`	`getStorefilesSize()`
`long`	`getStoreFileTtl()`
`static Path`	`getStoreHomedir(Path tabledir, HRegionInfo hri, byte[] family)` Deprecated.
`static Path`	`getStoreHomedir(Path tabledir, java.lang.String encodedName, byte[] family)` Deprecated.
`long`	`getStoreSizeUncompressed()`
`TableName`	`getTableName()`
`long`	`getTotalStaticBloomSize()` Returns the total byte size of all Bloom filter bit arrays.
`long`	`getTotalStaticIndexSize()` Returns the total size of all index blocks in the data block indexes, including the root level, intermediate levels, and the leaf level for multi-level indexes, or just the root level for single-level indexes.
`boolean`	`hasReferences()`
`boolean`	`hasTooManyStoreFiles()`
`long`	`heapSize()`
`boolean`	`isMajorCompaction()`
`boolean`	`isPrimaryReplicaStore()`
`boolean`	`needsCompaction()` See if there's too much store files in this store
`void`	`onConfigurationChange(Configuration conf)` This method would be called by the `ConfigurationManager` object when the `Configuration` object is reloaded from disk.
`Pair<Path,Path>`	`preBulkLoadHFile(java.lang.String srcPathStr, long seqNum)`
`void`	`refreshStoreFiles()` Checks the underlying store files, and opens the files that have not been opened, and removes the store file readers for store files no longer available.
`void`	`refreshStoreFiles(java.util.Collection<java.lang.String> newFiles)` Replaces the store files that the store has with the given files.
`void`	`registerChildren(ConfigurationManager manager)` Needs to be called to register the children to the manager.
`void`	`replayCompactionMarker(WALProtos.CompactionDescriptor compaction, boolean pickCompactionFiles, boolean removeFiles)` Call to complete a compaction.
`CompactionContext`	`requestCompaction()`
`CompactionContext`	`requestCompaction(int priority, CompactionRequest baseRequest)`
`CompactionContext`	`requestCompaction(int priority, CompactionRequest baseRequest, User user)`
`void`	`rollback(Cell cell)` Removes a Cell from the memstore.
`boolean`	`throttleCompaction(long compactionSize)`
`long`	`timeOfOldestEdit()` When was the last edit done in the memstore
`java.lang.String`	`toString()`
`void`	`triggerMajorCompaction()`
`long`	`updateColumnValue(byte[] row, byte[] f, byte[] qualifier, long newValue)` Used in tests.
`long`	`upsert(java.lang.Iterable<Cell> cells, long readpoint)` Adds or replaces the specified KeyValues.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - COMPACTCHECKER_INTERVAL_MULTIPLIER_KEY
```
public static final java.lang.String COMPACTCHECKER_INTERVAL_MULTIPLIER_KEY
```
    See Also:
    
    Constant Field Values
  - BLOCKING_STOREFILES_KEY
```
public static final java.lang.String BLOCKING_STOREFILES_KEY
```
    See Also:
    
    Constant Field Values
  - DEFAULT_COMPACTCHECKER_INTERVAL_MULTIPLIER
```
public static final int DEFAULT_COMPACTCHECKER_INTERVAL_MULTIPLIER
```
    See Also:
    
    Constant Field Values
  - DEFAULT_BLOCKING_STOREFILE_COUNT
```
public static final int DEFAULT_BLOCKING_STOREFILE_COUNT
```
    See Also:
    
    Constant Field Values
  - memstore
```
protected final MemStore memstore
```
  - region
```
protected final HRegion region
```
  - conf
```
protected Configuration conf
```
  - cacheConf
```
protected CacheConfig cacheConf
```
  - blocksize
```
protected final int blocksize
```
  - checksumType
```
protected ChecksumType checksumType
```
    Checksum configuration
  - bytesPerChecksum
```
protected int bytesPerChecksum
```
  - cryptoContext
```
protected Encryption.Context cryptoContext
```
  - FIXED_OVERHEAD
```
public static final long FIXED_OVERHEAD
```
  - DEEP_OVERHEAD
```
public static final long DEEP_OVERHEAD
```
- Constructor Detail
  - HStore
```
protected HStore(HRegion region,
                 HColumnDescriptor family,
                 Configuration confParam)
          throws java.io.IOException
```
    Constructor
    
    Parameters:
    
    region -
    
    family - HColumnDescriptor for this column
    
    confParam - configuration object failed. Can be null.
    
    Throws:
    
    java.io.IOException
- Method Detail
  - createCacheConf
```
protected void createCacheConf(HColumnDescriptor family)
```
    Creates the cache config.
    
    Parameters:
    
    family - The current column family.
  - createStoreEngine
```
protected StoreEngine<?,?,?,?> createStoreEngine(Store store,
                                                 Configuration conf,
                                                 KeyValue.KVComparator kvComparator)
                                          throws java.io.IOException
```
    Creates the store engine configured for the given Store.
    
    Parameters:
    
    store - The store. An unfortunate dependency needed due to it being passed to coprocessors via the compactor.
    
    conf - Store configuration.
    
    kvComparator - KVComparator for storeFileManager.
    
    Returns:
    
    StoreEngine to use.
    
    Throws:
    
    java.io.IOException
  - determineTTLFromFamily
```
public static long determineTTLFromFamily(HColumnDescriptor family)
```
    Parameters:
    
    family -
    
    Returns:
    
    TTL in seconds of the specified family
  - getColumnFamilyName
```
public java.lang.String getColumnFamilyName()
```
    Specified by:
    
    getColumnFamilyName in interface Store
  - getTableName
```
public TableName getTableName()
```
    Specified by:
    
    getTableName in interface Store
  - getFileSystem
```
public FileSystem getFileSystem()
```
    Specified by:
    
    getFileSystem in interface Store
  - getRegionFileSystem
```
public HRegionFileSystem getRegionFileSystem()
```
  - getStoreFileTtl
```
public long getStoreFileTtl()
```
    Specified by:
    
    getStoreFileTtl in interface StoreConfigInformation
    
    Returns:
    
    Gets the cf-specific time-to-live for store files.
  - getMemstoreFlushSize
```
public long getMemstoreFlushSize()
```
    Specified by:
    
    getMemstoreFlushSize in interface StoreConfigInformation
    
    Returns:
    
    Gets the Memstore flush size for the region that this store works with.
  - getFlushableSize
```
public long getFlushableSize()
```
    Specified by:
    
    getFlushableSize in interface Store
    
    Returns:
    
    The amount of memory we could flush from this memstore; usually this is equal to Store.getMemStoreSize() unless we are carrying snapshots and then it will be the size of outstanding snapshots.
  - getSnapshotSize
```
public long getSnapshotSize()
```
    Description copied from interface: Store
    
    Returns the memstore snapshot size
    
    Specified by:
    
    getSnapshotSize in interface Store
    
    Returns:
    
    size of the memstore snapshot
  - getCompactionCheckMultiplier
```
public long getCompactionCheckMultiplier()
```
    Specified by:
    
    getCompactionCheckMultiplier in interface StoreConfigInformation
    
    Returns:
    
    Gets the cf-specific compaction check frequency multiplier. The need for compaction (outside of normal checks during flush, open, etc.) will be ascertained every multiplier * HConstants.THREAD_WAKE_FREQUENCY milliseconds.
  - getBlockingFileCount
```
public long getBlockingFileCount()
```
    Description copied from interface: StoreConfigInformation
    
    The number of files required before flushes for this store will be blocked.
    
    Specified by:
    
    getBlockingFileCount in interface StoreConfigInformation
  - getBytesPerChecksum
```
public static int getBytesPerChecksum(Configuration conf)
```
    Returns the configured bytesPerChecksum value.
    
    Parameters:
    
    conf - The configuration
    
    Returns:
    
    The bytesPerChecksum that is set in the configuration
  - getChecksumType
```
public static ChecksumType getChecksumType(Configuration conf)
```
    Returns the configured checksum algorithm.
    
    Parameters:
    
    conf - The configuration
    
    Returns:
    
    The checksum algorithm that is set in the configuration
  - getCloseCheckInterval
```
public static int getCloseCheckInterval()
```
    Returns:
    
    how many bytes to write between status checks
  - getFamily
```
public HColumnDescriptor getFamily()
```
    Specified by:
    
    getFamily in interface Store
  - getMaxSequenceId
```
public long getMaxSequenceId()
```
    Specified by:
    
    getMaxSequenceId in interface Store
    
    Returns:
    
    The maximum sequence id in all store files. Used for log replay.
  - getMaxMemstoreTS
```
public long getMaxMemstoreTS()
```
    Specified by:
    
    getMaxMemstoreTS in interface Store
    
    Returns:
    
    The maximum memstoreTS in all store files.
  - getStoreHomedir
```
@Deprecated
public static Path getStoreHomedir(Path tabledir,
                                               HRegionInfo hri,
                                               byte[] family)
```
    Deprecated.
    
    Parameters:
    
    tabledir - Path to where the table is being stored
    
    hri - HRegionInfo for the region.
    
    family - HColumnDescriptor describing the column family
    
    Returns:
    
    Path to family/Store home directory.
  - getStoreHomedir
```
@Deprecated
public static Path getStoreHomedir(Path tabledir,
                                               java.lang.String encodedName,
                                               byte[] family)
```
    Deprecated.
    
    Parameters:
    
    tabledir - Path to where the table is being stored
    
    encodedName - Encoded region name.
    
    family - HColumnDescriptor describing the column family
    
    Returns:
    
    Path to family/Store home directory.
  - getDataBlockEncoder
```
public HFileDataBlockEncoder getDataBlockEncoder()
```
    Specified by:
    
    getDataBlockEncoder in interface Store
    
    Returns:
    
    the data block encoder
  - refreshStoreFiles
```
public void refreshStoreFiles()
                       throws java.io.IOException
```
    Checks the underlying store files, and opens the files that have not been opened, and removes the store file readers for store files no longer available. Mainly used by secondary region replicas to keep up to date with the primary region files.
    
    Specified by:
    
    refreshStoreFiles in interface Store
    
    Throws:
    
    java.io.IOException
  - refreshStoreFiles
```
public void refreshStoreFiles(java.util.Collection<java.lang.String> newFiles)
                       throws java.io.IOException
```
    Description copied from interface: Store
    
    Replaces the store files that the store has with the given files. Mainly used by secondary region replicas to keep up to date with the primary region files.
    
    Specified by:
    
    refreshStoreFiles in interface Store
    
    Throws:
    
    java.io.IOException
  - add
```
public Pair<java.lang.Long,Cell> add(Cell cell)
```
    Description copied from interface: Store
    
    Adds a value to the memstore
    
    Specified by:
    
    add in interface Store
    
    Returns:
    
    memstore size delta & newly added KV which maybe different than the passed in KV
  - timeOfOldestEdit
```
public long timeOfOldestEdit()
```
    Description copied from interface: Store
    
    When was the last edit done in the memstore
    
    Specified by:
    
    timeOfOldestEdit in interface Store
  - delete
```
protected long delete(KeyValue kv)
```
    Adds a value to the memstore
    
    Parameters:
    
    kv -
    
    Returns:
    
    memstore size delta
  - rollback
```
public void rollback(Cell cell)
```
    Description copied from interface: Store
    
    Removes a Cell from the memstore. The Cell is removed only if its key & memstoreTS match the key & memstoreTS value of the cell parameter.
    
    Specified by:
    
    rollback in interface Store
  - getStorefiles
```
public java.util.Collection<StoreFile> getStorefiles()
```
    Specified by:
    
    getStorefiles in interface Store
    
    Returns:
    
    All store files.
  - assertBulkLoadHFileOk
```
public void assertBulkLoadHFileOk(Path srcPath)
                           throws java.io.IOException
```
    Description copied from interface: Store
    
    This throws a WrongRegionException if the HFile does not fit in this region, or an InvalidHFileException if the HFile is not valid.
    
    Specified by:
    
    assertBulkLoadHFileOk in interface Store
    
    Throws:
    
    java.io.IOException
  - preBulkLoadHFile
```
public Pair<Path,Path> preBulkLoadHFile(java.lang.String srcPathStr,
                                        long seqNum)
                                 throws java.io.IOException
```
    Throws:
    
    java.io.IOException
  - bulkLoadHFile
```
public Path bulkLoadHFile(byte[] family,
                          java.lang.String srcPathStr,
                          Path dstPath)
                   throws java.io.IOException
```
    Description copied from interface: Store
    
    This method should only be called from Region. It is assumed that the ranges of values in the HFile fit within the stores assigned region. (assertBulkLoadHFileOk checks this)
    
    Specified by:
    
    bulkLoadHFile in interface Store
    
    Parameters:
    
    family - the column family
    
    Throws:
    
    java.io.IOException
  - bulkLoadHFile
```
public void bulkLoadHFile(StoreFileInfo fileInfo)
                   throws java.io.IOException
```
    Specified by:
    
    bulkLoadHFile in interface Store
    
    Throws:
    
    java.io.IOException
  - close
```
public <any> close()
            throws java.io.IOException
```
    Description copied from interface: Store
    
    Close all the readers We don't need to worry about subsequent requests because the Region holds a write lock that will prevent any more reads or writes.
    
    Specified by:
    
    close in interface Store
    
    Returns:
    
    the StoreFiles that were previously being used.
    
    Throws:
    
    java.io.IOException - on failure
  - flushCache
```
protected java.util.List<Path> flushCache(long logCacheFlushId,
                                          MemStoreSnapshot snapshot,
                                          MonitoredTask status)
                                   throws java.io.IOException
```
    Write out current snapshot. Presumes snapshot() has been called previously.
    
    Parameters:
    
    logCacheFlushId - flush sequence number
    
    snapshot -
    
    status -
    
    Returns:
    
    The path name of the tmp file to which the store was flushed
    
    Throws:
    
    java.io.IOException
  - createWriterInTmp
```
public StoreFile.Writer createWriterInTmp(long maxKeyCount,
                                          Compression.Algorithm compression,
                                          boolean isCompaction,
                                          boolean includeMVCCReadpoint,
                                          boolean includesTag)
                                   throws java.io.IOException
```
    Specified by:
    
    createWriterInTmp in interface Store
    
    Throws:
    
    java.io.IOException
  - getScanners
```
public java.util.List<KeyValueScanner> getScanners(boolean cacheBlocks,
                                                   boolean isGet,
                                                   boolean usePread,
                                                   boolean isCompaction,
                                                   ScanQueryMatcher matcher,
                                                   byte[] startRow,
                                                   byte[] stopRow,
                                                   long readPt)
                                            throws java.io.IOException
```
    Get all scanners with no filtering based on TTL (that happens further down the line).
    
    Specified by:
    
    getScanners in interface Store
    
    Returns:
    
    all scanners for this store
    
    Throws:
    
    java.io.IOException
  - addChangedReaderObserver
```
public void addChangedReaderObserver(ChangedReadersObserver o)
```
    Specified by:
    
    addChangedReaderObserver in interface Store
  - deleteChangedReaderObserver
```
public void deleteChangedReaderObserver(ChangedReadersObserver o)
```
    Specified by:
    
    deleteChangedReaderObserver in interface Store
  - compact
```
public java.util.List<StoreFile> compact(CompactionContext compaction,
                                         CompactionThroughputController throughputController)
                                  throws java.io.IOException
```
    Compact the StoreFiles. This method may take some time, so the calling thread must be able to block for long periods.
    During this time, the Store can work as usual, getting values from StoreFiles and writing new StoreFiles from the memstore. Existing StoreFiles are not destroyed until the new compacted StoreFile is completely written-out to disk.
    The compactLock prevents multiple simultaneous compactions. The structureLock prevents us from interfering with other write operations.
    We don't want to hold the structureLock for the whole time, as a compact() can be lengthy and we want to allow cache-flushes during this period.
    Compaction event should be idempotent, since there is no IO Fencing for the region directory in hdfs. A region server might still try to complete the compaction after it lost the region. That is why the following events are carefully ordered for a compaction: 1. Compaction writes new files under region/.tmp directory (compaction output) 2. Compaction atomically moves the temporary file under region directory 3. Compaction appends a WAL edit containing the compaction input and output files. Forces sync on WAL. 4. Compaction deletes the input files from the region directory. Failure conditions are handled like this: - If RS fails before 2, compaction wont complete. Even if RS lives on and finishes the compaction later, it will only write the new data file to the region directory. Since we already have this data, this will be idempotent but we will have a redundant copy of the data. - If RS fails between 2 and 3, the region will have a redundant copy of the data. The RS that failed won't be able to finish snyc() for WAL because of lease recovery in WAL. - If RS fails after 3, the region region server who opens the region will pick up the the compaction marker from the WAL and replay it by removing the compaction input files. Failed RS can also attempt to delete those files, but the operation will be idempotent See HBASE-2231 for details.
    
    Specified by:
    
    compact in interface Store
    
    Parameters:
    
    compaction - compaction details obtained from requestCompaction()
    
    Returns:
    
    Storefile we compacted into or null if we failed or opted out early.
    
    Throws:
    
    java.io.IOException
  - compact
```
public java.util.List<StoreFile> compact(CompactionContext compaction,
                                         CompactionThroughputController throughputController,
                                         User user)
                                  throws java.io.IOException
```
    Specified by:
    
    compact in interface Store
    
    Throws:
    
    java.io.IOException
  - replayCompactionMarker
```
public void replayCompactionMarker(WALProtos.CompactionDescriptor compaction,
                                   boolean pickCompactionFiles,
                                   boolean removeFiles)
                            throws java.io.IOException
```
    Call to complete a compaction. Its for the case where we find in the WAL a compaction that was not finished. We could find one recovering a WAL after a regionserver crash. See HBASE-2231.
    
    Specified by:
    
    replayCompactionMarker in interface Store
    
    Parameters:
    
    compaction -
    
    pickCompactionFiles - whether or not pick up the new compaction output files and add it to the store
    
    removeFiles - whether to remove/archive files from filesystem
    
    Throws:
    
    java.io.IOException
  - compactRecentForTestingAssumingDefaultPolicy
```
public void compactRecentForTestingAssumingDefaultPolicy(int N)
                                                  throws java.io.IOException
```
    This method tries to compact N recent files for testing. Note that because compacting "recent" files only makes sense for some policies, e.g. the default one, it assumes default policy is used. It doesn't use policy, but instead makes a compaction candidate list by itself.
    
    Parameters:
    
    N - Number of files.
    
    Throws:
    
    java.io.IOException
  - hasReferences
```
public boolean hasReferences()
```
    Specified by:
    
    hasReferences in interface Store
    
    Returns:
    
    true if the store has any underlying reference files to older HFiles
  - getCompactionProgress
```
public CompactionProgress getCompactionProgress()
```
    Description copied from interface: Store
    
    getter for CompactionProgress object
    
    Specified by:
    
    getCompactionProgress in interface Store
    
    Returns:
    
    CompactionProgress object; can be null
  - isMajorCompaction
```
public boolean isMajorCompaction()
                          throws java.io.IOException
```
    Specified by:
    
    isMajorCompaction in interface Store
    
    Returns:
    
    true if we should run a major compaction.
    
    Throws:
    
    java.io.IOException
  - requestCompaction
```
public CompactionContext requestCompaction()
                                    throws java.io.IOException
```
    Specified by:
    
    requestCompaction in interface Store
    
    Throws:
    
    java.io.IOException
  - requestCompaction
```
public CompactionContext requestCompaction(int priority,
                                           CompactionRequest baseRequest)
                                    throws java.io.IOException
```
    Specified by:
    
    requestCompaction in interface Store
    
    Throws:
    
    java.io.IOException
  - requestCompaction
```
public CompactionContext requestCompaction(int priority,
                                           CompactionRequest baseRequest,
                                           User user)
                                    throws java.io.IOException
```
    Specified by:
    
    requestCompaction in interface Store
    
    Throws:
    
    java.io.IOException
  - cancelRequestedCompaction
```
public void cancelRequestedCompaction(CompactionContext compaction)
```
    Specified by:
    
    cancelRequestedCompaction in interface Store
  - completeCompaction
```
protected void completeCompaction(java.util.Collection<StoreFile> compactedFiles)
                           throws java.io.IOException
```
    It works by processing a compaction that's been written to disk.
    It is usually invoked at the end of a compaction, but might also be invoked at HStore startup, if the prior execution died midway through.
    Moving the compacted TreeMap into place means:
```
 1) Unload all replaced StoreFile, close and collect list to delete.
 2) Compute new store size
 
```
    Parameters:
    
    compactedFiles - list of files that were compacted
    
    Throws:
    
    java.io.IOException
  - completeCompaction
```
protected void completeCompaction(java.util.Collection<StoreFile> compactedFiles,
                                  boolean removeFiles)
                           throws java.io.IOException
```
    It works by processing a compaction that's been written to disk.
    It is usually invoked at the end of a compaction, but might also be invoked at HStore startup, if the prior execution died midway through.
    Moving the compacted TreeMap into place means:
```
 1) Unload all replaced StoreFile, close and collect list to delete.
 2) Compute new store size
 
```
    Parameters:
    
    compactedFiles - list of files that were compacted
    
    Throws:
    
    java.io.IOException
  - getRowKeyAtOrBefore
```
public Cell getRowKeyAtOrBefore(byte[] row)
                         throws java.io.IOException
```
    Description copied from interface: Store
    
    Find the key that matches row exactly, or the one that immediately precedes it. WARNING: Only use this method on a table where writes occur with strictly increasing timestamps. This method assumes this pattern of writes in order to make it reasonably performant. Also our search is dependent on the axiom that deletes are for cells that are in the container that follows whether a memstore snapshot or a storefile, not for the current container: i.e. we'll see deletes before we come across cells we are to delete. Presumption is that the memstore#kvset is processed before memstore#snapshot and so on.
    
    Specified by:
    
    getRowKeyAtOrBefore in interface Store
    
    Parameters:
    
    row - The row key of the targeted row.
    
    Returns:
    
    Found Cell or null if none found.
    
    Throws:
    
    java.io.IOException
  - canSplit
```
public boolean canSplit()
```
    Specified by:
    
    canSplit in interface Store
  - getSplitPoint
```
public byte[] getSplitPoint()
```
    Description copied from interface: Store
    
    Determines if Store should be split
    
    Specified by:
    
    getSplitPoint in interface Store
    
    Returns:
    
    byte[] if store should be split, null otherwise.
  - getLastCompactSize
```
public long getLastCompactSize()
```
    Specified by:
    
    getLastCompactSize in interface Store
    
    Returns:
    
    aggregate size of all HStores used in the last compaction
  - getSize
```
public long getSize()
```
    Specified by:
    
    getSize in interface Store
    
    Returns:
    
    aggregate size of HStore
  - triggerMajorCompaction
```
public void triggerMajorCompaction()
```
    Specified by:
    
    triggerMajorCompaction in interface Store
  - getScanner
```
public KeyValueScanner getScanner(Scan scan,
                                  java.util.NavigableSet<byte[]> targetCols,
                                  long readPt)
                           throws java.io.IOException
```
    Description copied from interface: Store
    
    Return a scanner for both the memstore and the HStore files. Assumes we are not in a compaction.
    
    Specified by:
    
    getScanner in interface Store
    
    Parameters:
    
    scan - Scan to apply when scanning the stores
    
    targetCols - columns to scan
    
    Returns:
    
    a scanner over the current key values
    
    Throws:
    
    java.io.IOException - on failure
  - createScanner
```
protected KeyValueScanner createScanner(Scan scan,
                                        java.util.NavigableSet<byte[]> targetCols,
                                        long readPt,
                                        KeyValueScanner scanner)
                                 throws java.io.IOException
```
    Throws:
    
    java.io.IOException
  - toString
```
public java.lang.String toString()
```
    Overrides:
    
    toString in class java.lang.Object
  - getStorefilesCount
```
public int getStorefilesCount()
```
    Specified by:
    
    getStorefilesCount in interface Store
    
    Returns:
    
    Count of store files
  - getMaxStoreFileAge
```
public long getMaxStoreFileAge()
```
    Specified by:
    
    getMaxStoreFileAge in interface Store
    
    Returns:
    
    Max age of store files in this store
  - getMinStoreFileAge
```
public long getMinStoreFileAge()
```
    Specified by:
    
    getMinStoreFileAge in interface Store
    
    Returns:
    
    Min age of store files in this store
  - getAvgStoreFileAge
```
public long getAvgStoreFileAge()
```
    Specified by:
    
    getAvgStoreFileAge in interface Store
    
    Returns:
    
    Average age of store files in this store, 0 if no store files
  - getNumReferenceFiles
```
public long getNumReferenceFiles()
```
    Specified by:
    
    getNumReferenceFiles in interface Store
    
    Returns:
    
    Number of reference files in this store
  - getNumHFiles
```
public long getNumHFiles()
```
    Specified by:
    
    getNumHFiles in interface Store
    
    Returns:
    
    Number of HFiles in this store
  - getStoreSizeUncompressed
```
public long getStoreSizeUncompressed()
```
    Specified by:
    
    getStoreSizeUncompressed in interface Store
    
    Returns:
    
    The size of the store files, in bytes, uncompressed.
  - getStorefilesSize
```
public long getStorefilesSize()
```
    Specified by:
    
    getStorefilesSize in interface Store
    
    Returns:
    
    The size of the store files, in bytes.
  - getStorefilesIndexSize
```
public long getStorefilesIndexSize()
```
    Specified by:
    
    getStorefilesIndexSize in interface Store
    
    Returns:
    
    The size of the store file indexes, in bytes.
  - getTotalStaticIndexSize
```
public long getTotalStaticIndexSize()
```
    Description copied from interface: Store
    
    Returns the total size of all index blocks in the data block indexes, including the root level, intermediate levels, and the leaf level for multi-level indexes, or just the root level for single-level indexes.
    
    Specified by:
    
    getTotalStaticIndexSize in interface Store
    
    Returns:
    
    the total size of block indexes in the store
  - getTotalStaticBloomSize
```
public long getTotalStaticBloomSize()
```
    Description copied from interface: Store
    
    Returns the total byte size of all Bloom filter bit arrays. For compound Bloom filters even the Bloom blocks currently not loaded into the block cache are counted.
    
    Specified by:
    
    getTotalStaticBloomSize in interface Store
    
    Returns:
    
    the total size of all Bloom filters in the store
  - getMemStoreSize
```
public long getMemStoreSize()
```
    Specified by:
    
    getMemStoreSize in interface Store
    
    Returns:
    
    The size of this store's memstore, in bytes
  - getCompactPriority
```
public int getCompactPriority()
```
    Specified by:
    
    getCompactPriority in interface Store
  - throttleCompaction
```
public boolean throttleCompaction(long compactionSize)
```
    Specified by:
    
    throttleCompaction in interface Store
  - getHRegion
```
public HRegion getHRegion()
```
  - getCoprocessorHost
```
public RegionCoprocessorHost getCoprocessorHost()
```
    Specified by:
    
    getCoprocessorHost in interface Store
  - getRegionInfo
```
public HRegionInfo getRegionInfo()
```
    Specified by:
    
    getRegionInfo in interface Store
    
    Returns:
    
    the parent region info hosting this store
  - areWritesEnabled
```
public boolean areWritesEnabled()
```
    Specified by:
    
    areWritesEnabled in interface Store
  - getSmallestReadPoint
```
public long getSmallestReadPoint()
```
    Specified by:
    
    getSmallestReadPoint in interface Store
    
    Returns:
    
    The smallest mvcc readPoint across all the scanners in this region. Writes older than this readPoint, are included in every read operation.
  - updateColumnValue
```
public long updateColumnValue(byte[] row,
                              byte[] f,
                              byte[] qualifier,
                              long newValue)
                       throws java.io.IOException
```
    Used in tests. TODO: Remove Updates the value for the given row/family/qualifier. This function will always be seen as atomic by other readers because it only puts a single KV to memstore. Thus no read/write control necessary.
    
    Parameters:
    
    row - row to update
    
    f - family to update
    
    qualifier - qualifier to update
    
    newValue - the new value to set into memstore
    
    Returns:
    
    memstore size delta
    
    Throws:
    
    java.io.IOException
  - upsert
```
public long upsert(java.lang.Iterable<Cell> cells,
                   long readpoint)
            throws java.io.IOException
```
    Description copied from interface: Store
    
    Adds or replaces the specified KeyValues.
    For each KeyValue specified, if a cell with the same row, family, and qualifier exists in MemStore, it will be replaced. Otherwise, it will just be inserted to MemStore.
    This operation is atomic on each KeyValue (row/family/qualifier) but not necessarily atomic across all of them.
    
    Specified by:
    
    upsert in interface Store
    
    readpoint - readpoint below which we can safely remove duplicate KVs
    
    Returns:
    
    memstore size delta
    
    Throws:
    
    java.io.IOException
  - createFlushContext
```
public org.apache.hadoop.hbase.regionserver.StoreFlushContext createFlushContext(long cacheFlushId)
```
    Specified by:
    
    createFlushContext in interface Store
  - needsCompaction
```
public boolean needsCompaction()
```
    Description copied from interface: Store
    
    See if there's too much store files in this store
    
    Specified by:
    
    needsCompaction in interface Store
    
    Returns:
    
    true if number of store files is greater than the number defined in minFilesToCompact
  - getCacheConfig
```
public CacheConfig getCacheConfig()
```
    Description copied from interface: Store
    
    Used for tests.
    
    Specified by:
    
    getCacheConfig in interface Store
    
    Returns:
    
    cache configuration for this Store.
  - heapSize
```
public long heapSize()
```
    Specified by:
    
    heapSize in interface HeapSize
    
    Returns:
    
    Approximate 'exclusive deep size' of implementing object. Includes count of payload and hosting object sizings.
  - getComparator
```
public KeyValue.KVComparator getComparator()
```
    Specified by:
    
    getComparator in interface Store
  - getScanInfo
```
public ScanInfo getScanInfo()
```
    Specified by:
    
    getScanInfo in interface Store
  - hasTooManyStoreFiles
```
public boolean hasTooManyStoreFiles()
```
    Specified by:
    
    hasTooManyStoreFiles in interface Store
    
    Returns:
    
    Whether this store has too many store files.
  - getFlushedCellsCount
```
public long getFlushedCellsCount()
```
    Specified by:
    
    getFlushedCellsCount in interface Store
    
    Returns:
    
    The number of cells flushed to disk
  - getFlushedCellsSize
```
public long getFlushedCellsSize()
```
    Specified by:
    
    getFlushedCellsSize in interface Store
    
    Returns:
    
    The total size of data flushed to disk, in bytes
  - getFlushedOutputFileSize
```
public long getFlushedOutputFileSize()
```
    Specified by:
    
    getFlushedOutputFileSize in interface Store
    
    Returns:
    
    The total size of out output files on disk, in bytes
  - getCompactedCellsCount
```
public long getCompactedCellsCount()
```
    Specified by:
    
    getCompactedCellsCount in interface Store
    
    Returns:
    
    The number of cells processed during minor compactions
  - getCompactedCellsSize
```
public long getCompactedCellsSize()
```
    Specified by:
    
    getCompactedCellsSize in interface Store
    
    Returns:
    
    The total amount of data processed during minor compactions, in bytes
  - getMajorCompactedCellsCount
```
public long getMajorCompactedCellsCount()
```
    Specified by:
    
    getMajorCompactedCellsCount in interface Store
    
    Returns:
    
    The number of cells processed during major compactions
  - getMajorCompactedCellsSize
```
public long getMajorCompactedCellsSize()
```
    Specified by:
    
    getMajorCompactedCellsSize in interface Store
    
    Returns:
    
    The total amount of data processed during major compactions, in bytes
  - getStoreEngine
```
public StoreEngine<?,?,?,?> getStoreEngine()
```
    Returns the StoreEngine that is backing this concrete implementation of Store.
    
    Returns:
    
    Returns the StoreEngine object used internally inside this HStore object.
  - getOffPeakHours
```
protected OffPeakHours getOffPeakHours()
```
  - onConfigurationChange
```
public void onConfigurationChange(Configuration conf)
```
    This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.
    
    Specified by:
    
    onConfigurationChange in interface ConfigurationObserver
  - registerChildren
```
public void registerChildren(ConfigurationManager manager)
```
    Needs to be called to register the children to the manager.
    
    Specified by:
    
    registerChildren in interface PropagatingConfigurationObserver
    
    Parameters:
    
    manager - : to register to
  - deregisterChildren
```
public void deregisterChildren(ConfigurationManager manager)
```
    Needs to be called to deregister the children from the manager.
    
    Specified by:
    
    deregisterChildren in interface PropagatingConfigurationObserver
    
    Parameters:
    
    manager - : to deregister from
  - getCompactionPressure
```
public double getCompactionPressure()
```
    Description copied from interface: Store
    This value can represent the degree of emergency of compaction for this store. It should be greater than or equal to 0.0, any value greater than 1.0 means we have too many store files.
    - if getStorefilesCount <= getMinFilesToCompact, return 0.0
    - return (getStorefilesCount - getMinFilesToCompact) / (blockingFileCount - getMinFilesToCompact)
    And for striped stores, we should calculate this value by the files in each stripe separately and return the maximum value.
    It is similar to Store.getCompactPriority() except that it is more suitable to use in a linear formula.
    Specified by:
    
    getCompactionPressure in interface Store
  - isPrimaryReplicaStore
```
public boolean isPrimaryReplicaStore()
```
    Specified by:
    
    isPrimaryReplicaStore in interface Store

Class HStore

Field Summary

Fields inherited from interface org.apache.hadoop.hbase.regionserver.Store

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

COMPACTCHECKER_INTERVAL_MULTIPLIER_KEY

BLOCKING_STOREFILES_KEY

DEFAULT_COMPACTCHECKER_INTERVAL_MULTIPLIER

DEFAULT_BLOCKING_STOREFILE_COUNT

memstore

region

conf

cacheConf

blocksize

checksumType

bytesPerChecksum

cryptoContext

FIXED_OVERHEAD

DEEP_OVERHEAD

Constructor Detail

HStore

Method Detail

createCacheConf

createStoreEngine

determineTTLFromFamily

getColumnFamilyName

getTableName

getFileSystem

getRegionFileSystem

getStoreFileTtl

getMemstoreFlushSize

getFlushableSize

getSnapshotSize

getCompactionCheckMultiplier

getBlockingFileCount

getBytesPerChecksum

getChecksumType

getCloseCheckInterval

getFamily

getMaxSequenceId

getMaxMemstoreTS

getStoreHomedir

getStoreHomedir

getDataBlockEncoder

refreshStoreFiles

refreshStoreFiles

add

timeOfOldestEdit

delete

rollback

getStorefiles

assertBulkLoadHFileOk

preBulkLoadHFile

bulkLoadHFile

bulkLoadHFile

close

flushCache

createWriterInTmp

getScanners

addChangedReaderObserver

deleteChangedReaderObserver

compact

compact

replayCompactionMarker

compactRecentForTestingAssumingDefaultPolicy

hasReferences

getCompactionProgress

isMajorCompaction

requestCompaction

requestCompaction

requestCompaction

cancelRequestedCompaction

completeCompaction

completeCompaction

getRowKeyAtOrBefore

canSplit

getSplitPoint

getLastCompactSize