org.apache.hadoop.hbase.regionserver.compactions

Class FIFOCompactionPolicy

java.lang.Object

org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy

org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy

org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy

org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy

org.apache.hadoop.hbase.regionserver.compactions.FIFOCompactionPolicy

@InterfaceAudience.Private
public class FIFOCompactionPolicy
extends ExploringCompactionPolicy

FIFO compaction policy selects only files which have all cells expired. The column family MUST have non-default TTL. One of the use cases for this policy is when we need to store raw data which will be post-processed later and discarded completely after quite short period of time. Raw time-series vs. time-based roll up aggregates and compacted time-series. We collect raw time-series and store them into CF with FIFO compaction policy, periodically we run task which creates roll up aggregates and compacts time-series, the original raw data can be discarded after that.

Field Summary

Fields inherited from class org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy

comConf, storeConfigInfo

Constructor Summary

Constructors

Constructor and Description
FIFOCompactionPolicy(Configuration conf, StoreConfigInformation storeConfigInfo)

Method Summary

Modifier and Type	Method and Description
boolean	needsCompaction(Collection<StoreFile> storeFiles, List<StoreFile> filesCompacting) A heuristic method to decide whether to schedule a compaction request
CompactionRequest	selectCompaction(Collection<StoreFile> candidateFiles, List<StoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor)
boolean	shouldPerformMajorCompaction(Collection<StoreFile> filesToCompact)

Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy

applyCompactionPolicy, applyCompactionPolicy

Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy

createCompactionRequest, setMinThreshold

Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy

checkMinFilesCriteria, filterBulk, getCurrentEligibleFiles, getNextMajorCompactTime, preSelectCompactionForCoprocessor, removeExcessFiles, skipLargeFiles, throttleCompaction

Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FIFOCompactionPolicy

public FIFOCompactionPolicy(Configuration conf,
                            StoreConfigInformation storeConfigInfo)

Method Detail

selectCompaction

public CompactionRequest selectCompaction(Collection<StoreFile> candidateFiles,
                                          List<StoreFile> filesCompacting,
                                          boolean isUserCompaction,
                                          boolean mayUseOffPeak,
                                          boolean forceMajor)
                                   throws IOException

Overrides:

selectCompaction in class SortedCompactionPolicy

Parameters:

candidateFiles - candidate files, ordered from oldest to newest by seqId. We rely on DefaultStoreFileManager to sort the files by seqId to guarantee contiguous compaction based on seqId for data consistency.

Returns:

subset copy of candidate list that meets compaction criteria

Throws:

IOException

shouldPerformMajorCompaction

public boolean shouldPerformMajorCompaction(Collection<StoreFile> filesToCompact)
                                     throws IOException

Overrides:

shouldPerformMajorCompaction in class RatioBasedCompactionPolicy

Parameters:

filesToCompact - Files to compact. Can be null.

Returns:

True if we should run a major compaction.

Throws:

IOException

needsCompaction

public boolean needsCompaction(Collection<StoreFile> storeFiles,
                               List<StoreFile> filesCompacting)

Description copied from class: RatioBasedCompactionPolicy

A heuristic method to decide whether to schedule a compaction request

Overrides:

needsCompaction in class RatioBasedCompactionPolicy

Parameters:

storeFiles - files in the store.

filesCompacting - files being scheduled to compact.

Returns:

true to schedule a request.