org.apache.hadoop.hbase.regionserver

Class StoreScanner

java.lang.Object

org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner

org.apache.hadoop.hbase.regionserver.NonReversedNonLazyKeyValueScanner

org.apache.hadoop.hbase.regionserver.StoreScanner

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, ChangedReadersObserver, InternalScanner, KeyValueScanner

Direct Known Subclasses:

MobStoreScanner, ReversedMobStoreScanner

@InterfaceAudience.Private
public class StoreScanner
extends NonReversedNonLazyKeyValueScanner
implements KeyValueScanner, InternalScanner, ChangedReadersObserver

Scanner scans both the memstore and the Store. Coalesce KeyValue stream into List for a single row.

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	cacheBlocks
protected long	cellsPerHeartbeatCheck
protected boolean	closing
protected java.util.NavigableSet<byte[]>	columns
protected int	countPerRow
static long	DEFAULT_HBASE_CELLS_SCANNED_PER_HEARTBEAT_CHECK Default value of HBASE_CELLS_SCANNED_PER_HEARTBEAT_CHECK.
protected ExecutorService	executor
protected boolean	explicitColumnQuery
static java.lang.String	HBASE_CELLS_SCANNED_PER_HEARTBEAT_CHECK The number of cells scanned in between timeout checks.
protected KeyValueHeap	heap
protected boolean	isGet
protected boolean	isParallelSeekEnabled A flag that enables StoreFileScanner parallel-seeking
protected Cell	lastTop
protected static boolean	lazySeekEnabledGlobally Used during unit testing to ensure that lazy seek does save seek ops
protected java.util.concurrent.locks.ReentrantLock	lock
protected ScanQueryMatcher	matcher
protected long	maxRowSize
protected int	minVersions
protected long	now
protected long	oldestUnexpiredTS
protected long	readPt
protected Scan	scan
protected Store	store
protected int	storeLimit
protected int	storeOffset
static java.lang.String	STORESCANNER_PARALLEL_SEEK_ENABLE
protected boolean	useRowColBloom

Constructor Summary

Constructors

Modifier	Constructor and Description
	StoreScanner(Scan scan, ScanInfo scanInfo, ScanType scanType, java.util.NavigableSet<byte[]> columns, java.util.List<KeyValueScanner> scanners, long earliestPutTs, long readPt)
protected	StoreScanner(Store store, boolean cacheBlocks, Scan scan, java.util.NavigableSet<byte[]> columns, long ttl, int minVersions, long readPt) An internal constructor.
	StoreScanner(Store store, ScanInfo scanInfo, Scan scan, java.util.List<? extends KeyValueScanner> scanners, long smallestReadPoint, long earliestPutTs, byte[] dropDeletesFromRow, byte[] dropDeletesToRow) Used for compactions that drop deletes from a limited range of rows.
	StoreScanner(Store store, ScanInfo scanInfo, Scan scan, java.util.List<? extends KeyValueScanner> scanners, ScanType scanType, long smallestReadPoint, long earliestPutTs) Used for compactions.
	StoreScanner(Store store, ScanInfo scanInfo, Scan scan, java.util.NavigableSet<byte[]> columns, long readPt) Opens a scanner across memstore, snapshot, and all StoreFiles.

Method Summary

Modifier and Type	Method and Description
protected boolean	checkReseek()
protected void	checkScanOrder(Cell prevKV, Cell kv, KeyValue.KVComparator comparator) Check whether scan as expected order
void	close() Close the KeyValue scanner.
long	getEstimatedNumberOfKvsScanned()
Cell	getNextIndexedKey()
protected java.util.List<KeyValueScanner>	getScannersNoCompaction() Get a filtered list of scanners.
long	getSequenceID() Get the sequence id associated with this KeyValueScanner.
KeyValue	next() Return the next Cell in this scanner, iterating the scanner
boolean	next(java.util.List<Cell> outResult) Grab the next row's worth of values.
boolean	next(java.util.List<Cell> outResult, ScannerContext scannerContext) Get the next row of values from this Store.
Cell	peek() Look at the next Cell in this scanner, but do not iterate scanner.
boolean	reseek(Cell kv) Reseek the scanner at or after the specified KeyValue.
protected void	resetKVHeap(java.util.List<? extends KeyValueScanner> scanners, KeyValue.KVComparator comparator)
protected void	resetScannerStack(Cell lastTopKey)
boolean	seek(Cell key) Seek the scanner at or after the specified KeyValue.
protected boolean	seekAsDirection(Cell kv) Do a reseek in a normal StoreScanner(scan forward)
protected void	seekScanners(java.util.List<? extends KeyValueScanner> scanners, Cell seekKey, boolean isLazy, boolean isParallelSeek) Seek the specified scanners with the given key
protected boolean	seekToNextRow(Cell kv)
protected java.util.List<KeyValueScanner>	selectScannersFrom(java.util.List<? extends KeyValueScanner> allScanners) Filters the given list of scanners using Bloom filter, time range, and TTL.
void	updateReaders() Notify observers.

Methods inherited from class org.apache.hadoop.hbase.regionserver.NonReversedNonLazyKeyValueScanner

backwardSeek, seekToLastRow, seekToPreviousRow

Methods inherited from class org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner

doRealSeek, enforceSeek, isFileScanner, realSeekDone, requestSeek, shouldUseScanner

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.regionserver.KeyValueScanner

backwardSeek, enforceSeek, isFileScanner, realSeekDone, requestSeek, seekToLastRow, seekToPreviousRow, shouldUseScanner

Field Detail

store

protected Store store

matcher

protected ScanQueryMatcher matcher

heap

protected KeyValueHeap heap

cacheBlocks

protected boolean cacheBlocks

countPerRow

protected int countPerRow

storeLimit

protected int storeLimit

storeOffset

protected int storeOffset

closing

protected boolean closing

isGet

protected final boolean isGet

explicitColumnQuery

protected final boolean explicitColumnQuery

useRowColBloom

protected final boolean useRowColBloom

isParallelSeekEnabled

protected boolean isParallelSeekEnabled

A flag that enables StoreFileScanner parallel-seeking

executor

protected ExecutorService executor

scan

protected final Scan scan

columns

protected final java.util.NavigableSet<byte[]> columns

oldestUnexpiredTS

protected final long oldestUnexpiredTS

now

protected final long now

minVersions

protected final int minVersions

maxRowSize

protected final long maxRowSize

cellsPerHeartbeatCheck

protected final long cellsPerHeartbeatCheck

STORESCANNER_PARALLEL_SEEK_ENABLE

public static final java.lang.String STORESCANNER_PARALLEL_SEEK_ENABLE

See Also:

Constant Field Values

lazySeekEnabledGlobally

protected static boolean lazySeekEnabledGlobally

Used during unit testing to ensure that lazy seek does save seek ops

HBASE_CELLS_SCANNED_PER_HEARTBEAT_CHECK

public static final java.lang.String HBASE_CELLS_SCANNED_PER_HEARTBEAT_CHECK

The number of cells scanned in between timeout checks. Specifying a larger value means that timeout checks will occur less frequently. Specifying a small value will lead to more frequent timeout checks.

See Also:

Constant Field Values

DEFAULT_HBASE_CELLS_SCANNED_PER_HEARTBEAT_CHECK

public static final long DEFAULT_HBASE_CELLS_SCANNED_PER_HEARTBEAT_CHECK

Default value of HBASE_CELLS_SCANNED_PER_HEARTBEAT_CHECK.

See Also:

Constant Field Values

lastTop

protected Cell lastTop

lock

protected java.util.concurrent.locks.ReentrantLock lock

readPt

protected final long readPt

Constructor Detail

StoreScanner

protected StoreScanner(Store store,
                       boolean cacheBlocks,
                       Scan scan,
                       java.util.NavigableSet<byte[]> columns,
                       long ttl,
                       int minVersions,
                       long readPt)

An internal constructor.

StoreScanner

public StoreScanner(Store store,
                    ScanInfo scanInfo,
                    Scan scan,
                    java.util.NavigableSet<byte[]> columns,
                    long readPt)
             throws java.io.IOException

Opens a scanner across memstore, snapshot, and all StoreFiles. Assumes we are not in a compaction.

Parameters:

store - who we scan

scan - the spec

columns - which columns we are scanning

Throws:

java.io.IOException

StoreScanner

public StoreScanner(Store store,
                    ScanInfo scanInfo,
                    Scan scan,
                    java.util.List<? extends KeyValueScanner> scanners,
                    ScanType scanType,
                    long smallestReadPoint,
                    long earliestPutTs)
             throws java.io.IOException

Used for compactions.

Opens a scanner across specified StoreFiles.

Parameters:

store - who we scan

scan - the spec

scanners - ancillary scanners

smallestReadPoint - the readPoint that we should use for tracking versions

Throws:

java.io.IOException

StoreScanner

public StoreScanner(Store store,
                    ScanInfo scanInfo,
                    Scan scan,
                    java.util.List<? extends KeyValueScanner> scanners,
                    long smallestReadPoint,
                    long earliestPutTs,
                    byte[] dropDeletesFromRow,
                    byte[] dropDeletesToRow)
             throws java.io.IOException

Used for compactions that drop deletes from a limited range of rows.

Opens a scanner across specified StoreFiles.

Parameters:

store - who we scan

scan - the spec

scanners - ancillary scanners

smallestReadPoint - the readPoint that we should use for tracking versions

dropDeletesFromRow - The inclusive left bound of the range; can be EMPTY_START_ROW.

dropDeletesToRow - The exclusive right bound of the range; can be EMPTY_END_ROW.

Throws:

java.io.IOException

StoreScanner

public StoreScanner(Scan scan,
                    ScanInfo scanInfo,
                    ScanType scanType,
                    java.util.NavigableSet<byte[]> columns,
                    java.util.List<KeyValueScanner> scanners,
                    long earliestPutTs,
                    long readPt)
             throws java.io.IOException

Throws:

java.io.IOException

Method Detail

getScannersNoCompaction

protected java.util.List<KeyValueScanner> getScannersNoCompaction()
                                                           throws java.io.IOException

Get a filtered list of scanners. Assumes we are not in a compaction.

Returns:

list of scanners to seek

Throws:

java.io.IOException

seekScanners

protected void seekScanners(java.util.List<? extends KeyValueScanner> scanners,
                            Cell seekKey,
                            boolean isLazy,
                            boolean isParallelSeek)
                     throws java.io.IOException

Seek the specified scanners with the given key

Parameters:

scanners -

seekKey -

isLazy - true if using lazy seek

isParallelSeek - true if using parallel seek

Throws:

java.io.IOException

resetKVHeap

protected void resetKVHeap(java.util.List<? extends KeyValueScanner> scanners,
                           KeyValue.KVComparator comparator)
                    throws java.io.IOException

Throws:

java.io.IOException

selectScannersFrom

protected java.util.List<KeyValueScanner> selectScannersFrom(java.util.List<? extends KeyValueScanner> allScanners)

Filters the given list of scanners using Bloom filter, time range, and TTL.

peek

public Cell peek()

Description copied from interface: KeyValueScanner

Look at the next Cell in this scanner, but do not iterate scanner.

Specified by:

peek in interface KeyValueScanner

Returns:

the next Cell

next

public KeyValue next()

Description copied from interface: KeyValueScanner

Return the next Cell in this scanner, iterating the scanner

Specified by:

next in interface KeyValueScanner

Returns:

the next Cell

close

public void close()

Description copied from interface: KeyValueScanner

Close the KeyValue scanner.

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface InternalScanner

Specified by:

close in interface KeyValueScanner

seek

public boolean seek(Cell key)
             throws java.io.IOException

Description copied from interface: KeyValueScanner

Seek the scanner at or after the specified KeyValue.

Specified by:

seek in interface KeyValueScanner

Parameters:

key - seek value

Returns:

true if scanner has values left, false if end of scanner

Throws:

java.io.IOException

next

public boolean next(java.util.List<Cell> outResult)
             throws java.io.IOException

Description copied from interface: InternalScanner

Grab the next row's worth of values.

Specified by:

next in interface InternalScanner

Parameters:

outResult - return output array

Returns:

true if more rows exist after this one, false if scanner is done

Throws:

java.io.IOException - e

next

public boolean next(java.util.List<Cell> outResult,
                    ScannerContext scannerContext)
             throws java.io.IOException

Get the next row of values from this Store.

Specified by:

next in interface InternalScanner

Parameters:

outResult -

scannerContext -

Returns:

true if there are more rows, false if scanner is done

Throws:

java.io.IOException - e

updateReaders

public void updateReaders()
                   throws java.io.IOException

Description copied from interface: ChangedReadersObserver

Notify observers.

Specified by:

updateReaders in interface ChangedReadersObserver

Throws:

java.io.IOException - e

checkReseek

protected boolean checkReseek()
                       throws java.io.IOException

Returns:

true if top of heap has changed (and KeyValueHeap has to try the next KV)

Throws:

java.io.IOException

resetScannerStack

protected void resetScannerStack(Cell lastTopKey)
                          throws java.io.IOException

Throws:

java.io.IOException

checkScanOrder

protected void checkScanOrder(Cell prevKV,
                              Cell kv,
                              KeyValue.KVComparator comparator)
                       throws java.io.IOException

Check whether scan as expected order

Parameters:

prevKV -

kv -

comparator -

Throws:

java.io.IOException

seekToNextRow

protected boolean seekToNextRow(Cell kv)
                         throws java.io.IOException

Throws:

java.io.IOException

seekAsDirection

protected boolean seekAsDirection(Cell kv)
                           throws java.io.IOException

Do a reseek in a normal StoreScanner(scan forward)

Parameters:

kv -

Returns:

true if scanner has values left, false if end of scanner

Throws:

java.io.IOException

reseek

public boolean reseek(Cell kv)
               throws java.io.IOException

Description copied from interface: KeyValueScanner

Reseek the scanner at or after the specified KeyValue. This method is guaranteed to seek at or after the required key only if the key comes after the current position of the scanner. Should not be used to seek to a key which may come before the current position.

Specified by:

reseek in interface KeyValueScanner

Parameters:

kv - seek value (should be non-null)

Returns:

true if scanner has values left, false if end of scanner

Throws:

java.io.IOException

getSequenceID

public long getSequenceID()

Description copied from interface: KeyValueScanner

Get the sequence id associated with this KeyValueScanner. This is required for comparing multiple files to find out which one has the latest data. The default implementation for this would be to return 0. A file having lower sequence id will be considered to be the older one.

Specified by:

getSequenceID in interface KeyValueScanner

getEstimatedNumberOfKvsScanned

public long getEstimatedNumberOfKvsScanned()

Returns:

The estimated number of KVs seen by this scanner (includes some skipped KVs).

getNextIndexedKey

public Cell getNextIndexedKey()

Specified by:

getNextIndexedKey in interface KeyValueScanner

Overrides:

getNextIndexedKey in class NonLazyKeyValueScanner

Returns:

the next key in the index (the key to seek to the next block) if known, or null otherwise