org.apache.hadoop.hbase.regionserver

Interface Store

All Superinterfaces:

ConfigurationObserver, HeapSize, PropagatingConfigurationObserver, StoreConfigInformation

All Known Implementing Classes:

HStore, TestIOFencing.BlockCompactionsInCompletionHStore

@InterfaceAudience.LimitedPrivate(value="Coprocesssor")
 @InterfaceStability.Evolving
public interface Store
extends HeapSize, StoreConfigInformation, PropagatingConfigurationObserver

Interface for objects that hold a column family in a Region. Its a memstore and a set of zero or more StoreFiles, which stretch backwards over time.

Field Summary

Fields

Modifier and Type	Field and Description
static int	NO_PRIORITY
static int	PRIORITY_USER

Method Summary

Modifier and Type	Method and Description
Pair<Long,Cell>	add(Cell cell) Adds a value to the memstore
void	addChangedReaderObserver(ChangedReadersObserver o)
boolean	areWritesEnabled()
void	assertBulkLoadHFileOk(Path srcPath) This throws a WrongRegionException if the HFile does not fit in this region, or an InvalidHFileException if the HFile is not valid.
void	bulkLoadHFile(StoreFileInfo fileInfo)
Path	bulkLoadHFile(String srcPathStr, long sequenceId) This method should only be called from Region.
void	cancelRequestedCompaction(CompactionContext compaction)
boolean	canSplit()
Collection<StoreFile>	close() Close all the readers We don't need to worry about subsequent requests because the Region holds a write lock that will prevent any more reads or writes.
List<StoreFile>	compact(CompactionContext compaction, CompactionThroughputController throughputController)
org.apache.hadoop.hbase.regionserver.StoreFlushContext	createFlushContext(long cacheFlushId)
StoreFile.Writer	createWriterInTmp(long maxKeyCount, Compression.Algorithm compression, boolean isCompaction, boolean includeMVCCReadpoint, boolean includesTags)
void	deleteChangedReaderObserver(ChangedReadersObserver o)
CacheConfig	getCacheConfig() Used for tests.
String	getColumnFamilyName()
long	getCompactedCellsCount()
long	getCompactedCellsSize()
double	getCompactionPressure() This value can represent the degree of emergency of compaction for this store.
CompactionProgress	getCompactionProgress() getter for CompactionProgress object
int	getCompactPriority()
KeyValue.KVComparator	getComparator()
RegionCoprocessorHost	getCoprocessorHost()
HFileDataBlockEncoder	getDataBlockEncoder()
HColumnDescriptor	getFamily()
FileSystem	getFileSystem()
long	getFlushableSize()
long	getFlushedCellsCount()
long	getFlushedCellsSize()
long	getLastCompactSize()
long	getMajorCompactedCellsCount()
long	getMajorCompactedCellsSize()
long	getMaxMemstoreTS()
long	getMaxSequenceId()
long	getMemStoreSize()
HRegionInfo	getRegionInfo()
Cell	getRowKeyAtOrBefore(byte[] row) Find the key that matches row exactly, or the one that immediately precedes it.
ScanInfo	getScanInfo()
KeyValueScanner	getScanner(Scan scan, NavigableSet<byte[]> targetCols, long readPt) Return a scanner for both the memstore and the HStore files.
List<KeyValueScanner>	getScanners(boolean cacheBlocks, boolean isGet, boolean usePread, boolean isCompaction, ScanQueryMatcher matcher, byte[] startRow, byte[] stopRow, long readPt) Get all scanners with no filtering based on TTL (that happens further down the line).
long	getSize()
long	getSmallestReadPoint()
long	getSnapshotSize() Returns the memstore snapshot size
byte[]	getSplitPoint() Determines if Store should be split
Collection<StoreFile>	getStorefiles()
int	getStorefilesCount()
long	getStorefilesIndexSize()
long	getStorefilesSize()
long	getStoreSizeUncompressed()
TableName	getTableName()
long	getTotalStaticBloomSize() Returns the total byte size of all Bloom filter bit arrays.
long	getTotalStaticIndexSize() Returns the total size of all index blocks in the data block indexes, including the root level, intermediate levels, and the leaf level for multi-level indexes, or just the root level for single-level indexes.
boolean	hasReferences()
boolean	hasTooManyStoreFiles()
boolean	isMajorCompaction()
boolean	isPrimaryReplicaStore()
boolean	needsCompaction() See if there's too much store files in this store
void	refreshStoreFiles() Checks the underlying store files, and opens the files that have not been opened, and removes the store file readers for store files no longer available.
void	refreshStoreFiles(Collection<String> newFiles) Replaces the store files that the store has with the given files.
void	replayCompactionMarker(WALProtos.CompactionDescriptor compaction, boolean pickCompactionFiles, boolean removeFiles) Call to complete a compaction.
CompactionContext	requestCompaction()
CompactionContext	requestCompaction(int priority, CompactionRequest baseRequest)
void	rollback(Cell cell) Removes a Cell from the memstore.
boolean	throttleCompaction(long compactionSize)
long	timeOfOldestEdit() When was the last edit done in the memstore
void	triggerMajorCompaction()
long	upsert(Iterable<Cell> cells, long readpoint) Adds or replaces the specified KeyValues.

Methods inherited from interface org.apache.hadoop.hbase.io.HeapSize

heapSize

Methods inherited from interface org.apache.hadoop.hbase.regionserver.StoreConfigInformation

getBlockingFileCount, getCompactionCheckMultiplier, getMemstoreFlushSize, getStoreFileTtl

Methods inherited from interface org.apache.hadoop.hbase.conf.PropagatingConfigurationObserver

deregisterChildren, registerChildren

Methods inherited from interface org.apache.hadoop.hbase.conf.ConfigurationObserver

onConfigurationChange

Field Detail

PRIORITY_USER

static final int PRIORITY_USER

See Also:

Constant Field Values

NO_PRIORITY

static final int NO_PRIORITY

See Also:

Constant Field Values

Method Detail

getComparator

KeyValue.KVComparator getComparator()

getStorefiles

Collection<StoreFile> getStorefiles()

close

Collection<StoreFile> close()
                     throws IOException

Close all the readers We don't need to worry about subsequent requests because the Region holds a write lock that will prevent any more reads or writes.

Returns:

the StoreFiles that were previously being used.

Throws:

IOException - on failure

getScanner

KeyValueScanner getScanner(Scan scan,
                           NavigableSet<byte[]> targetCols,
                           long readPt)
                    throws IOException

Return a scanner for both the memstore and the HStore files. Assumes we are not in a compaction.

Parameters:

scan - Scan to apply when scanning the stores

targetCols - columns to scan

Returns:

a scanner over the current key values

Throws:

IOException - on failure

getScanners

List<KeyValueScanner> getScanners(boolean cacheBlocks,
                                  boolean isGet,
                                  boolean usePread,
                                  boolean isCompaction,
                                  ScanQueryMatcher matcher,
                                  byte[] startRow,
                                  byte[] stopRow,
                                  long readPt)
                           throws IOException

Get all scanners with no filtering based on TTL (that happens further down the line).

Parameters:

cacheBlocks -

isGet -

usePread -

isCompaction -

matcher -

startRow -

stopRow -

readPt -

Returns:

all scanners for this store

Throws:

IOException

getScanInfo

ScanInfo getScanInfo()

upsert

long upsert(Iterable<Cell> cells,
            long readpoint)
     throws IOException

Adds or replaces the specified KeyValues.

For each KeyValue specified, if a cell with the same row, family, and qualifier exists in MemStore, it will be replaced. Otherwise, it will just be inserted to MemStore.

This operation is atomic on each KeyValue (row/family/qualifier) but not necessarily atomic across all of them.

Parameters:

cells -

readpoint - readpoint below which we can safely remove duplicate KVs

Returns:

memstore size delta

Throws:

IOException

add

Pair<Long,Cell> add(Cell cell)

Adds a value to the memstore

Parameters:

cell -

Returns:

memstore size delta & newly added KV which maybe different than the passed in KV

timeOfOldestEdit

long timeOfOldestEdit()

When was the last edit done in the memstore

rollback

void rollback(Cell cell)

Removes a Cell from the memstore. The Cell is removed only if its key & memstoreTS match the key & memstoreTS value of the cell parameter.

Parameters:

cell -

getRowKeyAtOrBefore

Cell getRowKeyAtOrBefore(byte[] row)
                  throws IOException

Find the key that matches row exactly, or the one that immediately precedes it. WARNING: Only use this method on a table where writes occur with strictly increasing timestamps. This method assumes this pattern of writes in order to make it reasonably performant. Also our search is dependent on the axiom that deletes are for cells that are in the container that follows whether a memstore snapshot or a storefile, not for the current container: i.e. we'll see deletes before we come across cells we are to delete. Presumption is that the memstore#kvset is processed before memstore#snapshot and so on.

Parameters:

row - The row key of the targeted row.

Returns:

Found Cell or null if none found.

Throws:

IOException

getFileSystem

FileSystem getFileSystem()

createWriterInTmp

StoreFile.Writer createWriterInTmp(long maxKeyCount,
                                   Compression.Algorithm compression,
                                   boolean isCompaction,
                                   boolean includeMVCCReadpoint,
                                   boolean includesTags)
                            throws IOException

Throws:

IOException

throttleCompaction

boolean throttleCompaction(long compactionSize)

getCompactionProgress

CompactionProgress getCompactionProgress()

getter for CompactionProgress object

Returns:

CompactionProgress object; can be null

requestCompaction

CompactionContext requestCompaction()
                             throws IOException

Throws:

IOException

requestCompaction

CompactionContext requestCompaction(int priority,
                                    CompactionRequest baseRequest)
                             throws IOException

Throws:

IOException

cancelRequestedCompaction

void cancelRequestedCompaction(CompactionContext compaction)

compact

List<StoreFile> compact(CompactionContext compaction,
                        CompactionThroughputController throughputController)
                 throws IOException

Throws:

IOException

isMajorCompaction

boolean isMajorCompaction()
                   throws IOException

Returns:

true if we should run a major compaction.

Throws:

IOException

triggerMajorCompaction

void triggerMajorCompaction()

needsCompaction

boolean needsCompaction()

See if there's too much store files in this store

Returns:

true if number of store files is greater than the number defined in minFilesToCompact

getCompactPriority

int getCompactPriority()

createFlushContext

org.apache.hadoop.hbase.regionserver.StoreFlushContext createFlushContext(long cacheFlushId)

replayCompactionMarker

void replayCompactionMarker(WALProtos.CompactionDescriptor compaction,
                            boolean pickCompactionFiles,
                            boolean removeFiles)
                     throws IOException

Call to complete a compaction. Its for the case where we find in the WAL a compaction that was not finished. We could find one recovering a WAL after a regionserver crash. See HBASE-2331.

Parameters:

compaction - the descriptor for compaction

pickCompactionFiles - whether or not pick up the new compaction output files and add it to the store

removeFiles - whether to remove/archive files from filesystem

Throws:

IOException

canSplit

boolean canSplit()

getSplitPoint

byte[] getSplitPoint()

Determines if Store should be split

Returns:

byte[] if store should be split, null otherwise.

assertBulkLoadHFileOk

void assertBulkLoadHFileOk(Path srcPath)
                    throws IOException

This throws a WrongRegionException if the HFile does not fit in this region, or an InvalidHFileException if the HFile is not valid.

Throws:

IOException

bulkLoadHFile

Path bulkLoadHFile(String srcPathStr,
                   long sequenceId)
            throws IOException

This method should only be called from Region. It is assumed that the ranges of values in the HFile fit within the stores assigned region. (assertBulkLoadHFileOk checks this)

Parameters:

srcPathStr -

sequenceId - sequence Id associated with the HFile

Throws:

IOException

hasReferences

boolean hasReferences()

Returns:

true if the store has any underlying reference files to older HFiles

getMemStoreSize

long getMemStoreSize()

Returns:

The size of this store's memstore, in bytes

getFlushableSize

long getFlushableSize()

Returns:

The amount of memory we could flush from this memstore; usually this is equal to getMemStoreSize() unless we are carrying snapshots and then it will be the size of outstanding snapshots.

getSnapshotSize

long getSnapshotSize()

Returns the memstore snapshot size

Returns:

size of the memstore snapshot

getFamily

HColumnDescriptor getFamily()

getMaxSequenceId

long getMaxSequenceId()

Returns:

The maximum sequence id in all store files.

getMaxMemstoreTS

long getMaxMemstoreTS()

Returns:

The maximum memstoreTS in all store files.

getDataBlockEncoder

HFileDataBlockEncoder getDataBlockEncoder()

Returns:

the data block encoder

getLastCompactSize

long getLastCompactSize()

Returns:

aggregate size of all HStores used in the last compaction

getSize

long getSize()

Returns:

aggregate size of HStore

getStorefilesCount

int getStorefilesCount()

Returns:

Count of store files

getStoreSizeUncompressed

long getStoreSizeUncompressed()

Returns:

The size of the store files, in bytes, uncompressed.

getStorefilesSize

long getStorefilesSize()

Returns:

The size of the store files, in bytes.

getStorefilesIndexSize

long getStorefilesIndexSize()

Returns:

The size of the store file indexes, in bytes.

getTotalStaticIndexSize

long getTotalStaticIndexSize()

Returns the total size of all index blocks in the data block indexes, including the root level, intermediate levels, and the leaf level for multi-level indexes, or just the root level for single-level indexes.

Returns:

the total size of block indexes in the store

getTotalStaticBloomSize

long getTotalStaticBloomSize()

Returns the total byte size of all Bloom filter bit arrays. For compound Bloom filters even the Bloom blocks currently not loaded into the block cache are counted.

Returns:

the total size of all Bloom filters in the store

getCacheConfig

CacheConfig getCacheConfig()

Used for tests.

Returns:

cache configuration for this Store.

getRegionInfo

HRegionInfo getRegionInfo()

Returns:

the parent region info hosting this store

getCoprocessorHost

RegionCoprocessorHost getCoprocessorHost()

areWritesEnabled

boolean areWritesEnabled()

getSmallestReadPoint

long getSmallestReadPoint()

Returns:

The smallest mvcc readPoint across all the scanners in this region. Writes older than this readPoint, are included in every read operation.

getColumnFamilyName

String getColumnFamilyName()

getTableName

TableName getTableName()

getFlushedCellsCount

long getFlushedCellsCount()

Returns:

The number of cells flushed to disk

getFlushedCellsSize

long getFlushedCellsSize()

Returns:

The total size of data flushed to disk, in bytes

getCompactedCellsCount

long getCompactedCellsCount()

Returns:

The number of cells processed during minor compactions

getCompactedCellsSize

long getCompactedCellsSize()

Returns:

The total amount of data processed during minor compactions, in bytes

getMajorCompactedCellsCount

long getMajorCompactedCellsCount()

Returns:

The number of cells processed during major compactions

getMajorCompactedCellsSize

long getMajorCompactedCellsSize()

Returns:

The total amount of data processed during major compactions, in bytes

addChangedReaderObserver

void addChangedReaderObserver(ChangedReadersObserver o)

deleteChangedReaderObserver

void deleteChangedReaderObserver(ChangedReadersObserver o)

hasTooManyStoreFiles

boolean hasTooManyStoreFiles()

Returns:

Whether this store has too many store files.

refreshStoreFiles

void refreshStoreFiles()
                throws IOException

Checks the underlying store files, and opens the files that have not been opened, and removes the store file readers for store files no longer available. Mainly used by secondary region replicas to keep up to date with the primary region files.

Throws:

IOException

getCompactionPressure

double getCompactionPressure()

This value can represent the degree of emergency of compaction for this store. It should be greater than or equal to 0.0, any value greater than 1.0 means we have too many store files.

• if getStorefilesCount <= getMinFilesToCompact, return 0.0
• return (getStorefilesCount - getMinFilesToCompact) / (blockingFileCount - getMinFilesToCompact)

And for striped stores, we should calculate this value by the files in each stripe separately and return the maximum value.

It is similar to getCompactPriority() except that it is more suitable to use in a linear formula.

refreshStoreFiles

void refreshStoreFiles(Collection<String> newFiles)
                throws IOException

Replaces the store files that the store has with the given files. Mainly used by secondary region replicas to keep up to date with the primary region files.

Throws:

IOException

bulkLoadHFile

void bulkLoadHFile(StoreFileInfo fileInfo)
            throws IOException

Throws:

IOException

isPrimaryReplicaStore

boolean isPrimaryReplicaStore()