org.apache.hadoop.hbase.regionserver

Interface Region

All Superinterfaces:

ConfigurationObserver

All Known Implementing Classes:

HRegion, TestAtomicOperation.MockHRegion, TestIOFencing.BlockCompactionsInCompletionRegion, TestIOFencing.BlockCompactionsInPrepRegion, TestIOFencing.CompactionBlockerRegion, TestRegionObserverScannerOpenHook.CompactionCompletionNotifyingRegion

@InterfaceAudience.LimitedPrivate(value="Coprocesssor")
 @InterfaceStability.Evolving
public interface Region
extends ConfigurationObserver

Regions store data for a certain region of a table. It stores all columns for each row. A given table consists of one or more Regions.

An Region is defined by its table and its key extent.

Locking at the Region level serves only one purpose: preventing the region from being closed (and consequently split) while other operations are ongoing. Each row level operation obtains both a row lock and a region read lock for the duration of the operation. While a scanner is being constructed, getScanner holds a read lock. If the scanner is successfully constructed, it holds a read lock until it is closed. A close takes out a write lock and consequently will block for ongoing operations and will block new operations from starting while the close is in progress.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	Region.BulkLoadListener Listener class to enable callers of bulkLoadHFile() to perform any necessary pre/post processing of a given bulkload call
static interface	Region.FlushResult
static class	Region.Operation Operation enum is used in startRegionOperation() to provide context for various checks before any region operation begins.
static interface	Region.RowLock Row lock held by a given thread.

Method Summary

Modifier and Type	Method and Description
Result	append(Append append, long nonceGroup, long nonce) Perform one or more append operations on a row.
OperationStatus[]	batchMutate(Mutation[] mutations, long nonceGroup, long nonce) Perform a batch of mutations.
OperationStatus[]	batchReplay(WALSplitter.MutationReplay[] mutations, long replaySeqId) Replay a batch of mutations.
boolean	bulkLoadHFiles(Collection<Pair<byte[],String>> familyPaths, boolean assignSeqId, Region.BulkLoadListener bulkLoadListener) Attempts to atomically load a group of hfiles.
boolean	checkAndMutate(byte[] row, byte[] family, byte[] qualifier, CompareFilter.CompareOp compareOp, ByteArrayComparable comparator, Mutation mutation, boolean writeToWAL) Atomically checks if a row/family/qualifier value matches the expected val If it does, it performs the row mutations.
boolean	checkAndRowMutate(byte[] row, byte[] family, byte[] qualifier, CompareFilter.CompareOp compareOp, ByteArrayComparable comparator, RowMutations mutations, boolean writeToWAL) Atomically checks if a row/family/qualifier value matches the expected val If it does, it performs the row mutations.
void	checkFamilies(Collection<byte[]> families) Check the collection of families for validity.
void	checkTimestamps(Map<byte[],List<Cell>> familyMap, long now) Check the collection of families for valid timestamps
void	closeRegionOperation() Closes the region operation lock.
void	compact(boolean majorCompaction) Synchronously compact all stores in the region.
void	delete(Delete delete) Deletes the specified cells/row.
Message	execService(RpcController controller, ClientProtos.CoprocessorServiceCall call) Executes a single protocol buffer coprocessor endpoint Service method using the registered protocol handlers.
Region.FlushResult	flush(boolean force) Flush the cache.
Result	get(Get get) Do a get based on the get parameter.
List<Cell>	get(Get get, boolean withCoprocessor) Do a get based on the get parameter.
long	getBlockedRequestsCount()
long	getCheckAndMutateChecksFailed()
long	getCheckAndMutateChecksPassed()
Result	getClosestRowBefore(byte[] row, byte[] family) Return all the data for the row that matches row exactly, or the one that immediately preceeds it, at or immediately before ts.
AdminProtos.GetRegionInfoResponse.CompactionState	getCompactionState()
RegionCoprocessorHost	getCoprocessorHost()
long	getDataInMemoryWithoutWAL()
long	getEarliestFlushTimeForAllStores()
HDFSBlocksDistribution	getHDFSBlocksDistribution()
long	getMaxFlushedSeqId()
Map<byte[],Long>	getMaxStoreSeqId()
long	getMemstoreSize()
MetricsRegion	getMetrics()
long	getNumMutationsWithoutWAL()
long	getOldestHfileTs(boolean majorCompactioOnly) This can be used to determine the last time all files of this region were major compacted.
long	getOldestSeqIdOfStore(byte[] familyName)
long	getOpenSeqNum()
long	getReadpoint(IsolationLevel isolationLevel)
long	getReadRequestsCount()
HRegionInfo	getRegionInfo()
Region.RowLock	getRowLock(byte[] row, boolean waitForLock) Tries to acquire a lock on the given row.
RegionScanner	getScanner(Scan scan) Return an iterator that scans over the HRegion, returning the indicated columns and rows specified by the Scan.
Store	getStore(byte[] family) Return the Store for the given family
List<String>	getStoreFileList(byte[][] columns)
List<Store>	getStores() Return the list of Stores managed by this region
HTableDescriptor	getTableDesc()
long	getWriteRequestsCount()
Result	increment(Increment increment, long nonceGroup, long nonce) Perform one or more increment operations on a row.
boolean	isAvailable()
boolean	isClosed()
boolean	isClosing()
boolean	isLoadingCfsOnDemandDefault()
boolean	isReadOnly()
boolean	isRecovering()
void	mutateRow(RowMutations mutations) Performs multiple mutations atomically on a single row.
void	mutateRowsWithLocks(Collection<Mutation> mutations, Collection<byte[]> rowsToLock, long nonceGroup, long nonce) Perform atomic mutations within the region.
void	prepareDelete(Delete delete) Prepare a delete for a row mutation processor
void	prepareDeleteTimestamps(Mutation mutation, Map<byte[],List<Cell>> familyCellMap, byte[] now) Set up correct timestamps in the KVs in Delete object.
void	processRowsWithLocks(RowProcessor<?,?> processor) Performs atomic multiple reads and writes on a given row.
void	processRowsWithLocks(RowProcessor<?,?> processor, long nonceGroup, long nonce) Performs atomic multiple reads and writes on a given row.
void	processRowsWithLocks(RowProcessor<?,?> processor, long timeout, long nonceGroup, long nonce) Performs atomic multiple reads and writes on a given row.
void	put(Put put) Puts some data in the table.
boolean	refreshStoreFiles() Check the region's underlying store files, open the files that have not been opened yet, and remove the store file readers for store files no longer available.
boolean	registerService(Service instance) Registers a new protocol buffer Service subclass as a coprocessor endpoint to be available for handling Region#execService(com.google.protobuf.RpcController, org.apache.hadoop.hbase.protobuf.generated.ClientProtos.CoprocessorServiceCall)} calls.
void	releaseRowLocks(List<Region.RowLock> rowLocks) If the given list of row locks is not null, releases all locks.
void	startRegionOperation() This method needs to be called before any public call that reads or modifies data.
void	startRegionOperation(Region.Operation op) This method needs to be called before any public call that reads or modifies data.
void	triggerMajorCompaction() Trigger major compaction on all stores in the region.
void	updateCellTimestamps(Iterable<List<Cell>> values, byte[] now) Replace any cell timestamps set to HConstants#LATEST_TIMESTAMP with the provided current timestamp.
void	updateReadRequestsCount(long i) Update the read request count for this region
void	updateWriteRequestsCount(long i) Update the write request count for this region
void	waitForFlushesAndCompactions() Wait for all current flushes and compactions of the region to complete

Methods inherited from interface org.apache.hadoop.hbase.conf.ConfigurationObserver

onConfigurationChange

Method Detail

getRegionInfo

HRegionInfo getRegionInfo()

Returns:

region information for this region

getTableDesc

HTableDescriptor getTableDesc()

Returns:

table descriptor for this region

isAvailable

boolean isAvailable()

Returns:

true if region is available (not closed and not closing)

isClosed

boolean isClosed()

Returns:

true if region is closed

isClosing

boolean isClosing()

Returns:

True if closing process has started

isRecovering

boolean isRecovering()

Returns:

True if region is in recovering state

isReadOnly

boolean isReadOnly()

Returns:

True if region is read only

getStores

List<Store> getStores()

Return the list of Stores managed by this region

Use with caution. Exposed for use of fixup utilities.

Returns:

a list of the Stores managed by this region

getStore

Store getStore(byte[] family)

Return the Store for the given family

Use with caution. Exposed for use of fixup utilities.

Returns:

the Store for the given family

getStoreFileList

List<String> getStoreFileList(byte[][] columns)

Returns:

list of store file names for the given families

refreshStoreFiles

boolean refreshStoreFiles()
                   throws IOException

Check the region's underlying store files, open the files that have not been opened yet, and remove the store file readers for store files no longer available.

Throws:

IOException

getOpenSeqNum

long getOpenSeqNum()

Returns:

the latest sequence number that was read from storage when this region was opened

getMaxFlushedSeqId

long getMaxFlushedSeqId()

Returns:

the max sequence id of flushed data on this region

getOldestSeqIdOfStore

long getOldestSeqIdOfStore(byte[] familyName)

Returns:

the oldest sequence id found in the store for the given family

getOldestHfileTs

long getOldestHfileTs(boolean majorCompactioOnly)
               throws IOException

This can be used to determine the last time all files of this region were major compacted.

Parameters:

majorCompactioOnly - Only consider HFile that are the result of major compaction

Returns:

the timestamp of the oldest HFile for all stores of this region

Throws:

IOException

getMaxStoreSeqId

Map<byte[],Long> getMaxStoreSeqId()

Returns:

map of column family names to max sequence id that was read from storage when this region was opened

isLoadingCfsOnDemandDefault

boolean isLoadingCfsOnDemandDefault()

Returns:

true if loading column families on demand by default

getReadpoint

long getReadpoint(IsolationLevel isolationLevel)

Returns:

readpoint considering given IsolationLevel

getEarliestFlushTimeForAllStores

long getEarliestFlushTimeForAllStores()

Returns:

The earliest time a store in the region was flushed. All other stores in the region would have been flushed either at, or after this time.

getReadRequestsCount

long getReadRequestsCount()

Returns:

read requests count for this region

updateReadRequestsCount

void updateReadRequestsCount(long i)

Update the read request count for this region

Parameters:

i - increment

getWriteRequestsCount

long getWriteRequestsCount()

Returns:

write request count for this region

updateWriteRequestsCount

void updateWriteRequestsCount(long i)

Update the write request count for this region

Parameters:

i - increment

getMemstoreSize

long getMemstoreSize()

Returns:

memstore size for this region, in bytes

getNumMutationsWithoutWAL

long getNumMutationsWithoutWAL()

Returns:

the number of mutations processed bypassing the WAL

getDataInMemoryWithoutWAL

long getDataInMemoryWithoutWAL()

Returns:

the size of data processed bypassing the WAL, in bytes

getBlockedRequestsCount

long getBlockedRequestsCount()

Returns:

the number of blocked requests

getCheckAndMutateChecksPassed

long getCheckAndMutateChecksPassed()

Returns:

the number of checkAndMutate guards that passed

getCheckAndMutateChecksFailed

long getCheckAndMutateChecksFailed()

Returns:

the number of failed checkAndMutate guards

getMetrics

MetricsRegion getMetrics()

Returns:

the MetricsRegion for this region

getHDFSBlocksDistribution

HDFSBlocksDistribution getHDFSBlocksDistribution()

Returns:

the block distribution for all Stores managed by this region

startRegionOperation

void startRegionOperation()
                   throws IOException

This method needs to be called before any public call that reads or modifies data. Acquires a read lock and checks if the region is closing or closed.

closeRegionOperation() MUST then always be called after the operation has completed, whether it succeeded or failed.

Throws:

IOException

startRegionOperation

void startRegionOperation(Region.Operation op)
                   throws IOException

This method needs to be called before any public call that reads or modifies data. Acquires a read lock and checks if the region is closing or closed.

closeRegionOperation() MUST then always be called after the operation has completed, whether it succeeded or failed.

Parameters:

op - The operation is about to be taken on the region

Throws:

IOException

closeRegionOperation

void closeRegionOperation()
                   throws IOException

Closes the region operation lock.

Throws:

IOException

getRowLock

Region.RowLock getRowLock(byte[] row,
                          boolean waitForLock)
                   throws IOException

Tries to acquire a lock on the given row.

Parameters:

waitForLock - if true, will block until the lock is available. Otherwise, just tries to obtain the lock and returns false if unavailable.

Returns:

the row lock if acquired, null if waitForLock was false and the lock was not acquired

Throws:

IOException - if waitForLock was true and the lock could not be acquired after waiting

releaseRowLocks

void releaseRowLocks(List<Region.RowLock> rowLocks)

If the given list of row locks is not null, releases all locks.

append

Result append(Append append,
              long nonceGroup,
              long nonce)
       throws IOException

Perform one or more append operations on a row.

Parameters:

append -

nonceGroup -

nonce -

Returns:

result of the operation

Throws:

IOException

batchMutate

OperationStatus[] batchMutate(Mutation[] mutations,
                              long nonceGroup,
                              long nonce)
                       throws IOException

Perform a batch of mutations.

Note this supports only Put and Delete mutations and will ignore other types passed.

Parameters:

mutations - the list of mutations

nonceGroup -

nonce -

Returns:

an array of OperationStatus which internally contains the OperationStatusCode and the exceptionMessage if any.

Throws:

IOException

batchReplay

OperationStatus[] batchReplay(WALSplitter.MutationReplay[] mutations,
                              long replaySeqId)
                       throws IOException

Replay a batch of mutations.

Parameters:

mutations - mutations to replay.

replaySeqId -

Returns:

an array of OperationStatus which internally contains the OperationStatusCode and the exceptionMessage if any.

Throws:

IOException

checkAndMutate

boolean checkAndMutate(byte[] row,
                       byte[] family,
                       byte[] qualifier,
                       CompareFilter.CompareOp compareOp,
                       ByteArrayComparable comparator,
                       Mutation mutation,
                       boolean writeToWAL)
                throws IOException

Atomically checks if a row/family/qualifier value matches the expected val If it does, it performs the row mutations. If the passed value is null, t is for the lack of column (ie: non-existence)

Parameters:

row - to check

family - column family to check

qualifier - column qualifier to check

compareOp - the comparison operator

comparator -

mutation -

writeToWAL -

Returns:

true if mutation was applied, false otherwise

Throws:

IOException

checkAndRowMutate

boolean checkAndRowMutate(byte[] row,
                          byte[] family,
                          byte[] qualifier,
                          CompareFilter.CompareOp compareOp,
                          ByteArrayComparable comparator,
                          RowMutations mutations,
                          boolean writeToWAL)
                   throws IOException

Atomically checks if a row/family/qualifier value matches the expected val If it does, it performs the row mutations. If the passed value is null, t is for the lack of column (ie: non-existence)

Parameters:

row - to check

family - column family to check

qualifier - column qualifier to check

compareOp - the comparison operator

comparator -

mutations -

writeToWAL -

Returns:

true if mutation was applied, false otherwise

Throws:

IOException

delete

void delete(Delete delete)
     throws IOException

Deletes the specified cells/row.

Parameters:

delete -

Throws:

IOException

get

Result get(Get get)
    throws IOException

Do a get based on the get parameter.

Parameters:

get - query parameters

Returns:

result of the operation

Throws:

IOException

get

List<Cell> get(Get get,
               boolean withCoprocessor)
        throws IOException

Do a get based on the get parameter.

Parameters:

get - query parameters

withCoprocessor - invoke coprocessor or not. We don't want to always invoke cp.

Returns:

list of cells resulting from the operation

Throws:

IOException

getClosestRowBefore

Result getClosestRowBefore(byte[] row,
                           byte[] family)
                    throws IOException

Return all the data for the row that matches row exactly, or the one that immediately preceeds it, at or immediately before ts.

Parameters:

row -

family -

Returns:

result of the operation

Throws:

IOException

getScanner

RegionScanner getScanner(Scan scan)
                  throws IOException

Return an iterator that scans over the HRegion, returning the indicated columns and rows specified by the Scan.

This Iterator must be closed by the caller.

Parameters:

scan - configured Scan

Returns:

RegionScanner

Throws:

IOException - read exceptions

increment

Result increment(Increment increment,
                 long nonceGroup,
                 long nonce)
          throws IOException

Perform one or more increment operations on a row.

Parameters:

increment -

nonceGroup -

nonce -

Returns:

result of the operation

Throws:

IOException

mutateRow

void mutateRow(RowMutations mutations)
        throws IOException

Performs multiple mutations atomically on a single row. Currently Put and Delete are supported.

Parameters:

mutations - object that specifies the set of mutations to perform atomically

Throws:

IOException

mutateRowsWithLocks

void mutateRowsWithLocks(Collection<Mutation> mutations,
                         Collection<byte[]> rowsToLock,
                         long nonceGroup,
                         long nonce)
                  throws IOException

Perform atomic mutations within the region.

Parameters:

mutations - The list of mutations to perform. mutations can contain operations for multiple rows. Caller has to ensure that all rows are contained in this region.

rowsToLock - Rows to lock

nonceGroup - Optional nonce group of the operation (client Id)

nonce - Optional nonce of the operation (unique random id to ensure "more idempotence") If multiple rows are locked care should be taken that rowsToLock is sorted in order to avoid deadlocks.

Throws:

IOException

processRowsWithLocks

void processRowsWithLocks(RowProcessor<?,?> processor)
                   throws IOException

Performs atomic multiple reads and writes on a given row.

Parameters:

processor - The object defines the reads and writes to a row.

Throws:

IOException

processRowsWithLocks

void processRowsWithLocks(RowProcessor<?,?> processor,
                          long nonceGroup,
                          long nonce)
                   throws IOException

Performs atomic multiple reads and writes on a given row.

Parameters:

processor - The object defines the reads and writes to a row.

nonceGroup - Optional nonce group of the operation (client Id)

nonce - Optional nonce of the operation (unique random id to ensure "more idempotence")

Throws:

IOException

processRowsWithLocks

void processRowsWithLocks(RowProcessor<?,?> processor,
                          long timeout,
                          long nonceGroup,
                          long nonce)
                   throws IOException

Performs atomic multiple reads and writes on a given row.

Parameters:

processor - The object defines the reads and writes to a row.

timeout - The timeout of the processor.process() execution Use a negative number to switch off the time bound

nonceGroup - Optional nonce group of the operation (client Id)

nonce - Optional nonce of the operation (unique random id to ensure "more idempotence")

Throws:

IOException

put

void put(Put put)
  throws IOException

Puts some data in the table.

Parameters:

put -

Throws:

IOException

bulkLoadHFiles

boolean bulkLoadHFiles(Collection<Pair<byte[],String>> familyPaths,
                       boolean assignSeqId,
                       Region.BulkLoadListener bulkLoadListener)
                throws IOException

Attempts to atomically load a group of hfiles. This is critical for loading rows with multiple column families atomically.

Parameters:

familyPaths - List of Pair

bulkLoadListener - Internal hooks enabling massaging/preparation of a file about to be bulk loaded

assignSeqId -

Returns:

true if successful, false if failed recoverably

Throws:

IOException - if failed unrecoverably.

getCoprocessorHost

RegionCoprocessorHost getCoprocessorHost()

Returns:

the coprocessor host

execService

Message execService(RpcController controller,
                    ClientProtos.CoprocessorServiceCall call)
             throws IOException

Executes a single protocol buffer coprocessor endpoint Service method using the registered protocol handlers. Service implementations must be registered via the Region#registerService(com.google.protobuf.Service) method before they are available.

Parameters:

controller - an RpcContoller implementation to pass to the invoked service

call - a CoprocessorServiceCall instance identifying the service, method, and parameters for the method invocation

Returns:

a protocol buffer Message instance containing the method's result

Throws:

IOException - if no registered service handler is found or an error occurs during the invocation

See Also:

org.apache.hadoop.hbase.regionserver.Region#registerService(com.google.protobuf.Service)

registerService

boolean registerService(Service instance)

Registers a new protocol buffer Service subclass as a coprocessor endpoint to be available for handling Region#execService(com.google.protobuf.RpcController, org.apache.hadoop.hbase.protobuf.generated.ClientProtos.CoprocessorServiceCall)} calls.

Only a single instance may be registered per region for a given Service subclass (the instances are keyed on com.google.protobuf.Descriptors.ServiceDescriptor#getFullName(). After the first registration, subsequent calls with the same service name will fail with a return value of false.

Parameters:

instance - the Service subclass instance to expose as a coprocessor endpoint

Returns:

true if the registration was successful, false otherwise

checkFamilies

void checkFamilies(Collection<byte[]> families)
            throws NoSuchColumnFamilyException

Check the collection of families for validity.

Parameters:

families -

Throws:

NoSuchColumnFamilyException

checkTimestamps

void checkTimestamps(Map<byte[],List<Cell>> familyMap,
                     long now)
              throws FailedSanityCheckException

Check the collection of families for valid timestamps

Parameters:

familyMap -

now - current timestamp

Throws:

FailedSanityCheckException

prepareDelete

void prepareDelete(Delete delete)
            throws IOException

Prepare a delete for a row mutation processor

Parameters:

delete - The passed delete is modified by this method. WARNING!

Throws:

IOException

prepareDeleteTimestamps

void prepareDeleteTimestamps(Mutation mutation,
                             Map<byte[],List<Cell>> familyCellMap,
                             byte[] now)
                      throws IOException

Set up correct timestamps in the KVs in Delete object.

Caller should have the row and region locks.

Parameters:

mutation -

familyCellMap -

now -

Throws:

IOException

updateCellTimestamps

void updateCellTimestamps(Iterable<List<Cell>> values,
                          byte[] now)
                   throws IOException

Replace any cell timestamps set to HConstants#LATEST_TIMESTAMP with the provided current timestamp.

Parameters:

values -

now -

Throws:

IOException

flush

Region.FlushResult flush(boolean force)
                  throws IOException

Flush the cache.

When this method is called the cache will be flushed unless:

1. the cache is empty
2. the region is closed.
3. a flush is already in progress
4. writes are disabled

This method may block for some time, so it should not be called from a time-sensitive thread.

Parameters:

force - whether we want to force a flush of all stores

Returns:

FlushResult indicating whether the flush was successful or not and if the region needs compacting

Throws:

IOException - general io exceptions

DroppedSnapshotException - Thrown when abort is required. The caller MUST catch this exception and MUST abort. Any further operation to the region may cause data loss. because a snapshot was not properly persisted.

compact

void compact(boolean majorCompaction)
      throws IOException

Synchronously compact all stores in the region.

This operation could block for a long time, so don't call it from a time-sensitive thread.

Note that no locks are taken to prevent possible conflicts between compaction and splitting activities. The regionserver does not normally compact and split in parallel. However by calling this method you may introduce unexpected and unhandled concurrency. Don't do this unless you know what you are doing.

Parameters:

majorCompaction - True to force a major compaction regardless of thresholds

Throws:

IOException

triggerMajorCompaction

void triggerMajorCompaction()
                     throws IOException

Trigger major compaction on all stores in the region.

Compaction will be performed asynchronously to this call by the RegionServer's CompactSplitThread. See also Store.triggerMajorCompaction()

Throws:

IOException

getCompactionState

AdminProtos.GetRegionInfoResponse.CompactionState getCompactionState()

Returns:

if a given region is in compaction now.

waitForFlushesAndCompactions

void waitForFlushesAndCompactions()

Wait for all current flushes and compactions of the region to complete