HFile
and HFile
BlockCache
.See: Description
Interface | Description |
---|---|
BlockCache |
Block cache interface.
|
Cacheable |
Cacheable is an interface that allows for an object to be cached.
|
CacheableDeserializer<T extends Cacheable> |
Interface for a deserializer.
|
CachedBlock | |
HFile.CachingBlockReader |
An abstraction used by the block index.
|
HFile.Reader |
An interface used by clients to open and iterate an
HFile . |
HFile.Writer |
API required to write an
HFile |
HFileBlock.BlockIterator |
An interface allowing to iterate
HFileBlock s. |
HFileBlock.BlockWritable |
Something that can be written into a block.
|
HFileBlock.FSReader |
A full-fledged reader with iteration ability.
|
HFileDataBlockEncoder |
Controls what kind of data block encoding is used.
|
HFileScanner |
A scanner allows you to position yourself within a HFile and
scan through it.
|
InlineBlockWriter |
A way to write "inline" blocks into an
HFile . |
RandomDistribution.DiscreteRNG |
Interface for discrete (integer) random distributions.
|
ResizableBlockCache |
BlockCache which is resizable.
|
Class | Description |
---|---|
AbstractHFileReader |
Common functionality needed by all versions of
HFile readers. |
AbstractHFileReader.Scanner | |
AbstractHFileWriter |
Common functionality needed by all versions of
HFile writers. |
AgeSnapshot |
Snapshot of block cache age in cache.
|
BlockCacheKey |
Cache Key for use with implementations of
BlockCache |
BlockCacheUtil |
Utilty for aggregating counts in CachedBlocks and toString/toJSON CachedBlocks and BlockCaches.
|
BlockCacheUtil.CachedBlocksByFile |
Use one of these to keep a running account of cached blocks by file.
|
BlockWithScanInfo |
BlockWithScanInfo is wrapper class for HFileBlock with other attributes.
|
CacheableDeserializerIdManager |
This class is used to manage the identifiers for
CacheableDeserializer |
CacheConfig |
Stores all of the cache objects and configuration for a single HFile.
|
CacheStats |
Class that implements cache metrics.
|
CacheTestUtils | |
CacheTestUtils.ByteArrayCacheable | |
ChecksumUtil |
Utility methods to compute and validate checksums.
|
CombinedBlockCache |
CombinedBlockCache is an abstraction layer that combines
LruBlockCache and BucketCache . |
CombinedBlockCache.CombinedCacheStats | |
FixedFileTrailer |
The
HFile has a fixed trailer which contains offsets to other
variable parts of the file. |
HFile |
File format for hbase.
|
HFile.FileInfo |
Metadata for this file.
|
HFile.WriterFactory |
This variety of ways to construct writers is used throughout the code, and
we want to be able to swap writer implementations.
|
HFileBlock |
Reading
HFile version 1 and 2 blocks, and writing version 2 blocks. |
HFileBlock.Writer |
Unified version 2
HFile block writer. |
HFileBlock.Writer.BufferGrabbingByteArrayOutputStream | |
HFileBlockIndex |
Provides functionality to write (
HFileBlockIndex.BlockIndexWriter ) and read
(HFileBlockIndex.BlockIndexReader ) single-level and multi-level block indexes. |
HFileBlockIndex.BlockIndexReader |
The reader will always hold the root level index in the memory.
|
HFileBlockIndex.BlockIndexWriter |
Writes the block index into the output stream.
|
HFileContext |
This carries the information on some of the meta data about the HFile.
|
HFileContextBuilder |
A builder that helps in building up the HFileContext
|
HFileDataBlockEncoderImpl |
Do different kinds of data block encoding according to column family
options.
|
HFilePrettyPrinter |
Implements pretty-printing functionality for
HFile s. |
HFileReaderV2 |
HFile reader for version 2. |
HFileReaderV2.AbstractScannerV2 | |
HFileReaderV2.EncodedScannerV2 |
ScannerV2 that operates on encoded data blocks.
|
HFileReaderV2.ScannerV2 |
Implementation of
HFileScanner interface. |
HFileReaderV3 |
HFile reader for version 3. |
HFileReaderV3.EncodedScannerV3 |
ScannerV3 that operates on encoded data blocks.
|
HFileReaderV3.ScannerV3 |
Implementation of
HFileScanner interface. |
HFileUtil | |
HFileWriterV2 |
Writes HFile format version 2.
|
HFileWriterV3 |
HFile writer for version 3. |
InclusiveCombinedBlockCache | |
LruBlockCache |
A block cache implementation that is memory-aware using
HeapSize ,
memory-bound using an LRU eviction algorithm, and concurrent: backed by a
ConcurrentHashMap and with a non-blocking eviction thread giving
constant-time LruBlockCache.cacheBlock(org.apache.hadoop.hbase.io.hfile.BlockCacheKey, org.apache.hadoop.hbase.io.hfile.Cacheable, boolean, boolean) and LruBlockCache.getBlock(org.apache.hadoop.hbase.io.hfile.BlockCacheKey, boolean, boolean, boolean) operations. |
LruCachedBlock |
Represents an entry in the
LruBlockCache . |
LruCachedBlockQueue |
A memory-bound queue that will grow until an element brings
total size >= maxSize.
|
MemcachedBlockCache |
Class to store blocks into memcached.
|
NanoTimer |
A nano-second timer.
|
NoOpDataBlockEncoder |
Does not perform any kind of encoding/decoding.
|
PrefetchExecutor | |
RandomDistribution |
A class that generates random numbers that follow some distribution.
|
RandomDistribution.Binomial |
Binomial distribution.
|
RandomDistribution.Flat |
P(i)=1/(max-min)
|
RandomDistribution.Zipf |
Zipf distribution.
|
RandomKeyValueUtil |
These helper methods generate random byte[]'s data for KeyValues
|
TestBlockCacheReporting | |
TestCacheConfig |
Tests that
CacheConfig does as expected. |
TestCachedBlockQueue | |
TestCacheOnWrite |
Tests
HFile cache-on-write functionality for the following block
types: data blocks, non-root index blocks, and Bloom filter blocks. |
TestChecksum | |
TestFixedFileTrailer | |
TestForceCacheImportantBlocks |
Make sure we always cache important block types, such as index blocks, as
long as we have a block cache, even though block caching might be disabled
for the column family.
|
TestHFile |
test hfile features.
|
TestHFileBlock | |
TestHFileBlockCompatibility |
This class has unit tests to prove that older versions of
HFiles (without checksums) are compatible with current readers.
|
TestHFileBlockCompatibility.Writer |
This is the version of the HFileBlock.Writer that is used to
create V2 blocks with minor version 0.
|
TestHFileBlockIndex | |
TestHFileBlockPositionalRead |
Unit test suite covering HFileBlock positional read logic.
|
TestHFileDataBlockEncoder | |
TestHFileEncryption | |
TestHFileInlineToRootChunkConversion |
Test a case when an inline index chunk is converted to a root one.
|
TestHFileSeek |
test the performance for seek.
|
TestHFileWriterV2 |
Testing writing a version 2
HFile . |
TestHFileWriterV3 |
Testing writing a version 3
HFile . |
TestLazyDataBlockDecompression | |
TestLruBlockCache |
Tests the concurrent LruBlockCache.
|
TestLruCachedBlock | |
TestPrefetch | |
TestReseekTo | |
TestScannerSelectionUsingKeyRange |
Test the optimization that does not scan files where all key ranges are excluded.
|
TestScannerSelectionUsingTTL |
Test the optimization that does not scan files where all timestamps are
expired.
|
TestSeekBeforeWithInlineBlocks | |
TestSeekTo |
Test
HFileScanner.seekTo(byte[]) and its variants. |
Enum | Description |
---|---|
BlockPriority | |
BlockType |
Various types of HFile blocks.
|
BlockType.BlockCategory | |
TagUsage |
Used in testcases only.
|
Exception | Description |
---|---|
AbstractHFileReader.BlockIndexNotLoadedException | |
AbstractHFileReader.NotSeekedException |
An exception thrown when an operation requiring a scanner to be seeked
is invoked on a scanner that is not seeked.
|
CorruptHFileException |
This exception is thrown when attempts to read an HFile fail due to corruption or truncation
issues.
|
InvalidHFileException |
Thrown when an invalid HFile format is detected
|
HFile
and HFile
BlockCache
. Caches are configured (and instantiated)
by CacheConfig
. See head of the
CacheConfig
class for constants that define
cache options and configuration keys to use setting cache options. Cache implementations
include the default, native on-heap LruBlockCache
and a
BucketCache
that has a bunch of deploy formats
including acting as a L2 for LruBlockCache -- when a block is evicted from LruBlockCache, it
goes to the BucketCache and when we search a block, we look in both places -- or, the
most common deploy type,
using CombinedBlockCache
, BucketCache is used as
a host for data blocks with meta blocks in an instance of LruBlockCache. BucketCache
can also be onheap, offheap, and file-backed.
BucketCache
See the HBase Reference Guide Enable BucketCache.