public class TableSnapshotInputFormatImpl
extends java.lang.Object
Modifier and Type | Class and Description |
---|---|
static class |
TableSnapshotInputFormatImpl.InputSplit
Implementation class for InputSplit logic common between mapred and mapreduce.
|
static class |
TableSnapshotInputFormatImpl.RecordReader
Implementation class for RecordReader logic common between mapred and mapreduce.
|
Modifier and Type | Field and Description |
---|---|
static Logger |
LOG |
static java.lang.String |
NUM_SPLITS_PER_REGION
For MapReduce jobs running multiple mappers per region, determines
number of splits to generate per region.
|
protected static java.lang.String |
RESTORE_DIR_KEY |
static boolean |
SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_DEFAULT |
static java.lang.String |
SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_KEY
Whether to calculate the block location for splits.
|
static java.lang.String |
SPLIT_ALGO
For MapReduce jobs running multiple mappers per region, determines
what split algorithm we should be using to find split points for scanners.
|
Constructor and Description |
---|
TableSnapshotInputFormatImpl() |
Modifier and Type | Method and Description |
---|---|
static Scan |
extractScanFromConf(Configuration conf) |
static java.util.List<java.lang.String> |
getBestLocations(Configuration conf,
HDFSBlocksDistribution blockDistribution) |
static java.util.List<HRegionInfo> |
getRegionInfosFromManifest(SnapshotManifest manifest) |
static SnapshotManifest |
getSnapshotManifest(Configuration conf,
java.lang.String snapshotName,
Path rootDir,
FileSystem fs) |
static RegionSplitter.SplitAlgorithm |
getSplitAlgo(Configuration conf) |
static java.util.List<TableSnapshotInputFormatImpl.InputSplit> |
getSplits(Configuration conf) |
static java.util.List<TableSnapshotInputFormatImpl.InputSplit> |
getSplits(Scan scan,
SnapshotManifest manifest,
java.util.List<HRegionInfo> regionManifests,
Path restoreDir,
Configuration conf) |
static java.util.List<TableSnapshotInputFormatImpl.InputSplit> |
getSplits(Scan scan,
SnapshotManifest manifest,
java.util.List<HRegionInfo> regionManifests,
Path restoreDir,
Configuration conf,
RegionSplitter.SplitAlgorithm sa,
int numSplits) |
static void |
setInput(Configuration conf,
java.lang.String snapshotName,
Path restoreDir)
Configures the job to use TableSnapshotInputFormat to read from a snapshot.
|
static void |
setInput(Configuration conf,
java.lang.String snapshotName,
Path restoreDir,
RegionSplitter.SplitAlgorithm splitAlgo,
int numSplitsPerRegion)
Configures the job to use TableSnapshotInputFormat to read from a snapshot.
|
public static final Logger LOG
protected static final java.lang.String RESTORE_DIR_KEY
public static final java.lang.String SPLIT_ALGO
public static final java.lang.String NUM_SPLITS_PER_REGION
public static final java.lang.String SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_KEY
public static final boolean SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_DEFAULT
public static java.util.List<TableSnapshotInputFormatImpl.InputSplit> getSplits(Configuration conf) throws java.io.IOException
java.io.IOException
public static RegionSplitter.SplitAlgorithm getSplitAlgo(Configuration conf) throws java.io.IOException
java.io.IOException
public static java.util.List<HRegionInfo> getRegionInfosFromManifest(SnapshotManifest manifest)
public static SnapshotManifest getSnapshotManifest(Configuration conf, java.lang.String snapshotName, Path rootDir, FileSystem fs) throws java.io.IOException
java.io.IOException
public static Scan extractScanFromConf(Configuration conf) throws java.io.IOException
java.io.IOException
public static java.util.List<TableSnapshotInputFormatImpl.InputSplit> getSplits(Scan scan, SnapshotManifest manifest, java.util.List<HRegionInfo> regionManifests, Path restoreDir, Configuration conf) throws java.io.IOException
java.io.IOException
public static java.util.List<TableSnapshotInputFormatImpl.InputSplit> getSplits(Scan scan, SnapshotManifest manifest, java.util.List<HRegionInfo> regionManifests, Path restoreDir, Configuration conf, RegionSplitter.SplitAlgorithm sa, int numSplits) throws java.io.IOException
java.io.IOException
public static java.util.List<java.lang.String> getBestLocations(Configuration conf, HDFSBlocksDistribution blockDistribution)
public static void setInput(Configuration conf, java.lang.String snapshotName, Path restoreDir) throws java.io.IOException
conf
- the job to configurationsnapshotName
- the name of the snapshot to read fromrestoreDir
- a temporary directory to restore the snapshot into. Current user should have
write permissions to this directory, and this should not be a subdirectory of rootdir.
After the job is finished, restoreDir can be deleted.java.io.IOException
- if an error occurspublic static void setInput(Configuration conf, java.lang.String snapshotName, Path restoreDir, RegionSplitter.SplitAlgorithm splitAlgo, int numSplitsPerRegion) throws java.io.IOException
conf
- the job to configuresnapshotName
- the name of the snapshot to read fromrestoreDir
- a temporary directory to restore the snapshot into. Current user should
have write permissions to this directory, and this should not be a subdirectory of rootdir.
After the job is finished, restoreDir can be deleted.numSplitsPerRegion
- how many input splits to generate per one regionsplitAlgo
- SplitAlgorithm to be used when generating InputSplitsjava.io.IOException
- if an error occurs