public abstract class MultiTableInputFormatBase
extends <any>
MultiTableInputFormat
s. Receives a list of
Scan
instances that define the input tables and
filters etc. Subclasses may use other TableRecordReader implementations.Constructor and Description |
---|
MultiTableInputFormatBase() |
Modifier and Type | Method and Description |
---|---|
<any> |
createRecordReader(InputSplit split,
TaskAttemptContext context)
Builds a TableRecordReader.
|
protected java.util.List<Scan> |
getScans()
Allows subclasses to get the list of
Scan objects. |
java.util.List<InputSplit> |
getSplits(JobContext context)
Calculates the splits that will serve as input for the map tasks.
|
protected boolean |
includeRegionInSplit(byte[] startKey,
byte[] endKey)
Test if the given region is to be included in the InputSplit while
splitting the regions of a table.
|
protected void |
setScans(java.util.List<Scan> scans)
Allows subclasses to set the list of
Scan objects. |
protected void |
setTableRecordReader(TableRecordReader tableRecordReader)
Allows subclasses to set the
TableRecordReader . |
public <any> createRecordReader(InputSplit split, TaskAttemptContext context) throws java.io.IOException, java.lang.InterruptedException
split
- The split to work with.context
- The current context.java.io.IOException
- When creating the reader fails.java.lang.InterruptedException
- when record reader initialization failsorg.apache.hadoop.mapreduce.InputFormat#createRecordReader(
org.apache.hadoop.mapreduce.InputSplit,
org.apache.hadoop.mapreduce.TaskAttemptContext)
public java.util.List<InputSplit> getSplits(JobContext context) throws java.io.IOException
context
- The current job context.java.io.IOException
- When creating the list of splits fails.org.apache.hadoop.mapreduce.InputFormat#getSplits(org.apache.hadoop.mapreduce.JobContext)
protected boolean includeRegionInSplit(byte[] startKey, byte[] endKey)
This optimization is effective when there is a specific reasoning to
exclude an entire region from the M-R job, (and hence, not contributing to
the InputSplit), given the start and end keys of the same.
Useful when we need to remember the last-processed top record and revisit
the [last, current) interval for M-R processing, continuously. In addition
to reducing InputSplits, reduces the load on the region server as well, due
to the ordering of the keys.
Note: It is possible that endKey.length() == 0
, for the last
(recent) region.
Override this method, if you want to bulk exclude regions altogether from
M-R. By default, no region is excluded( i.e. all regions are included).
startKey
- Start key of the regionendKey
- End key of the regionprotected java.util.List<Scan> getScans()
Scan
objects.protected void setScans(java.util.List<Scan> scans)
Scan
objects.scans
- The list of Scan
used to define the inputprotected void setTableRecordReader(TableRecordReader tableRecordReader)
TableRecordReader
.tableRecordReader
- A different TableRecordReader
implementation.