public class Import
extends Configured
Export
.Modifier and Type | Class and Description |
---|---|
static class |
Import.CellImporter
A mapper that just writes out KeyValues.
|
static class |
Import.CellReducer |
static class |
Import.CellSortImporter |
static class |
Import.CellWritableComparable |
static class |
Import.CellWritableComparablePartitioner |
static class |
Import.Importer
Write table content out to files in hdfs.
|
static class |
Import.KeyValueImporter
Deprecated.
Use
Import.CellImporter . Will be removed from 3.0 onwards |
static class |
Import.KeyValueReducer
Deprecated.
Use
Import.CellReducer . Will be removed from 3.0 onwards |
static class |
Import.KeyValueSortImporter
Deprecated.
Use
Import.CellSortImporter . Will be removed from 3.0 onwards |
static class |
Import.KeyValueWritableComparable |
static class |
Import.KeyValueWritableComparablePartitioner
Deprecated.
Use
Import.CellWritableComparablePartitioner . Will be removed
from 3.0 onwards |
Modifier and Type | Field and Description |
---|---|
static java.lang.String |
BULK_OUTPUT_CONF_KEY |
static java.lang.String |
CF_RENAME_PROP |
static java.lang.String |
FILTER_ARGS_CONF_KEY |
static java.lang.String |
FILTER_CLASS_CONF_KEY |
static java.lang.String |
HAS_LARGE_RESULT |
static java.lang.String |
TABLE_NAME |
static java.lang.String |
WAL_DURABILITY |
Constructor and Description |
---|
Import() |
Modifier and Type | Method and Description |
---|---|
static void |
addFilterAndArguments(Configuration conf,
java.lang.Class<? extends Filter> clazz,
java.util.List<java.lang.String> filterArgs)
Add a Filter to be instantiated on import
|
static void |
configureCfRenaming(Configuration conf,
java.util.Map<java.lang.String,java.lang.String> renameMap)
Sets a configuration property with key
CF_RENAME_PROP in conf that tells
the mapper how to rename column families. |
static Job |
createSubmittableJob(Configuration conf,
java.lang.String[] args)
Sets up the actual job.
|
static Cell |
filterKv(Filter filter,
Cell c)
Attempt to filter out the keyvalue
|
static void |
flushRegionsIfNecessary(Configuration conf)
If the durability is set to
Durability#SKIP_WAL and the data is imported to hbase, we
need to flush all the regions of the table as the data is held in memory and is also not
present in the Write Ahead Log to replay in scenarios of a crash. |
static Filter |
instantiateFilter(Configuration conf)
Create a
Filter to apply to all incoming keys (KeyValues ) to
optionally not include in the job output |
static void |
main(java.lang.String[] args)
Main entry point.
|
int |
run(java.lang.String[] args) |
public static final java.lang.String CF_RENAME_PROP
public static final java.lang.String BULK_OUTPUT_CONF_KEY
public static final java.lang.String FILTER_CLASS_CONF_KEY
public static final java.lang.String FILTER_ARGS_CONF_KEY
public static final java.lang.String TABLE_NAME
public static final java.lang.String WAL_DURABILITY
public static final java.lang.String HAS_LARGE_RESULT
public static Filter instantiateFilter(Configuration conf)
Filter
to apply to all incoming keys (KeyValues
) to
optionally not include in the job outputconf
- Configuration
from which to load the filterjava.lang.IllegalArgumentException
- if the filter is misconfiguredpublic static Cell filterKv(Filter filter, Cell c) throws java.io.IOException
c
- Cell
on which to apply the filterCell
java.io.IOException
public static void configureCfRenaming(Configuration conf, java.util.Map<java.lang.String,java.lang.String> renameMap)
Sets a configuration property with key CF_RENAME_PROP
in conf that tells
the mapper how to rename column families.
Alternately, instead of calling this function, you could set the configuration key
CF_RENAME_PROP
yourself. The value should look like
srcCf1:destCf1,srcCf2:destCf2,..... This would have the same effect on the mapper behavior.
conf
- the Configuration in which the CF_RENAME_PROP
key will be
setrenameMap
- a mapping from source CF names to destination CF namespublic static void addFilterAndArguments(Configuration conf, java.lang.Class<? extends Filter> clazz, java.util.List<java.lang.String> filterArgs) throws java.io.IOException
conf
- Configuration to update (will be passed to the job)clazz
- Filter
subclass to instantiate on the server.filterArgs
- List of arguments to pass to the filter on instantiationjava.io.IOException
public static Job createSubmittableJob(Configuration conf, java.lang.String[] args) throws java.io.IOException
conf
- The current configuration.args
- The command line parameters.java.io.IOException
- When setting up the job fails.public static void flushRegionsIfNecessary(Configuration conf) throws java.io.IOException, java.lang.InterruptedException
Durability#SKIP_WAL
and the data is imported to hbase, we
need to flush all the regions of the table as the data is held in memory and is also not
present in the Write Ahead Log to replay in scenarios of a crash. This method flushes all the
regions of the table in the scenarios of import data to hbase with Durability#SKIP_WAL
java.io.IOException
java.lang.InterruptedException
public int run(java.lang.String[] args) throws java.lang.Exception
java.lang.Exception
public static void main(java.lang.String[] args) throws java.lang.Exception
args
- The command line parameters.java.lang.Exception
- When running the job fails.