org.apache.hadoop.hive.ql.exec.mr
Class MapRedTask
java.lang.Object
org.apache.hadoop.hive.ql.exec.Task<MapredWork>
org.apache.hadoop.hive.ql.exec.mr.ExecDriver
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
- All Implemented Interfaces:
- Serializable, HadoopJobExecHook, Node
public class MapRedTask
- extends ExecDriver
- implements Serializable
Extension of ExecDriver:
- can optionally spawn a map-reduce task from a separate jvm
- will make last minute adjustments to map-reduce job parameters, viz:
* estimating number of reducers
* estimating whether job should run locally
- See Also:
- Serialized Form
Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.Task |
Task.FeedType |
Fields inherited from class org.apache.hadoop.hive.ql.exec.Task |
BACKUP_COMMON_JOIN, COMMON_JOIN, CONVERTED_MAPJOIN, CONVERTED_MAPJOIN_LOCAL, CONVERTED_SORTMERGEJOIN, HINTED_MAPJOIN, HINTED_MAPJOIN_LOCAL, MAPJOIN_ONLY_NOBACKUP, NO_TAG, taskCounters, taskHandle |
Methods inherited from class org.apache.hadoop.hive.ql.exec.mr.ExecDriver |
checkFatalErrors, generateCmdLine, getCurrentDB, getName, getTopOperators, getType, hasReduce, initialize, isMapRedTask, logPlanProgress, main, requireLock |
Methods inherited from class org.apache.hadoop.hive.ql.exec.Task |
addDependentTask, done, equals, executeTask, getAndInitBackupTask, getBackupChildrenTasks, getBackupTask, getChildren, getChildTasks, getCounters, getDependentTasks, getDriverContext, getFeedSubscribers, getId, getInitialized, getJobID, getParentTasks, getQueryPlan, getQueued, getResultSchema, getTaskHandle, getTaskTag, getWork, hashCode, ifRetryCmdWhenFail, isLocalMode, isMapRedLocalTask, isRootTask, isRunnable, removeDependentTask, removeFromChildrenTasks, setBackupChildrenTasks, setBackupTask, setChildTasks, setConsole, setDone, setFeedSubscribers, setId, setInitialized, setLocalMode, setParentTasks, setQueryPlan, setQueued, setRetryCmdWhenFail, setRootTask, setStarted, setTaskTag, setWork, started, subscribeFeed, toString |
MapRedTask
public MapRedTask()
execute
public int execute(DriverContext driverContext)
- Description copied from class:
ExecDriver
- Execute a query plan using Hadoop.
- Overrides:
execute
in class ExecDriver
- Returns:
- status of executing the task
mapStarted
public boolean mapStarted()
- Overrides:
mapStarted
in class ExecDriver
reduceStarted
public boolean reduceStarted()
- Overrides:
reduceStarted
in class ExecDriver
mapDone
public boolean mapDone()
- Overrides:
mapDone
in class ExecDriver
reduceDone
public boolean reduceDone()
- Overrides:
reduceDone
in class ExecDriver
isEligibleForLocalMode
public static String isEligibleForLocalMode(HiveConf conf,
int numReducers,
long inputLength,
long inputFileCount)
- Find out if a job can be run in local mode based on it's characteristics
- Parameters:
conf
- Hive ConfigurationnumReducers
- total number of reducers for this jobinputLength
- the size of the inputinputFileCount
- the number of files of input
- Returns:
- String null if job is eligible for local mode, reason otherwise
getReducer
public Operator<? extends OperatorDesc> getReducer()
- Overrides:
getReducer
in class Task<MapredWork>
shutdown
public void shutdown()
- Overrides:
shutdown
in class ExecDriver
Copyright © 2014 The Apache Software Foundation. All rights reserved.