|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor
public class MapJoinProcessor
Implementation of one of the rule-based map join optimization. User passes hints to specify map-joins and during this optimization, all user specified map joins are converted to MapJoins - the reduce sink operator above the join are converted to map sink operators. In future, once statistics are implemented, this transformation can also be done based on costs.
Nested Class Summary | |
---|---|
static class |
MapJoinProcessor.CurrentMapJoin
CurrentMapJoin. |
static class |
MapJoinProcessor.Default
Default. |
static class |
MapJoinProcessor.MapJoinDefault
MapJoinDefault. |
static class |
MapJoinProcessor.MapJoinFS
MapJoinFS. |
static class |
MapJoinProcessor.MapJoinWalkerCtx
MapJoinWalkerCtx. |
Constructor Summary | |
---|---|
MapJoinProcessor()
empty constructor. |
Method Summary | |
---|---|
static int |
checkMapJoin(int mapJoinPos,
JoinCondDesc[] condns)
|
static MapJoinOperator |
convertJoinOpMapJoinOp(HiveConf hconf,
LinkedHashMap<Operator<? extends OperatorDesc>,OpParseContext> opParseCtxMap,
JoinOperator op,
QBJoinTree joinTree,
int mapJoinPos,
boolean noCheckOuterJoin)
|
static MapJoinOperator |
convertMapJoin(HiveConf conf,
LinkedHashMap<Operator<? extends OperatorDesc>,OpParseContext> opParseCtxMap,
JoinOperator op,
QBJoinTree joinTree,
int mapJoinPos,
boolean noCheckOuterJoin,
boolean validateMapJoinTree)
convert a regular join to a a map-side join. |
static MapJoinOperator |
convertSMBJoinToMapJoin(HiveConf hconf,
Map<Operator<? extends OperatorDesc>,OpParseContext> opParseCtxMap,
SMBMapJoinOperator smbJoinOp,
QBJoinTree joinTree,
int bigTablePos,
boolean noCheckOuterJoin)
convert a sortmerge join to a a map-side join. |
MapJoinOperator |
generateMapJoinOperator(ParseContext pctx,
JoinOperator op,
QBJoinTree joinTree,
int mapJoinPos)
|
static void |
genLocalWorkForMapJoin(MapredWork newWork,
MapJoinOperator newMapJoinOp,
int mapJoinPos)
|
static void |
genMapJoinOpAndLocalWork(HiveConf conf,
MapredWork newWork,
JoinOperator op,
int mapJoinPos)
Convert the join to a map-join and also generate any local work needed. |
static Set<Integer> |
getBigTableCandidates(JoinCondDesc[] condns)
Get a list of big table candidates. |
static NodeProcessor |
getCurrentMapJoin()
|
static NodeProcessor |
getDefault()
|
static NodeProcessor |
getMapJoinDefault()
|
static NodeProcessor |
getMapJoinFS()
|
ParseContext |
transform(ParseContext pactx)
Transform the query tree. |
Methods inherited from class java.lang.Object |
---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public MapJoinProcessor()
Method Detail |
---|
public static void genMapJoinOpAndLocalWork(HiveConf conf, MapredWork newWork, JoinOperator op, int mapJoinPos) throws SemanticException
newWork
- MapredWork in which the conversion is to happenop
- The join operator that needs to be converted to map-joinbigTablePos
-
SemanticException
public static void genLocalWorkForMapJoin(MapredWork newWork, MapJoinOperator newMapJoinOp, int mapJoinPos) throws SemanticException
SemanticException
public static MapJoinOperator convertMapJoin(HiveConf conf, LinkedHashMap<Operator<? extends OperatorDesc>,OpParseContext> opParseCtxMap, JoinOperator op, QBJoinTree joinTree, int mapJoinPos, boolean noCheckOuterJoin, boolean validateMapJoinTree) throws SemanticException
opParseCtxMap
- op
- join operatorjoinTree
- qb join treemapJoinPos
- position of the source to be read as part of map-reduce framework. All other sources
are cached in memorynoCheckOuterJoin
-
SemanticException
public static MapJoinOperator convertJoinOpMapJoinOp(HiveConf hconf, LinkedHashMap<Operator<? extends OperatorDesc>,OpParseContext> opParseCtxMap, JoinOperator op, QBJoinTree joinTree, int mapJoinPos, boolean noCheckOuterJoin) throws SemanticException
SemanticException
public static MapJoinOperator convertSMBJoinToMapJoin(HiveConf hconf, Map<Operator<? extends OperatorDesc>,OpParseContext> opParseCtxMap, SMBMapJoinOperator smbJoinOp, QBJoinTree joinTree, int bigTablePos, boolean noCheckOuterJoin) throws SemanticException
opParseCtxMap
- smbJoinOp
- join operatorjoinTree
- qb join treebigTablePos
- position of the source to be read as part of map-reduce framework. All other sources
are cached in memorynoCheckOuterJoin
-
SemanticException
public MapJoinOperator generateMapJoinOperator(ParseContext pctx, JoinOperator op, QBJoinTree joinTree, int mapJoinPos) throws SemanticException
SemanticException
public static Set<Integer> getBigTableCandidates(JoinCondDesc[] condns)
condns
-
public static int checkMapJoin(int mapJoinPos, JoinCondDesc[] condns)
mapJoinPos
- the position of big table as determined by either hints or auto conversion.condns
- the join conditions
SemanticException
- if given position is not in the big table candidates.public ParseContext transform(ParseContext pactx) throws SemanticException
transform
in interface Transform
pactx
- current parse context
SemanticException
public static NodeProcessor getMapJoinFS()
public static NodeProcessor getMapJoinDefault()
public static NodeProcessor getDefault()
public static NodeProcessor getCurrentMapJoin()
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |