org.apache.hadoop.hive.ql.optimizer
Class SortedDynPartitionOptimizer

java.lang.Object
  org.apache.hadoop.hive.ql.optimizer.SortedDynPartitionOptimizer

All Implemented Interfaces:

Transform

public class SortedDynPartitionOptimizer
extends Object
implements Transform

When dynamic partitioning (with or without bucketing and sorting) is enabled, this optimization sorts the records on partition, bucket and sort columns respectively before inserting records into the destination table. This enables reducers to keep only one record writer all the time thereby reducing the the memory pressure on the reducers. This optimization will force a reducer even when hive.enforce.bucketing and hive.enforce.sorting is set to false.

Constructor Summary

SortedDynPartitionOptimizer()

Method Summary

ParseContext

transform(ParseContext pCtx) All transformation steps implement this interface.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SortedDynPartitionOptimizer

public SortedDynPartitionOptimizer()

Method Detail

transform

public ParseContext transform(ParseContext pCtx)
                       throws SemanticException

Description copied from interface: Transform

All transformation steps implement this interface.

Specified by:

transform in interface Transform

Parameters:

pCtx - input parse context

Returns:

ParseContext

Throws:

SemanticException