|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object org.apache.hadoop.hive.ql.stats.CounterStatsAggregatorTez
public class CounterStatsAggregatorTez
This class aggregates stats via counters and does so for Tez Tasks. With dbclass=counters this class will compute table/partition statistics using hadoop counters. They will be published using special keys and then retrieved on the client after the insert/ctas statement ran.
Constructor Summary | |
---|---|
CounterStatsAggregatorTez()
|
Method Summary | |
---|---|
String |
aggregateStats(String keyPrefix,
String statType)
This method aggregates a given statistic from all tasks (partial stats). |
boolean |
cleanUp(String keyPrefix)
This method is called after all statistics have been aggregated. |
boolean |
closeConnection()
This method closes the connection to the temporary storage. |
boolean |
connect(org.apache.hadoop.conf.Configuration hconf,
Task sourceTask)
This method connects to the temporary storage. |
Methods inherited from class java.lang.Object |
---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public CounterStatsAggregatorTez()
Method Detail |
---|
public boolean connect(org.apache.hadoop.conf.Configuration hconf, Task sourceTask)
StatsAggregator
connect
in interface StatsAggregator
hconf
- HiveConf that contains the connection parameters.
public String aggregateStats(String keyPrefix, String statType)
StatsAggregator
aggregateStats
in interface StatsAggregator
keyPrefix
- a prefix of the keys used in StatsPublisher to publish stats.
Any rows that starts with the same prefix will be aggregated. For example, if
the StatsPublisher uses the following compound key to publish stats:
the output directory name (unique per FileSinkOperator) +
the partition specs (only for dynamic partitions) +
taskID (last component of task file)
The keyPrefix for aggregation could be first 2 components. This will aggregates stats
across all tasks for each partition.statType
- a string noting the key to be published. Ex: "numRows".
public boolean closeConnection()
StatsAggregator
closeConnection
in interface StatsAggregator
public boolean cleanUp(String keyPrefix)
StatsAggregator
cleanUp
in interface StatsAggregator
keyPrefix
- a prefix of the keys used in StatsPublisher to publish stats. It is the same
as the first parameter in aggregateStats().
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |