|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object org.apache.hadoop.hive.ql.stats.CounterStatsAggregator
public class CounterStatsAggregator
Constructor Summary | |
---|---|
CounterStatsAggregator()
|
Method Summary | |
---|---|
String |
aggregateStats(String counterGrpName,
String statType)
This method aggregates a given statistic from all tasks (partial stats). |
boolean |
cleanUp(String keyPrefix)
This method is called after all statistics have been aggregated. |
boolean |
closeConnection()
This method closes the connection to the temporary storage. |
boolean |
connect(org.apache.hadoop.conf.Configuration hconf,
Task sourceTask)
This method connects to the temporary storage. |
Methods inherited from class java.lang.Object |
---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public CounterStatsAggregator()
Method Detail |
---|
public boolean connect(org.apache.hadoop.conf.Configuration hconf, Task sourceTask)
StatsAggregator
connect
in interface StatsAggregator
hconf
- HiveConf that contains the connection parameters.
public String aggregateStats(String counterGrpName, String statType)
StatsAggregator
aggregateStats
in interface StatsAggregator
counterGrpName
- a prefix of the keys used in StatsPublisher to publish stats.
Any rows that starts with the same prefix will be aggregated. For example, if
the StatsPublisher uses the following compound key to publish stats:
the output directory name (unique per FileSinkOperator) +
the partition specs (only for dynamic partitions) +
taskID (last component of task file)
The keyPrefix for aggregation could be first 2 components. This will aggregates stats
across all tasks for each partition.statType
- a string noting the key to be published. Ex: "numRows".
public boolean closeConnection()
StatsAggregator
closeConnection
in interface StatsAggregator
public boolean cleanUp(String keyPrefix)
StatsAggregator
cleanUp
in interface StatsAggregator
keyPrefix
- a prefix of the keys used in StatsPublisher to publish stats. It is the same
as the first parameter in aggregateStats().
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |