org.apache.hadoop.hive.ql.io
Class HiveContextAwareRecordReader<K,V>

java.lang.Object
  org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader<K,V>

All Implemented Interfaces:

org.apache.hadoop.mapred.RecordReader<K,V>

Direct Known Subclasses:

BucketizedHiveRecordReader, CombineHiveRecordReader, HiveRecordReader

public abstract class HiveContextAwareRecordReader<K,V>
extends Object
implements org.apache.hadoop.mapred.RecordReader<K,V>

This class prepares an IOContext, and provides the ability to perform a binary search on the data. The binary search can be used by setting the value of inputFormatSorted in the MapreduceWork to true, but it should only be used if the data is going to a FilterOperator, which filters by comparing a value in the data with a constant, using one of the comparisons =, <, >, <=, >=. If the RecordReader's underlying format is an RCFile, this object can perform a binary search to find the block to begin reading from, and stop reading once it can be determined no other entries will match the filter.

Constructor Summary

HiveContextAwareRecordReader(org.apache.hadoop.mapred.JobConf conf)

HiveContextAwareRecordReader(org.apache.hadoop.mapred.RecordReader recordReader)

HiveContextAwareRecordReader(org.apache.hadoop.mapred.RecordReader recordReader, org.apache.hadoop.mapred.JobConf conf)

Method Summary

void	close()
abstract void	doClose() Close this InputSplit to future operations.
boolean	doNext(K key, V value)
boolean	foundAllTargets() Returns true if the current comparison is in the list of stop comparisons, i.e.
IOContext	getIOContext()
float	getProgress()
void	initIOContext(org.apache.hadoop.mapred.FileSplit split, org.apache.hadoop.mapred.JobConf job, Class inputFormatClass)
void	initIOContext(org.apache.hadoop.mapred.FileSplit split, org.apache.hadoop.mapred.JobConf job, Class inputFormatClass, org.apache.hadoop.mapred.RecordReader recordReader)
void	initIOContext(long startPos, boolean isBlockPointer, org.apache.hadoop.fs.Path inputPath)
void	initIOContextSortedProps(org.apache.hadoop.mapred.FileSplit split, org.apache.hadoop.mapred.RecordReader recordReader, org.apache.hadoop.mapred.JobConf job)
boolean	next(K key, V value)
void	setRecordReader(org.apache.hadoop.mapred.RecordReader recordReader)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.mapred.RecordReader

createKey, createValue, getPos

Constructor Detail

HiveContextAwareRecordReader

public HiveContextAwareRecordReader(org.apache.hadoop.mapred.JobConf conf)
                             throws IOException

Throws:

IOException

HiveContextAwareRecordReader

public HiveContextAwareRecordReader(org.apache.hadoop.mapred.RecordReader recordReader)

HiveContextAwareRecordReader

public HiveContextAwareRecordReader(org.apache.hadoop.mapred.RecordReader recordReader,
                                    org.apache.hadoop.mapred.JobConf conf)
                             throws IOException

Throws:

IOException

Method Detail

setRecordReader

public void setRecordReader(org.apache.hadoop.mapred.RecordReader recordReader)

doClose

public abstract void doClose()
                      throws IOException

Close this InputSplit to future operations.

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface org.apache.hadoop.mapred.RecordReader<K,V>

Throws:

IOException

next

public boolean next(K key,
                    V value)
             throws IOException

Specified by:

next in interface org.apache.hadoop.mapred.RecordReader<K,V>

Throws:

IOException

getIOContext

public IOContext getIOContext()

initIOContext

public void initIOContext(long startPos,
                          boolean isBlockPointer,
                          org.apache.hadoop.fs.Path inputPath)

initIOContext

public void initIOContext(org.apache.hadoop.mapred.FileSplit split,
                          org.apache.hadoop.mapred.JobConf job,
                          Class inputFormatClass)
                   throws IOException

Throws:

IOException

initIOContext

public void initIOContext(org.apache.hadoop.mapred.FileSplit split,
                          org.apache.hadoop.mapred.JobConf job,
                          Class inputFormatClass,
                          org.apache.hadoop.mapred.RecordReader recordReader)
                   throws IOException

Throws:

IOException

initIOContextSortedProps

public void initIOContextSortedProps(org.apache.hadoop.mapred.FileSplit split,
                                     org.apache.hadoop.mapred.RecordReader recordReader,
                                     org.apache.hadoop.mapred.JobConf job)

getProgress

public float getProgress()
                  throws IOException

Specified by:

getProgress in interface org.apache.hadoop.mapred.RecordReader<K,V>

Throws:

IOException

doNext

public boolean doNext(K key,
                      V value)
               throws IOException

Throws:

IOException

foundAllTargets

public boolean foundAllTargets()

Returns true if the current comparison is in the list of stop comparisons, i.e. we've found all records which won't be filtered

Returns:

true if the current comparison is found