org.apache.hadoop.hive.ql.io
Class RCFileOutputFormat
java.lang.Object
org.apache.hadoop.mapred.FileOutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>
org.apache.hadoop.hive.ql.io.RCFileOutputFormat
- All Implemented Interfaces:
- HiveOutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>, org.apache.hadoop.mapred.OutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>
public class RCFileOutputFormat
- extends org.apache.hadoop.mapred.FileOutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>
- implements HiveOutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>
RCFileOutputFormat.
Nested classes/interfaces inherited from class org.apache.hadoop.mapred.FileOutputFormat |
org.apache.hadoop.mapred.FileOutputFormat.Counter |
Method Summary |
static int |
getColumnNumber(org.apache.hadoop.conf.Configuration conf)
Returns the number of columns set in the conf for writers. |
FileSinkOperator.RecordWriter |
getHiveRecordWriter(org.apache.hadoop.mapred.JobConf jc,
org.apache.hadoop.fs.Path finalOutPath,
Class<? extends org.apache.hadoop.io.Writable> valueClass,
boolean isCompressed,
Properties tableProperties,
org.apache.hadoop.util.Progressable progress)
create the final out file. |
org.apache.hadoop.mapred.RecordWriter<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable> |
getRecordWriter(org.apache.hadoop.fs.FileSystem ignored,
org.apache.hadoop.mapred.JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
|
static void |
setColumnNumber(org.apache.hadoop.conf.Configuration conf,
int columnNum)
set number of columns into the given configuration. |
Methods inherited from class org.apache.hadoop.mapred.FileOutputFormat |
checkOutputSpecs, getCompressOutput, getOutputCompressorClass, getOutputPath, getPathForCustomFile, getTaskOutputPath, getUniqueName, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputPath, setWorkOutputPath |
Methods inherited from interface org.apache.hadoop.mapred.OutputFormat |
checkOutputSpecs |
RCFileOutputFormat
public RCFileOutputFormat()
setColumnNumber
public static void setColumnNumber(org.apache.hadoop.conf.Configuration conf,
int columnNum)
- set number of columns into the given configuration.
- Parameters:
conf
- configuration instance which need to set the column numbercolumnNum
- column number for RCFile's Writer
getColumnNumber
public static int getColumnNumber(org.apache.hadoop.conf.Configuration conf)
- Returns the number of columns set in the conf for writers.
- Parameters:
conf
-
- Returns:
- number of columns for RCFile's writer
getRecordWriter
public org.apache.hadoop.mapred.RecordWriter<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable> getRecordWriter(org.apache.hadoop.fs.FileSystem ignored,
org.apache.hadoop.mapred.JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
throws IOException
-
- Specified by:
getRecordWriter
in interface org.apache.hadoop.mapred.OutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>
- Specified by:
getRecordWriter
in class org.apache.hadoop.mapred.FileOutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>
- Throws:
IOException
getHiveRecordWriter
public FileSinkOperator.RecordWriter getHiveRecordWriter(org.apache.hadoop.mapred.JobConf jc,
org.apache.hadoop.fs.Path finalOutPath,
Class<? extends org.apache.hadoop.io.Writable> valueClass,
boolean isCompressed,
Properties tableProperties,
org.apache.hadoop.util.Progressable progress)
throws IOException
- create the final out file.
- Specified by:
getHiveRecordWriter
in interface HiveOutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>
- Parameters:
jc
- the job configuration filefinalOutPath
- the final output file to be createdvalueClass
- the value class used for createisCompressed
- whether the content is compressed or nottableProperties
- the tableInfo of this file's corresponding tableprogress
- progress used for status report
- Returns:
- the RecordWriter for the output file
- Throws:
IOException
Copyright © 2014 The Apache Software Foundation. All rights reserved.