org.apache.hadoop.hive.ql.io.parquet
Class MapredParquetOutputFormat
java.lang.Object
org.apache.hadoop.mapred.FileOutputFormat<Void,org.apache.hadoop.io.ArrayWritable>
org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat
- All Implemented Interfaces:
- HiveOutputFormat<Void,org.apache.hadoop.io.ArrayWritable>, org.apache.hadoop.mapred.OutputFormat<Void,org.apache.hadoop.io.ArrayWritable>
public class MapredParquetOutputFormat
- extends org.apache.hadoop.mapred.FileOutputFormat<Void,org.apache.hadoop.io.ArrayWritable>
- implements HiveOutputFormat<Void,org.apache.hadoop.io.ArrayWritable>
A Parquet OutputFormat for Hive (with the deprecated package mapred)
Nested classes/interfaces inherited from class org.apache.hadoop.mapred.FileOutputFormat |
org.apache.hadoop.mapred.FileOutputFormat.Counter |
Method Summary |
void |
checkOutputSpecs(org.apache.hadoop.fs.FileSystem ignored,
org.apache.hadoop.mapred.JobConf job)
|
FileSinkOperator.RecordWriter |
getHiveRecordWriter(org.apache.hadoop.mapred.JobConf jobConf,
org.apache.hadoop.fs.Path finalOutPath,
Class<? extends org.apache.hadoop.io.Writable> valueClass,
boolean isCompressed,
Properties tableProperties,
org.apache.hadoop.util.Progressable progress)
Create the parquet schema from the hive schema, and return the RecordWriterWrapper which
contains the real output format |
org.apache.hadoop.mapred.RecordWriter<Void,org.apache.hadoop.io.ArrayWritable> |
getRecordWriter(org.apache.hadoop.fs.FileSystem ignored,
org.apache.hadoop.mapred.JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
|
Methods inherited from class org.apache.hadoop.mapred.FileOutputFormat |
getCompressOutput, getOutputCompressorClass, getOutputPath, getPathForCustomFile, getTaskOutputPath, getUniqueName, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputPath, setWorkOutputPath |
MapredParquetOutputFormat
public MapredParquetOutputFormat()
MapredParquetOutputFormat
public MapredParquetOutputFormat(org.apache.hadoop.mapreduce.OutputFormat<Void,org.apache.hadoop.io.ArrayWritable> mapreduceOutputFormat)
checkOutputSpecs
public void checkOutputSpecs(org.apache.hadoop.fs.FileSystem ignored,
org.apache.hadoop.mapred.JobConf job)
throws IOException
- Specified by:
checkOutputSpecs
in interface org.apache.hadoop.mapred.OutputFormat<Void,org.apache.hadoop.io.ArrayWritable>
- Overrides:
checkOutputSpecs
in class org.apache.hadoop.mapred.FileOutputFormat<Void,org.apache.hadoop.io.ArrayWritable>
- Throws:
IOException
getRecordWriter
public org.apache.hadoop.mapred.RecordWriter<Void,org.apache.hadoop.io.ArrayWritable> getRecordWriter(org.apache.hadoop.fs.FileSystem ignored,
org.apache.hadoop.mapred.JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
throws IOException
- Specified by:
getRecordWriter
in interface org.apache.hadoop.mapred.OutputFormat<Void,org.apache.hadoop.io.ArrayWritable>
- Specified by:
getRecordWriter
in class org.apache.hadoop.mapred.FileOutputFormat<Void,org.apache.hadoop.io.ArrayWritable>
- Throws:
IOException
getHiveRecordWriter
public FileSinkOperator.RecordWriter getHiveRecordWriter(org.apache.hadoop.mapred.JobConf jobConf,
org.apache.hadoop.fs.Path finalOutPath,
Class<? extends org.apache.hadoop.io.Writable> valueClass,
boolean isCompressed,
Properties tableProperties,
org.apache.hadoop.util.Progressable progress)
throws IOException
- Create the parquet schema from the hive schema, and return the RecordWriterWrapper which
contains the real output format
- Specified by:
getHiveRecordWriter
in interface HiveOutputFormat<Void,org.apache.hadoop.io.ArrayWritable>
- Parameters:
jobConf
- the job configuration filefinalOutPath
- the final output file to be createdvalueClass
- the value class used for createisCompressed
- whether the content is compressed or nottableProperties
- the table properties of this file's corresponding tableprogress
- progress used for status report
- Returns:
- the RecordWriter for the output file
- Throws:
IOException
Copyright © 2014 The Apache Software Foundation. All rights reserved.