Package org.apache.hadoop.hive.ql.io.orc

The Optimized Row Columnar (ORC) File Format.

See:
Description

Interface Summary

BinaryColumnStatistics	Statistics for binary columns.
BooleanColumnStatistics	Statistics for boolean columns.
ColumnStatistics	Statistics that are available for all types of columns.
DateColumnStatistics	Statistics for DATE columns.
DecimalColumnStatistics	Statistics for decimal columns.
DirectDecompressionCodec
DoubleColumnStatistics	Statistics for float and double columns.
IntegerColumnStatistics	Statistics for all of the integer columns, such as byte, short, int, and long.
OrcFile.WriterCallback
OrcFile.WriterContext
Reader	The interface for reading ORC files.
RecordReader	A row-by-row iterator for ORC files.
StringColumnStatistics	Statistics for string columns.
StripeInformation	Information about the stripes in an ORC file that is provided by the Reader.
Writer	The interface for writing ORC files.

Class Summary

FileDump	A tool for printing out the file structure of ORC files.
Metadata
OrcFile	Contains factory methods to read or write ORC files.
OrcFile.ReaderOptions
OrcFile.WriterOptions	Options for creating ORC file writers.
OrcInputFormat	A MapReduce/Hive input format for ORC files.
OrcNewInputFormat	An InputFormat for ORC files.
OrcNewOutputFormat	An OutputFormat that writes ORC files.
OrcNewSplit	OrcFileSplit.
OrcOutputFormat	A Hive OutputFormat for ORC files.
OrcRawRecordMerger	Merges a base and a list of delta files together into a single stream of events.
OrcRecordUpdater	A RecordUpdater where the files are stored as ORC.
OrcRecordUpdater.OrcOptions	An extension to AcidOutputFormat that allows users to add additional options.
OrcSerde	A serde class for ORC.
OrcSplit	OrcFileSplit.
OrcStruct
Reader.Options	Options for creating a RecordReader.
StripeStatistics
VectorizedOrcInputFormat	A MapReduce/Hive input format for ORC files.
VectorizedOrcSerde	A serde class for ORC.

Enum Summary

CompressionKind	An enumeration that lists the generic compression algorithms that can be applied to ORC files.
OrcFile.OrcTableProperties	Enum container for all orc table properties.
OrcFile.Version	Create a version number for the ORC file format, so that we can add non-forward compatible changes in the future.

Package org.apache.hadoop.hive.ql.io.orc Description

The Optimized Row Columnar (ORC) File Format. This format:

• Decomposes complex column types into primitives
• Uses type-specific encoders for each column
  • Dictionary encodings for low cardinality columns
  • Run length encoding of data
  • variable length encoding of integers
• Divides file into large stripes
• Each stripe includes light-weight indexes that enable the reader to skip large sets of rows that don't satisfy the filter condition
• A file footer that contains meta-information about file
  • Precise byte range for each stripe
  • Type information for the file
  • Any user meta-information
• Seek to row number is implemented to support secondary indexes
• Support for additional generic compression: LZO, SNAPPY, ZLIB.

Format:

 HEADER (3 bytes) "ORC"
 STRIPE (0 or more stripes)
 FILE-FOOTER
 POST SCRIPT
 PS LENGTH (1 byte)
 
 

Stripe:

 INDEX-STREAM (0 or more)
 DATA-STREAM (0 or more)
 STRIPE-FOOTER