ORC vs Parquet in CDP

The differences between Optimized Row Columnar (ORC) file format for storing Hive data and Parquet for storing Impala data are important to understand. Query performance improves when you use the appropriate format for your application.

ORC and Parquet capabilities comparison

The following table compares Hive and Impala support for ORC and Parquet. The CDP Services column shows the supported services:
  • Hive
  • Hive metastore (HMS)
  • Impala
  • Spark
Table 1.
Capability Data Warehouse ORC Parquet CDP Services
Read non-transactional data Apache Hive Hive & HMS
Read non-transactional data Apache Impala Impala & HMS
Read/Write Full ACID tables Apache Hive Hive & HMS
Read Full ACID tables Apache Impala Impala & HMS
Read Insert-only managed tables Apache Impala Impala & HMS
Hive Warehouse Connector reads Apache Hive Hive & Spark & HMS
Hive Warehouse Connector writes Apache Hive Hive & Spark & HMS
Column index Apache Hive Hive & HMS
Column index Apache Impala Impala & HMS
CBO uses column metadata Apache Hive Hive & HMS
Recommended format Apache Hive Hive & HMS
Recommended format Apache Impala Impala & HMS
Vectorized reader Apache Hive Hive & HMS
Read complex types Apache Impala Impala & HMS
Read/write complex types Apache Hive Hive & HMS