Developing Apache Spark ApplicationsPDF version

Reading Hive ORC tables

For existing Hive tables, Spark can read them without createOrReplaceTempView. If the table is stored as ORC format (the default), predicate push-down, partition pruning, and vectorized query execution are also applied according to the configuration.

spark.sql("SELECT * FROM hiveTable WHERE age = 20")