Apache Spark Component Guide
Also available as:
PDF
loading table of contents...

Configuring the Spark SQL Hive Warehouse Directory

Since Spark 2.0.0, Spark references spark.sql.warehouse.dir as the default Spark SQL Hive Warehouse location. To create databases, both the spark-defaults.conf and spark-thrift-sparkconf.conf files should have a spark.sql.warehouse.dir property that has the same value as the hive.metastore.warehouse.dir property. The value of this property is usually /apps/hive/warehouse:

spark.sql.warehouse.dir=/apps/hive/warehouse

On an Ambari cluster, select Spark2 > Configs, then use Add Property in "Custom spark2-defaults" and "Custom spark2-thrift-sparkconf" to add a spark.sql.warehouse.dir property with the value /apps/hive/warehouse.