Configuring the Spark SQL Hive Warehouse Directory
Since Spark 2.0.0, Spark references spark.sql.warehouse.dir
as the default
Spark SQL Hive Warehouse location. To create databases, both the
spark-defaults.conf
and spark-thrift-sparkconf.conf
files should
have a spark.sql.warehouse.dir
property that has the same value as the
hive.metastore.warehouse.dir
property. The value of this property is usually
/apps/hive/warehouse
:
spark.sql.warehouse.dir=/apps/hive/warehouse
On an Ambari cluster, select Spark2 > Configs, then use
Add Property in "Custom spark2-defaults" and "Custom
spark2-thrift-sparkconf" to add a spark.sql.warehouse.dir
property with the value
/apps/hive/warehouse
.