Apache Spark Component Guide
Also available as:
PDF
loading table of contents...

​Configuring the Spark SQL Hive Warehouse Directory

Since Spark 2.0.0, Spark references spark.sql.warehouse.dir as the default Spark SQL Hive Warehouse location. To create databases, both the spark-defaults.conf and spark-thrift-sparkconf.conf files should have a spark.sql.warehouse.dir property that has the same value as the hive.metastore.warehouse.dir property. The value of this property is usually /apps/hive/warehouse:

spark.sql.warehouse.dir=/apps/hive/warehouse

On an Ambari cluster, select Spark2 > Configs, then use Add Property in "Custom spark2-defaults" and "Custom spark2-thrift-sparkconf" to add a spark.sql.warehouse.dir property with the value /apps/hive/warehouse.