Managed table location
Creating a managed table with nonempty location is not allowed.
Type of change: Property/Spark core changes
Spark 1.6 - 2.3
You can create a managed table having a nonempty location.
Spark 2.4
Creating a managed table with nonempty location is not allowed. In Spark 2.4,
an error occurs when there is a write operation, such as
df.write.mode(SaveMode.Overwrite).saveAsTable("testdb.testtable")
.
The error side-effects are the cluster is terminated while the
write is in progress, a temporary network issue occurs, or the job is interrupted.
Action Required
Set
spark.sql.legacy.allowCreatingManagedTableUsingNonemptyLocation
to true at
runtime as
follows:spark.conf.set("spark.sql.legacy.allowCreatingManagedTableUsingNonemptyLocation","true")