HWC and DataFrame API limitations

These limitations are in addition to Direct Reader mode, JDBC mode, and HWC and DataFrames API limitations.

  • Table stats (basic stats and column stats) are not generated when you write a DataFrame to Hive.
  • When the HWC API save mode is overwrite, writes are limited.

    You cannot read from and overwrite the same table. If your query accesses only one table and you try to overwrite that table using an HWC API write method, a deadlock state might occur. Do not attempt this operation.

    Example: Operation Not Supported

    scala> val df = hive.executeQuery("select * from t1")
    scala> df.write.format("com.hortonworks.spark.sql.hive.llap.HiveWarehouseConnector"). \
    mode("overwrite").option("table", "t1").save