Integrating Apache Hive with Apache Spark and BIPDF version

Commit transaction in Spark Direct Reader mode

In Spark Direct Reader mode, you need to know how to commit or abort transactions.

A sql listener normally handles this task automatically when a dataframe operation or spark sql query finishes. In some cases when .explain() , .rdd() , or .cache() are invoked on a dataframe, the transaction is not automatically closed. In Spark Direct Reader mode, commit or abort a transaction as follows:
scala> com.qubole.spark.hiveacid.transaction.HiveAcidTxnManagerObject.commitTxn(spark)     
scala> hive.commitTxn
Or, if you are using Hive Warehouse Connector with Direct Reader Mode enabled, you can invoke following API to commit transaction:
 scala> hive.commitTxn