Use HWC/Spark Direct Reader for Spark Apps/ETL
Use the Hive Warehouse Connector (HWC) to access Spark data after upgrading. HWC is a Spark library/plugin that is launched with the Spark app. Use the Spark Direct Reader and HWC for ETL.
The Hive Warehouse Connector is designed to access managed ACID v2 Hive tables from Spark. Apache Ranger and the HiveWarehouseConnector library provide row and column, fine-grained access to the data. HWC supports spark-submit and pyspark. The spark thrift server is not supported.