Direct Reader mode introduction
Direct Reader mode is a transparent connection that Hive Warehouse Connector (HWC) makes to Apache Hive metastore (HMS) to get transaction information. In Direct Reader mode, Spark reads the data directly from the managed table location using the transaction snapshot. You use this mode if you do not need production-level Ranger authorization. Direct Reader mode does not support Ranger authorization.
Requirements and recommendations
Spark Direct Reader mode requires a connection to Hive metastore. A HiveServer (HS2) connection is not needed.
Spark Direct Reader for reading Hive ACID, transactional tables from Spark is supported for production use. Use Spark Direct Reader mode if your ETL jobs do not require authorization and run as super user.
Component interaction
The following diagram shows component interaction ifor Direct Reader reads.
