You can submit a Python app based on the HiveWarehouseConnector library by
submitting a Scala or Java application, and then adding a Python package.
-
Locate the hive-warehouse-connector-assembly jar in the
/hive_warehouse_connector/ directory.
-
Add the connector jar to the app submission using the
--jars
option.
pyspark --jars <path to jars>/hive_warehouse_connector/hive-warehouse-connector-assembly-<version>.jar
-
Locate the pyspark_hwc zip package in the
/hive_warehouse_connector/ directory.
-
Add the Python package for the connector to the app
submission.
pyspark --jars <path to jars>/hive_warehouse_connector/hive-warehouse-connector-assembly-<version>.jar --py-files <path>/hive_warehouse_connector/pyspark_hwc-<version>.zip
