Integrating Apache Hive with Spark and KafkaPDF version

Kerberos configurations for HWC

You learn how to set up HWC for Kerberos, or not. You set properties and values depending on the JDBC_CLUSTER or JDBC_CLIENT option you configure.

In CDP Private Cloud Base, you need to configure HWC options in configuration/spark-defaults.conf, depending on the read option you select. Alternatively, you can set the properties using the spark-submit/spark-shell --conf option.

For Spark applications on a kerberized Yarn cluster, set the following property: spark.sql.hive.hiveserver2.jdbc.url.principal. This property must be equal to hive.server2.authentication.kerberos.principal.

On a kerberized YARN cluster, set the following property:
  • JDBC_CLUSTER option in a secured cluster
    • Property: spark.security.credentials.hiveserver2.enabled
    • Value: true
    • Comment: true by default
  • JDBC_CLIENT option in a secured cluster
    • Property: spark.security.credentials.hiveserver2.enabled
    • Value: false

In an unsecured cluster, set the following property:

  • Property: spark.security.credentials.hiveserver2.enabled
  • Value: false

We want your opinion

How can we improve this page?

What kind of feedback do you have?