Apache Spark Component Guide
Also available as:
PDF
loading table of contents...

Forming JDBC Connection Strings for Spark SQL

A JDBC URL connection string supplies connection information to the JDBC data source. Connection strings for the Spark SQL JDBC driver have the following format:

jdbc:hive2://<host>:<port>/<dbName>;<sessionConfs>?<hiveConfs>#<hiveVars>

JDBC ParameterDescription
hostThe node hosting the Thrift Server
portThe port number on which the Thrift Server listens
dbNameThe name of the Hive database to run the query against
sessionConfsOptional configuration parameters for the JDBC or ODBC driver in the following format: <key1>=<value1>;<key2>=<key2>...;
hiveConfs

Optional configuration parameters for Hive on the server in the following format: <key1>=<value1>;<key2>=<key2>; ...

These settings last for the duration of the user session.

hiveVars

Optional configuration parameters for Hive variables in the following format: <key1>=<value1>;<key2>=<key2>; ...

These settings persist for the duration of the user session.

[Note]Note

The Spark Thrift server is a variant of HiveServer2, so you can use many of the same settings. For more information about JDBC connection strings, including transport and security settings, see Hive JDBC and ODBC Drivers in the HDP Data Access Guide.

The following connection string example accesses Spark SQL through JDBC on a Kerberos-enabled cluster:

beeline> !connect jdbc:hive2://localhost:10002/default;httpPath=/;principal=hive/hdp-team.example.com@EXAMPLE.COM

The following connection string example accesses Spark SQL through JDBC over HTTP transport on a Kerberos-enabled cluster:

beeline> !connect jdbc:hive2://localhost:10002/default;transportMode=http;httpPath=/;principal=hive/hdp-team.example.com@EXAMPLE.COM