(Optional) Starting the Spark Thrift Server
To enable and start the Spark Thrift Server:
From SPARK_HOME
, start the Spark SQL Thrift Server. Specify the port
value of the Thrift Server (the default is 10015).
For example:
su spark
./sbin/start-thriftserver.sh --master yarn-client --executor-memory
512m --hiveconf hive.server2.thrift.port=100015
Use this port when you connect via Beeline.
Kerberos Considerations
If you are installing the Spark Thrift Server on a Kerberos-secured cluster, the following instructions apply:
The Spark Thrift Server must run in the same host as
HiveServer2
, so that it can access thehiveserver2
keytab.Edit permissions in
/var/run/spark
and/var/log/spark
to specify read/write permissions to the Hive service account.Use the Hive service account to start the
thriftserver
process.
Note | |
---|---|
We recommend that you run the Spark Thrift Server as user |
Important | |
---|---|
When the Spark Thrift Server runs queries as user |
For Spark jobs that are not submitted through the Thrift Server, the user submitting the job must have access to the Hive metastore in secure mode (via kinit).