Enabling Spark 3 engine in Hue

Go to Clusters > HDFS > Configuration and add the following lines in the Cluster-wide Advanced Configuration Snippet (Safety Valve) for core-site.xml field:

<property>
	<name>hadoop.proxyuser.hue.groups</name>
	<value>*</value>
<name>hadoop.proxyuser.hue.hosts</name>
	<value>*</value>
	<name>hadoop.proxyuser.spark.groups</name>
	<value>*</value>
	<name>hadoop.proxyuser.spark.hosts</name>
	<value>*</value>
	<name>hadoop.proxyuser.livy.groups</name>
	<value>*</value>
	<name>hadoop.proxyuser.livy.hosts</name>
	<value>*</value>
</property>

Click Save Changes.
Go to Clusters > Livy for Spark 3 service > Configuration and add the following configurations:
1. Add the hue user in the Admin Users (livy.superusers) field.
2. Go to the HMS Service field and select Hive.
3. Click Save Changes.
Go to Clusters > SPARK_ON_YARN > Configuration > Admin Users, add hue to the list of admin users (spark.history.ui.admin.acls) and click Save Changes.
Go to Clusters > SPARK > Configuration > Admin Users, add hue to the list of admin users (spark.history.ui.admin.acls) and click Save Changes.
Go to Clusters > SPARK 3 > Configuration > Admin Users, add hue to the list of admin users (spark.history.ui.admin.acls) and click Save Changes.
Go to Clusters > Hue > Configuration and enter the following lines in the Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini field and click Save Changes:
```
[desktop]
app_blacklist=zookeeper, pig #custom list of blocked apps
[spark]
#This is not a thrift server port
#If this TLS/SSL is enabled then check to see whether the livy url is on https or http and modify the url accordingly.
livy_server_url=http(s)://[***LIVY-FOR-SPARK3-SERVER-HOST***]:[***LIVY-FOR-SPARK3-SERVER-PORT***]   
ssl_cert_ca_verify=false
security_enabled=true
[notebook]
[[interpreters]]
[[[sparksql]]]
name=Spark SQL
interface=livy
```
note
- Ensure that the Spark application is not on the blocked list.
- Set ssl_cert_ca_verify=false if an SSL certificate is not present in Hue's truststore. ssl_cert_ca_verify=true if your environment is a secured environment.
- Set security_enabled=true for Kerberized clusters.
Restart the affected services.

You can now select the Spark SQL dialect on the Hue editor and run Spark queries from Hue.

Enabling Spark 3 engine in Hue

We want your opinion

How can we improve this page?