Enabling Spark authentication
Spark authentication here refers to an internal authentication mechanism, and not to Kerberos authentication, which is enabled automatically for all Cloudera Data Platform deployments.
Minimum Required Role: Security Administrator (also provided by Full Administrator)
Spark has an internal mechanism that authenticates executors with the
driver controlling a given application. This mechanism is enabled using
the Cloudera Manager Admin Console, as detailed below. Cluster
administrators can enable the
mechanism to authenticate the various processes that support a Spark
To enable this feature on the cluster:
- In the Cloudera Data Platform (CDP) Management Console, go to Data Hub Clusters.
- Find and select the cluster you want to configure.
- Click the link for the Cloudera Manager URL.
- Go to .
- Scroll down to the Spark Authentication
setting, or search for
spark.authenticateto find it.
- In the Spark Authentication setting, click the checkbox next to the Spark (Service-Wide) property to activate the setting.
- Enter the reason for the change at the bottom of the screen, and then click Save Changes.
- Restart YARN:
- Select .
- Select Restart from the Actions drop-down selector.
- Re-deploy the client configurations:
- Select Deploy Client Configurations from the Actions drop-down selector.
- Restart stale services.