Spark Encryption

Spark supports the following means of encrypting Spark data at rest, and data in transit.

Enabling Spark Encryption Using Cloudera Manager

Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)

  1. Open the Cloudera Manager Admin Console and go to the Spark service.
  2. Click the Configuration tab.
  3. (Prerequisite) Search for the Spark Authentication property and make sure it has been enabled. If this property is not set, the following settings to enable encryption will not work.
  4. Search for the Enable Network Encryption property. Use the checkbox to enable encrypted communication between Spark processes belonging to the same application.
  5. Search for the Enable I/O Encryption property. Use the checkbox to enabled encryption for temporary shuffle and cache files stored by Spark on local disks.
  6. Enter a Reason for change, and then click Save Changes to commit the changes.
  7. Redeploy client configuration.
  8. Restart stale services (if indicated by Cloudera Manager).