Using the workload secret in the Spark application code

To use the workload secret credentials, you can read the credentials that are mounted into the Spark drivers and executors as read-only files.

The workload secrets are mounted into the Spark drivers and executors in this path: /etc/dex/secrets/<workload-credential-name>/<credential-key-1> /etc/dex/secrets/<workload-credential-name>/<credential-key-2>

Example workload credentials:

The workload-credential was created with the payload below.

{
  "workloadCred": {
    "aws-secret": "secret123",
    "db-pass": "dbpass123"
  },
  "name": "workload-cred-1",
  "type": "workload-credential",
  "description": "workload credential description"
}

The secrets can be read as local files from the paths below within the Spark drivers and executors:

/etc/dex/secrets/workload-cred-1/aws-secret

/etc/dex/secrets/workload-cred-1/db-pass

Example of a Pyspark application code to read a secret:

from pyspark.sql import SparkSession
 
spark = SparkSession \
    .builder \
    .appName("Sample DB Connection") \
    .getOrCreate()
 
# read the password from the local file
dbPass=open("/etc/dex/secrets/workload-cred-1/db-pass").read()
 
# use the password in a jdbc connection
jdbcDF= spark.read \