Spark and Livy on FIPS 140-3 clusters

Overview

Spark 3 and Livy for Spark 3 run on clusters that are already configured for FIPS 140-3. Platform preparation (host FIPS mode, validated cryptographic modules, and JDK configuration) is documented in the Cloudera platform FIPS installation guides, not in this Runtime library.

In Cloudera Runtime 7.3.2.10000 SP1, Spark 3 includes updated cryptographic modules and Cloudera SASL support for FIPS 140-3. Livy for Spark 3 uses the same cluster TLS and Kerberos settings as Spark 3.

Default JVM trust store on BCFKS clusters

On FIPS-enabled hosts, the JVM default trust store (java.home/lib/security/cacerts) is often a Bouncy Castle FIPS (BCFKS) keystore. Spark and Livy must supply the trust store password when the JVM does not already have javax.net.ssl.trustStorePassword set.

Spark 3

Set spark.ssl.defaultTrustStorePassword. When this property is set, Spark adds -Djavax.net.ssl.trustStorePassword=<password> to driver and executor JVM options at submit time and when a SecurityManager is created (for example, Spark History Server).

Livy for Spark 3

Set livy.ssl.defaultTrustStorePassword in Livy server configuration. When set, the Livy server sets javax.net.ssl.trustStorePassword on the Livy JVM if it is not already defined.

On Cloudera Manager-managed clusters, these properties may be populated automatically when the default cacerts keystore is BCFKS. If Spark History Server or Livy fail to start with BCFKS MAC or invalid keystore errors, verify the trust store password configuration and the BCFKS cacerts conversion on cluster hosts.

Spark Atlas Connector

The Spark Atlas Connector does not ship a separate FIPS cryptographic module. Lineage collection uses Spark 3 and Apache Atlas on the cluster (TLS, Kerberos, and Atlas REST). Ensure Spark 3 and Atlas meet your FIPS requirements.