Configuration options for Spark to work with Ozone File System (ofs)
After setting up ofs
, you can make configuration updates specific to
components such as Spark to ensure that they work with Ozone.
To run Spark jobs with ofs
on a secure Kerberos-enabled cluster, ensure
that you assign a valid URI by setting the value of the Spark Client Advanced
Configuration Snippet (Safety Valve) property for the
spark.conf
or the spark-defaults.conf
file through
the Cloudera Manager web UI.
For
example:
spark.yarn.access.hadoopFileSystems=ofs://service1/vol1/bucket1/