Security-Hardened Spark image migration guide
From Cloudera Data Engineering version 1.24.1, the Redhat-based images for Apache Spark 3.x versions were deprecated and Security-Hardened images are provided for runtime usage. From Cloudera Data Engineering version 1.26.2, the support for the Redhat-based images are removed and only Security-Hardened images are supported for Runtime usage.
Before performing the in-place upgrade or the backup and restore procedure, check Migrating Cloudera Data Engineering Services deployed on deprecated Redhat UBI images to Security-Hardened images.
When manually creating clusters through the Cloudera Data Engineering UI, the API, or the CDP CLI, the default option is the security hardened image.
Steps:
- Migrate your workloads to the Security-Hardened images by creating new Virtual Clusters
with the Security-Hardened images under the same Cloudera Data Engineering
Service.
For more information, see Migrating Cloudera Data Engineering Services deployed on deprecated Red Hat UBI images to Security-Hardened images.
- Perform the in-place upgrade or the backup and restore procedure.
- Resolve any incompatibilities relating to your Cloudera Data Engineering applications.
For any packages excluded from the security hardened image, missing for Python or Java modules, you can build a custom image with the required libraries from the security hardened base Spark runtime image. For more information, see Using Custom Spark Runtime Docker Images via API/CLI.
