Compatibility for Cloudera AI 1.5.5 SP1, Cloudera Runtime components and Data Visualization

Explore Cloudera AI 1.5.5 SP1 and its compatibility with Cloudera Runtime components and with Cloudera Data Visualization across various versions.

important

If you switch from the Red Hat Universal Base Image (Redhat UBI image) to the security hardened image, note that the updated component versions and libraries can impact the Cloudera AI operation. Cloudera strongly recommends testing the new implementation in a non-production environment first.
For backward compatibility, you can continue using the Redhat UBI image. The security hardened image is the recommended image; however, note that it requires testing and validation.
For more information, see Security hardened Spark image migration guide.
From Cloudera Data Services on premises 1.5.5 onwards, Spark 3.4 is no longer available.
Hive Warehouse Connector (HWC) is supported on all Spark 2.x versions. However, on Spark 3.x versions, HWC is supported on Spark 3.3.2 and higher.

Cloudera AI version

The following table lists the supported Cloudera AI versions for each Cloudera Data Services on premises release.

Table 1. Cloudera AI versions for each Cloudera Data Services on premises release
Cloudera Data Services on premises version	Cloudera AI version
1.5.5 SP1	Cloudera AI Workbench: 2.0.51-b321 Cloudera AI Registry: 1.9.0-b55 Cloudera AI Inference service: 1.6.0-b73

The following table displays the compatibility between Cloudera Runtime components and Cloudera AI.

Table 2. Cloudera AI compatibility with Cloudera Runtime component details and with Cloudera Data Visualization for Security Hardened image
Cloudera Runtime Version	Spark 3.2.x	Spark 3.3.x	Spark 3.5.x	Airflow	Iceberg	Kubernetes	Cloudera Data Visualization (Data tab)
7.1.7 SP 3	Spark 3.2.3 Scala 2.12.15 Python 3.9.23 Java 11.0.27	NA	NA	Airflow 2.11.0 Python 3.11.14 Java 17.0.16	Iceberg 0.14.1	1.31	8.0.6-b93
7.1.9 SP1	Spark 3.2.3 Scala 2.12.15 Python 3.9.23 Java 11.0.27	Spark 3.3.2 Scala 2.12.15 Python 3.10.18 Java 17.0.16	NA	Airflow 2.11.0 Python 3.11.14 Java 17.0.16	Iceberg 1.3.0	1.31	8.0.6-b93
7.3.1	NA	NA	Spark 3.5.4 Scala 2.12.18 Python 3.11.13 Java 17.0.16	Airflow 2.11.0 Python 3.11.14 Java 17.0.16	Iceberg 1.5.2	1.31	8.0.6-b93

Table 3. Cloudera AI compatibility with Cloudera Runtime component details and with Cloudera Data Visualization for Redhat insecure image
Cloudera Runtime Version	Spark 2.4.x	Spark 3.2.x	Spark 3.3.x	Airflow	Iceberg	Kubernetes	Cloudera Data Visualization (Data tab)
7.1.7 SP3	Spark 2.4.7 Scala 2.11.12 Python 2.7.18 Python 3.6.8 Java 1.8.0_452	Spark 3.2.3 Scala 2.12.15 Python 2.7.18 Python 3.6.8 Java 11.0.25	NA	Airflow 2.11.0 Python 3.11.14 Java 17.0.16	Iceberg 0.14.1	1.31	8.0.6-b93
7.1.9 SP1	Spark 2.4.7 Spark 2.4.8 Scala 2.11.12 Python 2.7.18 Python 3.6.8 Java 1.8.0_452	Spark 3.2.3 Scala 2.12.15 Python 2.7.18 Python 3.6.8 Java 11.0.25	Spark 3.3.2 Scala 2.12.15 Python 2.7.18 Python 3.8.17 Java 11.0.25	Airflow 2.11.0 Python 3.11.14 Java 17.0.16	Iceberg 1.3.0	1.31	8.0.6-b93