CDP Private Cloud Data Services 1.5.1 Release Summary

This release provides new features, enhancements and critical bug fixes. Upgrades are supported from CDP Private Cloud Data Services versions 1.4.1 and 1.5.0. Release highlights include:

Platform🔗

• Security
  • Fixes for CVEs
  • DirName Kerberos support for CDW and CML
• Data Services Enablements
  • Cadence support to enable CML DRS capability
  • Client configuration for enabling CDE with Kafka
  • ServiceDiscovery API to enable Data Services access to Base storage services
  • Spark3 client configurations for CML
  • Ozone integration with all Data Services
• Certifications
  • FreeIPA
  • OCP 4.11
  • RKE2 (v1.24) and Longhorn (1.4.2) version upgrades

Cloudera Data Engineering (CDE)🔗

• Scale and Performance
  • Upgrade to Airflow 2.3.4, plus performance improvements
  • Faster autoscaling with Gang scheduling
• Spark
  • Interactive sessions (PySpark/Scala) (Tech Preview)
  • Data connectors: Ozone, Iceberg v2 (Tech Preview)
• Automated Upgrade
  • Handles: job history, job logs, and persisting end points
• Migration
  • Spark: spark-submit migration tool
  • YARN to Quota management Elastic quotas
  • Airflow custom operators and plugins (API-only)

Cloudera Data Warehouse (CDW)🔗

• Iceberg v2 support on HDFS and Ozone (Tech Preview)
• CDW Impala Virtual Warehouse access control
• Impala Ozone performance improvements and Embedded Container Service support
• Configuration sync between Base and CDW
• Improved logging and troubleshooting
• Support for multiple CDW Private Cloud environments in the same AD
• Impala workload-aware autoscaling (Tech Preview)

Cloudera Machine Learning (CML)🔗

• CDSW-to-CML migration
• Single Node mode for CDSW-to-CML migration
• Spark pushdown to CDP Base
• Backup/restore on ECS and OCP (Tech Preview)
• Disable legacy engines
• External registry support
• Ozone support

A total of 1802 projects were completed for this release, including 69 new features and 1,203 other improvements and bug fixes.