CDS 3 Powered by Apache Spark
CDS 3.3 Overview
CDS 3.3 Requirements
Installing CDS 3.3
Enabling CDS 3.3 with GPU Support
Set up a Yarn role group
Configure Shuffle Manager
Updating Spark 2 apps for Spark 3
Running Spark 3 Applications with CDS 3.3
Running applications with CDS 3.3 with GPU Support
CDS 3.3 Packaging, and Download
Using the CDS 3.3 Maven Repo
CDS 3.3 Maven Artifacts
Apache Spark 3 integration with Schema Registry
Configuration
Fetching Spark schema by name
Building and deploying your app
Running in a Kerberos-enabled cluster
Unsupported features
Cumulative hotfixes for CDS
Cumulative hotfix CDS 3.3.7190.2-1
Cumulative hotfix CDS 3.3.7190.3-1
Cumulative hotfix CDS 3.3.7190.4-1
Cumulative hotfix CDS 3.3.7190.5-2
Cumulative hotfix CDS 3.3.7190.7-2
Cumulative hotfix CDS 3.3.7190.8-2
Cumulative hotfix CDS 3.3.7190.9-1
Cumulative hotfix CDS 3.3.7190.10-1
Cumulative hotfix CDS 3.3.7191000.3-1
Cumulative hotfix CDS 3.3.7191000.4-1
Using Apache Iceberg in CDS
Prerequisites and limitations for using Iceberg in Spark
Accessing Iceberg tables
Editing a storage handler policy to access Iceberg files on HDFS or S3
Creating a SQL policy to query an Iceberg table
Creating a new Iceberg table from Spark 3
Configuring Hive Metastore for Iceberg column changes
Importing and migrating Iceberg table in Spark 3
Importing and migrating Iceberg table format v2
Configuring Catalog
Loading data into an unpartitioned table
Querying data in an Iceberg table
Updating Iceberg table data
Iceberg library dependencies for Spark applications