Using Apache Iceberg in Cloudera Data Engineering
Cloudera Data Engineering supports Apache Iceberg which provides a table format
for huge analytic datasets in the cloud. Iceberg enables you to work with large tables,
especially on object stores, and supports concurrent reads and writes on all storage media. You
can use Cloudera Data Engineering virtual clusters running Spark 3 to interact with Apache
Iceberg tables.
Prerequisites and limitations for using Iceberg Cloudera Data Engineering , Spark, and Data Lake to use with Apache Iceberg Cloudera Data Engineering . Accessing Iceberg tables Cloudera uses Apache Ranger to provide centralized security administration and management. The Ranger Admin UI is the central interface for security administration. You can use Ranger to create two policies that allow users to query Iceberg tables.Creating Virtual Cluster with Spark 3 Creating a new Iceberg table from Spark 3 Configuring Hive Metastore for Iceberg column changes Importing and migrating Iceberg table in Spark 3 Importing and migrating Iceberg table format v2 Configuring Catalog Loading data into an unpartitioned table Querying data in an Iceberg table Updating Iceberg table data Iceberg library dependencies for Spark applications 