Creating a new Iceberg table from Spark 3
You can create an Iceberg table using Spark SQL.
An example Spark SQL creation command to create a new Iceberg table is as
follows:
spark.sql("""CREATE EXTERNAL TABLE ice_t (idx int, name string, state string)
USING iceberg
PARTITIONED BY (state)""")For information about creating tables, see the Iceberg documentation.
Creating an Iceberg table format v2
To use the Iceberg table format v2, set the format-version property to
2 as shown below:
CREATE TABLE logs (app string, lvl string, message string, event_ts timestamp) USING iceberg TBLPROPERTIES ('format-version' = '2')<delete-mode>
<update-mode> and <merge-mode> can be specified
during table creation for modes of the respective operation. If unspecified, they default to
merge-on-read.
Creating an Iceberg table format v3 [Technical Preview]
To use the Iceberg table format v3, set the format-version property
to 3 as shown below. In Cloudera Runtime 7.3.2 SP1,
Spark supports deletion vectors and row lineage tracking on V3 tables only; see Prerequisites and
limitations for using Iceberg in Spark for supported and unsupported V3
capabilities.
CREATE TABLE ice_3 (i INT, s STRING) USING iceberg TBLPROPERTIES ('format-version' = '3')Unsupported Feature: CREATE TABLE … LIKE
The
CREATE TABLE ... LIKE feature is not supported in
Spark:CREATE TABLE <target> LIKE <source> USING iceberg