Iceberg data types

References include Iceberg data types and a table of equivalent SQL data types by Impala SQL engine types.

Iceberg supported data types

Table 1.
Iceberg data type SQL data type Impala
binary BINARY
boolean BOOLEAN BOOLEAN
date DATE DATE
decimal(P, S) DECIMAL(P, S) DECIMAL (P, S)
double DOUBLE DOUBLE
fixed(L) Not supported
float FLOAT FLOAT
int TINYINT, SMALLINT, INT INTEGER
list ARRAY Read only
long BIGINT BIGINT
map MAP Read only
string VARCHAR, CHAR STRING
struct STRUCT Read only
time Not supported
timestamp TIMESTAMP TIMESTAMP
timestamptz TIMESTAMP WITH LOCAL TIME ZONE

Read timestamptz into TIMESTAMP values

Writing not supported

uuid none Not supported

Data type limitations

An implicit conversion to an Iceberg type occurs only if there is an exact match; otherwise, a cast is needed. For example, to insert a VARCHAR(N) column into an Iceberg table you need a cast to the VARCHAR type as Iceberg does not support the VARCHAR(N) type. To insert a SMALLINT or TINYINT into an Iceberg table, you need a cast to the INT type as Iceberg does not support these types.

Iceberg supports two timestamp types:
  • timestamp (without timezone)
  • timestamptz ( with timezone)
In Spark 3.3 and earlier, Spark SQL supports a single TIMESTAMP type, which maps to the Iceberg timestamptz type. However, Impala is unable to write to Iceberg tables with timestamptz columns. To create Iceberg tables from Spark with timestamp rather than timestamptz columns, set the following configurations to true:
  • spark.sql.iceberg.handle-timestamp-without-timezone
  • spark.sql.iceberg.use-timestamp-without-timezone-in-new-tables

Configure these properties only on Spark 3.3 and earlier.

Spark still handles the timestamp column as a timestamp with local timezone. Inconsistent results occur unless Spark is running in UTC.

Unsupported data types

Impala does not support the following Iceberg data types:
  • TIMESTAMPTZ (only read support)

  • FIXED

  • UUID