Schema evolution feature

You learn that the Hive or Impala schema changes when the associated Iceberg table changes. You see examples of changing the schema.

Although you can change the schema of your table over time, you can still read old data files because Iceberg uniquely identifies schema elements. The Iceberg table schema is synchronized with the Hive/Impala table schema. A change to the schema of the Iceberg table by an outside entity, such as Spark, changes the corresponding Hive/Impala table. You can change the Iceberg table using ALTER TABLE to make the following changes:

From Hive:

  • Add a column
  • Replace a column
  • Change a column type or its position in the table

From Impala:

  • Add a column
  • Rename a column
  • Drop a column
  • Change a column type or its position in the table
An unsafe change to a column type, which would require updating each row of the table for example, is not allowed. The following type changes are safe:
  • int to long
  • float to double
  • decimal(P, S) to decimal(P', S) if precision is increased
You can drop a column by changing the old column to the new column.

Hive syntax

ALTER TABLE table_name ADD COLUMNS (col_name type[, …])

ALTER TABLE table_name CHANGE COLUMN col_name1 col_name2 type

ALTER TABLE table_name CHANGE COLUMN col_name1 col_name2 type FIRST

ALTER TABLE table_name CHANGE COLUMN col_name1 col_name1 int

ALTER TABLE table_name REPLACE COLUMNS (col_name type)

Impala syntax

ALTER TABLE table_name ADD COLUMNS(col_name type[, …])

ALTER TABLE table_name CHANGE COLUMN col_name1 col_name2 type FIRST

ALTER TABLE table_name DROP COLUMN col_name

Hive examples

ALTER TABLE t ADD COLUMNS(message STRING, price DECIMAL(8,1));

ALTER TABLE t REPLACE COLUMNS (i int comment ‘...’, a string, ...);

ALTER TABLE t CHANGE COLUMN d d DECIMAL (22, 3) AFTER i;

Impala examples

ALTER TABLE t ADD COLUMNS(message STRING, price DECIMAL(8,1));

ALTER TABLE t DROP COLUMN i;

ALTER TABLE t CHANGE COLUMN d d DECIMAL (22, 3) AFTER i;