Cloudera Docs

Schema evolution feature

You learn that the Hive or Impala schema changes when the associated Iceberg table changes. You see examples of changing the schema.

Although you can change the schema of your table over time, you can still read old data files because Iceberg uniquely identifies schema elements. A schema change results in a new metadata.json and a commit, but does not create a new snapshot.

The Iceberg table schema is synchronized with the Hive/Impala table schema. A change to the schema of the Iceberg table by an outside entity, such as Spark, changes the corresponding Hive/Impala table. You can change the Iceberg table using ALTER TABLE to make the following changes:

From Hive:

  • Add a column
  • Replace a column
  • Change a column type or its position in the table

From Impala:

  • Add a column
  • Rename a column
  • Drop a column
  • Change a column type
An unsafe change to a column type, which would require updating each row of the table for example, is not allowed. The following type changes are safe:
  • int to long
  • float to double
  • decimal(P, S) to decimal(P', S) if precision is increased
You can drop a column by changing the old column to the new column.

Hive syntax

ALTER TABLE table_name ADD COLUMNS (col_name type[, …])

ALTER TABLE table_name CHANGE COLUMN col_old_name col_new_name type

ALTER TABLE table_name CHANGE COLUMN col_old_name col_new_name type [FIRST|AFTER col_name] [existing_col_name
]
ALTER TABLE table_name REPLACE COLUMNS (col_name type)

Impala syntax

ALTER TABLE table_name ADD COLUMNS(col_name type[, …])

ALTER TABLE table_name CHANGE COLUMN col_old_name col_new_name type

ALTER TABLE table_name DROP COLUMN col_name

Hive examples

ALTER TABLE t ADD COLUMNS(message STRING, price DECIMAL(8,1));

ALTER TABLE t REPLACE COLUMNS (i int comment ‘...’, a string, ...);

ALTER TABLE t CHANGE COLUMN col_x col_x DECIMAL (22, 3) AFTER col_y;

Impala examples

ALTER TABLE ice_12 ADD COLUMNS(message STRING, price DECIMAL(8,1));

ALTER TABLE ice_12 DROP COLUMN i;

ALTER TABLE ice_12 CHANGE COLUMN s str STRING;