Known Issues Iceberg

Learn about the known issues in Iceberg, the impact or changes to the functionality, and the workaround.

CDPD-57551: Performance issue can occur on reads after writes of Iceberg tables: Hive might generate too many small files, which causes performance degradation.; Maintain a relatively small number of data files under the iceberg table/partition directory to have efficient reads. To alleviate poor performance caused by too many small files, run the following queries:
TRUNCATE TABLE target; INSERT OVERWRITE TABLE target select * from target FOR SYSTEM_VERSION AS OF <preTruncateSnapshotId>;

CDPD-75422: Impala schema case sensitivity issue with workaround: Impala's schema is case insensitive, causing errors with mixed case schema elements created through Spark during predicate pushdown.; Create tables through Impala to ensure lower case schema.

Avoid upper case in Spark: Do not use upper case when creating tables through Spark.

Fix existing tables: Use ALTER TABLE to rename upper case columns:
ALTER TABLE `database`.`iceberg_table` CHANGE COLUMN ID id string;

Technical Service Bulletins

TSB 2024-746: Concurrent compactions and modify statements can corrupt Iceberg tables: Apache Hive (Hive) and Apache Impala (Impala) modify statements (DELETE/UPDATE/MERGE) on Apache Iceberg (Iceberg) V2 tables can corrupt the tables if there is a concurrent table compaction from Apache Spark. The issue happens when the compaction and modify statement run in parallel, and when the compaction job commits before the modify statement. In this case the position delete files of the modify statement still point to the old files. This means the following in case of

DELETE statements

Deleting records pointing to old files have no effect

UPDATE / MERGE statements

Deleting records pointing to old files have no effect

The table will also have the newly added data records

Rewritten records will still be active

This issue does not affect Apache NiFi (NiFi) and Apache Flink (Flink) as these components write equality delete files.
Knowledge article: For the latest update on this issue see the corresponding Knowledge article: TSB 2024-746: Concurrent compactions and modify statements can corrupt Iceberg tables