Flexible partitioning
Iceberg partition evolution, which is a unique Iceberg feature, and the partition transform feature, greatly simplify partitioning tables and changing partitions.
Partitions based on transforms are stored in the Iceberg metadata layer, not in the directory structure. You can change the partitioning completely, or just refine existing partitioning, and write new data based on the new partition layout--no need to rewrite existing data files. For example, change a partition by month to a partition by day.
Use partition transforms, such as IDENTITY, TRUNCATE, BUCKET, YEAR, MONTH, DAY, HOUR. Iceberg
solves scalability problems caused by having too many partitions. Partitioning can also include
columns with a large number of distinct values. Partitioning is hidden in the Iceberg
metadata layer, eliminating the need to explicitly write partition columns (YEAR, MONTH for
example) or to add extra predicates to queries for partition
pruning.
SELECT * FROM tbl WHERE ts = ‘2023-04-21 20:56:08’
AND YEAR = 2023 AND MONTH = 4 AND DAY = 21