Load data inpath feature

From Impala, you can load Parquet or ORC data from a file in a directory on your file system or object store into an Iceberg table. You might need to set the mem_limit or pool configuration (max-query-mem-limit, min-query-mem-limit) to accommodate the load.

Impala syntax

LOAD DATA INPATH ‘<path to file>’ INTO table t;

Impala example

In this example, you create a table using the LIKE clause to point to a table stored as Parquet. This is required for Iceberg to infer the schema. You also load data stored as ORC.

CREATE TABLE test_iceberg LIKE my_parquet_table STORED AS ICEBERG;
SET MEM_LIMIT=1MB;
LOAD DATA INPATH 's3a://testbucket/files/my_data.parq' INTO TABLE test_iceberg;

LOAD DATA INPATH 's3a://testbucket/files/my_data.orc' INTO TABLE test2_iceberg;