On-demand Metadata
With the on-demand metadata feature, the Impala coordinators pull
metadata as needed from catalogd
and cache it locally. The
cached metadata gets evicted automatically under memory pressure.
The granularity of on-demand metadata fetches is at the partition level between the
coordinator and catalogd
. Common use cases like add/drop partitions do
not trigger unnecessary serialization/deserialization of large metadata.
The feature can be used in either of the following modes.
- Metadata on-demand mode
- In this mode, all coordinators use the metadata on-demand.
- Mixed mode
- In this mode, only some coordinators are enabled to use the metadata on-demand.
Limitation:
Global INVALIDATE METADATA
statement is not supported
when this feature is enabled. If your workload requires global
INVALIDATE METADATA
, disable this feature.