Automatic Invalidation of Metadata Cache
To keep the size of metadata bounded, the Impala Catalog Server periodically scans all the tables and invalidates those not recently used.
There are two types of configurations in Catalog Server that control the automatic invalidation of metadata in the Catalog Server Command Line Argument Advanced Configuration Snippet (Safety Valve) field in Cloudera Manager.
- Time-based cache invalidation
Catalogdinvalidates tables that are not recently used in the specified time period (in seconds).
‑‑invalidate_tables_timeout_sflag needs to be applied to both
- Memory-based cache invalidation
When the memory pressure reaches 60% of JVM heap size after a Java garbage collection
catalogd, Impala invalidates 10% of the least recently used tables.
‑‑invalidate_tables_on_memory_pressureflag needs to be applied to both
Automatic invalidation of metadata provides more stability with lower chances of running out of memory, but the feature could potentially cause performance issues and may require tuning.