Automatic Invalidation/Refresh of Metadata

New Default Behavior

When raw data is ingested into Tables, new HMS metadata and filesystem metadata are generated. In CDH, to pick up this new information, you must manually issue an Invalidate or Refresh command. However in CDP, this feature is controlled by the hms_event_polling_interval_s flag. This flag is set to 2 seconds by default. This option automatically refreshes the tables as changes are detected in HMS. When automatic invalidate/refresh of metadata is enabled, the Catalog Server polls Hive Metastore (HMS) notification events at a configurable interval and automatically applies the changes to Impala catalog. If specific tables that are not supported by event polling need to be refreshed, you must run a table level Invalidate or Refresh command.

For more information on Automatic Invalidation of metadata, see Automatic Invalidation