Cloudera Docs

Disabling metadata synchronization

By default, metadata is synchronized when refreshed/invalided across multiple Impala Virtual Warehouses that share a Database Catalog. When you run a command to refresh a table or invalidate metadata in any one of these Impala Virtual Warehouses, metadata is refreshed/invalidated in parallel. You learn how to disable this feature if you do not want to synchronize metadata for some reason.

By default, running a command from an Impala Virtual Warehouse to refresh tables or invalidate metadata raises events in the Hive metastore. Catalog daemons process events synchronously across all Virtual Warehouses that share metadata. Metadata is refreshed/invalidated in parallel across all your Virtual Warehouses. The enable_reload_events configuration parameter enables or disables raising events for metadata synchronization, effectively enabling or disabling the synchronization:
  • enable_reload_events = true (default)
  • enable_reload_events = false

Metadata has to be synchronized when you make changes to the data that causes inconsistent file metadata. For example, external clients create, update, and delete partitioned files on the object store.

To synchronize the file metadata in the catalog cache, you run a refresh/invalidate command in a single virtual warehouse. The catalog cache across all the Virtual Warehouses that share a Database Catalog is synchronized and consistent.

In this task, you disable metadata synchronization.

  • The DWAdmin role is required to configure CDW.
  • An Impala Virtual Warehouse running 1.6.3-b319 (released May 5, 2023) or later
  1. Navigate to Data Warehouses > Virtual Warehouses, select your Impala Virtual Warehouse, and click > Edit.
  2. Click CONFIGURATIONS > Impala catalogd.
  3. Select flagfile from the drop-down list, and configure the enable_reload_events key to the value false.
  4. Apply the change.