Cloudera Docs

Understanding the Cloudera Observability metastore analytics UI elements

Learn about the Cloudera Observability Metastore Analytics UI elements that display the Hive Metastore (HMS) metadata information about your tables.

About the Cloudera Observability data temperatures

In Cloudera Observability Hot and Cold represents the number of times a query accesses the table. Where, the color and the depth of color represents the number of times a query accesses the table in relation to all the other tables in your system:
  • Hot tables (red) - are tables that were frequently accessed during the selected time-period.
  • Cold tables (blue) - are tables that were infrequently accessed during the selected time-period. This includes tables where no queries (zero) accessed their data during the selected time-period and are considered the coldest tables by definition.

About the data temperature charts

The Cloudera Observability Metastore Analytics feature has several UI elements that describe your table data. The following charts display the data temperature charts and chart widgets:
  • The Data Temperature heat map chart widget automatically displays the top 25 hot tables and the bottom 25 cold tables in the Hot Tables and the Cold Tables heat map charts. Located on the Summary page of the environment's cluster, it displays the tables queried by both the Hive and Impala engines.

  • The Hot Table heat map chart is found on the Hive and Impala engine Summary pages and displays the tables that were frequently queried by their engine using the Hive Metastore metadata respectively.

Clicking the table's name of interest in either the Hot or Cold table heat map chart or the List of Tables View in the HIVE METASTORE section, opens the table's Overview Details side drawer panel, which displays more information about the table.

About the Overview Details side drawer panel

The Overview Details side drawer panel describes more information about the table. Based on the table’s HMS metadata, such as the table's schema, database location, partitions, structure, and relationships, the information displayed may vary. It also describes the table's columns, such as the column names and their data types, and the table's metadata properties that include user-defined as well as predefined key-value pairs.

It is accessed by clicking on the table's name of interest in either the Hot Table or Cold Table heat map chart or by selecting a table name in the List of Tables View in the HIVE METASTORE section.

Each tab in the panel displays the selected table's Volume, Rows, Partitions, and Total File values. The panel contains the following tabs:
  • Details
  • Columns
  • Properties
The HMS metadata parameters displayed are dependent on the table’s underlying data on which it is built. The following tables describe the most common parameters displayed in the Details, Columns, and Properties tabs:
Table 1. Details
Parameter Description
Historical Trend chart Displays the historical values for the Rows, Data Volume, and Partitions.
Database name The database in which the table resides.
Compressed Displays a True or False value depending on whether data compression been applied.
Location The table’s location in HDFS.
Partition Keys The name/s of the partition keys that are responsible for data distribution across the nodes.
Raw Data Size The raw data size of the table, in the nearest byte unit.
Storage Format The table’s storage format, such as but not limited to:
  • JDBC
  • LazySimple
  • Orc
  • Parquet
Stats Enabled Displays a True or False value depending on whether statistics were enabled.
Table Type The table’s type, such as but not limited to:
  • EXTERNAL_TABLE, which defines a table whose data is stored in the location specified during table creation.
  • MANAGED_TABLE, which defines a table whose data is stored in the warehouse directory.
  • VIRTUAL_TABLE, which defines a table that is the result of a query which has not materialized and whose data is not stored.
Transactional Displays a True or False value depending on whether the table contains one or more ACID semantic properties.
Created The date when the table was created, using the MM- DD-YYYY date format. For example, 06-25-2023.
Table 2. Columns
Parameter Description
Column Name Lists the Column field names.
Type The Hive data type, as one of the following:
  • bigint
  • binary
  • boolean
  • chara
  • date
  • decimal
  • double
  • float
  • int
  • smallint
  • string
  • timestamp
  • tinyint
  • varchar
Comment An informative note about the column that was added during table creation.
Table 3. Properties
Parameter Sections Description
Table Properties Predefined and user-defined metadata key-value pair properties.
SerDe Properties Serialization and deserialization properties.
Storage Descriptor Properties Metadata that describes the physical storage properties of the data residing in the table.

About the List of Tables View

The HMS Extract, which is updated daily and is displayed in the List of Tables View in the HIVE METASTORE section, lists the details about each table available in your system, regardless of whether they have been queried or not.

The List of Tables View lists the following:
Column Name Description
Table The name of the table.
Database The database in which the table resides.
Partitions The number of partitions.
Volume The total table size in bytes.
Rows The number of records in the table.
Files The number of files that make up the table.
Frequency of Access The number of times queries have accessed the table.
Storage Format The table’s storage format, such as but not limited to:
  • JDBC
  • LazySimple
  • Orc
  • Parquet
Table Type The table’s type, such as but not limited to:
  • EXTERNAL_TABLE, which defines a table whose data is stored in the location specified during table creation.
  • MANAGED_TABLE, which defines a table whose data is stored in the warehouse directory.
  • VIRTUAL_TABLE, which defines a table that is the result of a query which has not materialized and whose data is not stored.