Cloudera Docs

Navigation support for hive entities within Lineage

The Lineage in the Overview tab shows the chain of custody for the data from relevant metadata repositories such as Apache Atlas. Lineage overview shows both upstream paths (lineage) into and downstream paths (impact) out of a given asset.

When you click a Hive entity within a lineage, the Asset Details page of the selected Hive entity is displayed. In the Overview tab, you can change the detail level of the lineage with the Depth drop-down and Show Process:

  • The enriched_flight_data table (1) is created by joining the following source tables:

    • airline_operations.stg_flight_manifests@cm (3): This is the main source of transactional data.

    • airline_operations.dim_aircraft@cm (4): This is a dimension or lookup table. It provides supplemental information.

  • s3a://eng-sdx-weekly-qe/cloudera-catalog-hue-lake-4/7c002e988b2048f3b7197c5125694e7e/storage/warehouse/tablespace/external/hive/airline_operations.db/enriched_flight_data/@cm (5) is the physical path for storing airline_operations.stg_flight_manifests@cm and airline_operations.dim_aircraft@cm.

  • airline_operations.agg_route_performance@cm (7) is table created from aggregated some items from enriched_flight_data with an additional query.

  • s3a://eng-sdx-weekly-qe/cloudera-catalog-hue-lake-4/7c002e988b2048f3b7197c5125694e7e/storage/warehouse/tablespace/external/hive/airline_operations.db/agg_route_performance/@cm (6) is the physical path for storing airline_operations.agg_route_performance@cm.

  • airline_operations.route_performance_archive_hive@cm (8) is a direct copy of the aggregation table, intended for historical record-keeping in this example.

Alternatively, if you do not want to navigate away from the current page and want to view the information with respect to any entity, hover on the entity and click the information icon to view the details.

The screenshot depicts the slider information for the clicked entity:

This field shows the Hive operation writing data from airline_operations.enriched_flight_data@cm to the airline_operations.agg_route_performance@cm table.