Impala entity metadata migration

Impala metadata entities are fully migrated from Navigator to Atlas.

The following sections describe how metadata is mapped from Navigator to Atlas; if Atlas requires metadata that wasn't available in Navigator, the migration notes describe how the Atlas metadata values are generated.

For entity metadata that is common to all entities, see System metadata migration.

Impala Process

Navigator impala_operation entities are migrated to Atlas impala_process entities.

Navigator Metadata Atlas Metadata Migration Notes
inputs inputs Points to the input entities as relationship attributes.
outputs outputs Points to the output entities as relationship attributes.
queryHash Not used in Atlas.
queryText attributes.queryText Not used currently in Atlas (see Impala Process Execution).
sourceId Not used in Atlas.
unparsed Not used in Atlas.
wfIds Not used in Atlas.
attributes.endTime Not used currently in Atlas.
attributes.operationType Defaults to null.
attributes.qualifiedName Generated as a string with the input entities and output entities, where each entity is noted by <asset_qualifiedName>:<createTime> and entries are separated by colons, and an arrow shows the break between input and output entities. For example:
dbblue.table_aqua@clustercolor:1589373411000->dbblue.table_teal@clustercolor:1589390050000
attributes.queryGraph Defaults to null.
attributes.queryId Defaults to null.
attributes.recentQueries Defaults to null.
attributes.startTime Not used currently in Atlas.
attributes.userName Not used currently in Atlas.

Impala Column Lineage

Navigator impala_sub_operation entities are migrated to Atlas impala_column_lineage entities.

Navigator Metadata Atlas Metadata Migration Notes
inputs attributes.inputs Points to the input column entities as relationship attributes.
outputs attributes.outputs Points to the output column entities as relationship attributes.
firstClassParentId attributes.query Points to the parent impala_process entity as a relationship attribute.
originalName attributes.qualifiedName Generated as a string with the input entities, output entities, and target column name, where each entity is noted by <column_qualifiedName>:<createTime> and entries are separated by colons, and an arrow shows the break between input and output entities. For example:
dbblue.table_aqua@clustercolor:1589373411000->dbblue.table_teal@cm:1589390050000:column_beach
attributes.dependencyType Set to "SIMPLE".
attributes.expression Defaults to null.

Impala Process Execution

Navigator impala_operation_execution entities are migrated to Atlas impala_process_execution entities.

Navigator Metadata Atlas Metadata Migration Notes
inputs inputs Points to the input entities as relationship attributes.
outputs outputs Points to the output entities as relationship attributes.
ended attributes.endTime
operation attributes.process Points to the parent impala_process entity as a relationship attribute.
originalName attributes.queryText
principal attributes.userName
started attributes.startTime
attributes.hostName Defaults to null.
attributes.qualifiedName Generated as a string with the input entities, output entities, and execution start and end timestamps, where each entity is noted by <asset_qualifiedName>:<createTime> and entries are separated by colons, and an arrow shows the break between input and output entities. For example:
dbblue.table_aqua@clustercolor:1589373411000->dbblue.table_teal@clustercolor:1589390050000:1589390056000:1589390058000
attributes.queryGraph Defaults to null.
attributes.queryId Defaults to null.
attributes.queryPlan Set to "Not Supported".