High-level transition process

Before transitioning from Navigator to Apache Atlas, review the transition paths. You must extract, transform, and import the content from Navigator to Apache Atlas. After the transition, services start producing metadata for Atlas and audits for Ranger.

There are two main paths that describe a Navigator-to-Atlas transition scenario:

Upgrading Cloudera Manager to CDP 7 and upgrading all of your CDH clusters to CDP Runtime.
In this case, you can stop Cloudera Navigator after migrating its content to Atlas.
Upgrading Cloudera Manager to CDP 7 but managing some or all of your existing CDH clusters as CDH 5.x or 6.x.
In this case, CDP runs Cloudera Navigator to continue extracting metadata and audits from existing CDH clusters and runs Atlas and Ranger to support metadata and audit extraction from new or potential new CDP runtime clusters.

In both scenarios, you'll complete the upgrade of Cloudera Manager first. While Cloudera Manager is upgrading, Navigator pauses collection of metadata and audits from cluster activities. After the upgrade is complete, Navigator processes the queued metadata and audits.

In the timeline diagrams that follow, the blue color indicates steps that because you trigger them manually, you can control their timing.

The transition of Navigator content to Atlas occurs in the upgrade from CDH to CDP. The transition involves three phases:

Extracting metadata from Navigator.
The Atlas installation includes a script (cnav.sh) that calls Navigator APIs to extract all technical and business metadata from Navigator. The process takes about 4 minutes per one million Navigator entities. The script compresses the result and writes it to the local file system on the host where the Atlas server is installed. Plan for about 100 MB for every one million Navigator entities; lower requirements for larger numbers of entities.
Transforming the Navigator metadata into a form that Atlas can consume.
The Atlas installation includes a script (nav2atlas.sh) that converts the extracted content and again compresses it and writes it to the local file system. This process takes about 1.5 minutes per million Navigator entities. The script compresses the results and writes it to the local file system on the host where the Atlas server is installed. Plan for about 100 to 150 MB for every one million Navigator entities; higher end of the range for larger numbers of entities.
Importing the transformed metadata into Atlas.
After the CDP upgrade completes, Atlas starts up in "migration mode," where it waits to find the transformed data file and does not collect metadata from cluster services. When the transformation is complete, Atlas begins importing the content, creating equivalent Atlas entities for each Navigator entity. This process takes about 35 minutes for each one million Navigator entities, counting only the entities that are migrated into Atlas.

To make sure you don't miss metadata for cluster operations, give time after the CM upgrade and before the CDH upgrade for Navigator to process all the metadata produced by CDH service operations. See Navigator Extraction Timing for more information.

You can start extracting metadata from Navigator as soon as the CDP parcel is deployed on the cluster. After CDP is started, Navigator no longer collects metadata or audits from the services on that cluster; instead services produce metadata for Atlas and audits for Ranger.

The following topics describe the details of the events in these timelines: