Custom Lineage Connector

Tool Permissions Prerequisites

Enable read permission for Cloudera Octopai Windows NT User to the folder (which contains the filled template files).

Setting up Custom Lineage Connector Metadata Source

Metadata Sources are set on the Cloudera Octopai Client.

1. From the Cloudera Octopai Client, click the Custom Lineage Connector.
2. In the New Metadata Source wizard, choose the authentication method between a Local File or a Github File, and enter the appropriate information:
   • Local File: Choose the Local File option if you want to upload files from a local directory for data lineage analysis.
   • Github File: Choose the Github File option if you want to connect to your Github repository to extract the required files for data lineage analysis.
   • Local File
   • Github File

   1. Connection Name: Provide a meaningful name for this connection to help you easily identify it later in the Cloudera Octopai application.
   2. Tool Name: Specify the tool or language environment you are using.
   3. Connection Type: Specify if the custom source is a database object, custom ETL process, or a report.
   4. Input File: Select the input CSV template file.

   
   
   

   1. Connection Name: Provide a meaningful name for this connection to help you easily identify it later in the Cloudera Octopai application.
   2. Tool Name: Specify the tool or language environment you are using.
   3. Connection Type: Specify if the custom source is a database object, custom ETL process, or a report.
   4. Personal Access Token (PAT): Enter your Github Personal Access Token to authenticate and authorize the connector to access your repository.
   5. Repository (owner/repo): Enter the repository name in the format 'owner/repository-name'.
   6. Branch: Specify the branch within the repository from which you want to extract files (for example, 'main').
   7. Folder path inside the repository: Provide the specific path to the directory within the repository that contains the files you want to include.

   
   
   

3. Click Next > Finish > Run.