Step 3: Register a GCP environment in CDP

The third (and last) step is to register your GCP environment in CDP. You will:

  • Use the credential created in Step1.
  • Point CDP to the resources created in Step 2.

Prerequisites

  • This assumes that you have installed CDP CLI. If you haven’t, refer to CLI slient setup.
  • You need an RSA key pair. You will be asked to provide a public key and you will use the matching private key for admin access to CDP instances.

Register a GCP environment using CDP CLI

  1. Open the terminal app on your computer.

  2. Create your environment using the following command. Replace the following with actual values:
    • <NAME_OF_YOUR_CDP_CREDENTIAL> - Replace this with the actual name that you provided on the CDP web UI in step 1.
    • <REGION> - Replace this with the ID of the region selected in step 2.
    • <RSA_PUBLIC_KEY> - Replace this with your RSA public key. You will use the matching private key for admin access to CDP instances.
    • <PREFIX> - Replace this with the prefix specified in step 2.
    • <PROJECT_ID> - Replace this with the ID of the GCP project specified in step 2.
    cdp environments create-gcp-environment  --environment-name <PREFIX>-cdp-env \
        --credential-name <NAME_OF_YOUR_CDP_CREDENTIAL> \
        --region "<REGION>" \
        --public-key "<RSA_PUBLIC_KEY>" \
        --log-storage storageLocationBase="gs://<PREFIX>-cdp-logs/",serviceAccountEmail="<PREFIX>-log-sa@<PROJECT_ID>.iam.gserviceaccount.com" \
        --existing-network-params networkName="<PREFIX>-cdp-network",subnetNames="<PREFIX>-cdp-network-subnet-1",sharedProjectId="<PROJECT_ID>"  \
        --enable-tunnel \
        --use-public-ip    
  3. Find your user CRN using the following command:
    user_crn=$(cdp iam get-user | jq -r .user.crn)
  4. Set the IDBroker mappings between users and service accounts using the following command. Replace the following with actual values:
    • <PREFIX> - Same as used earlier
    • <PROJECT_ID> - Same as used earlier
    • <USER_CRN> - Replace with your user CRN.
    cdp environments set-id-broker-mappings \
        --environment-name "<PREFIX>-cdp-env" \
        --baseline-role "<PREFIX>-rgraud-sa@<PROJECT_ID>.iam.gserviceaccount.com" \
        --data-access-role "<PREFIX>-dladm-sa@<PROJECT_ID>.iam.gserviceaccount.com" \
        --mappings accessorCrn="<USER_CRN",role="<PREFIX>-dladm-sa@<PROJECT_ID>.iam.gserviceaccount.com"
  5. Create the Data Lake using the following command. Replace the following with actual values:
    • <PREFIX> - Same as used earlier
    • <PROJECT_ID> - Same as used earlier
    cdp datalake create-gcp-datalake --datalake-name <PREFIX>-cdp-dl \
        --environment-name <PREFIX>-cdp-env \
        --cloud-provider-configuration "serviceAccountEmail=<PREFIX>-idb-sa@<PROJECT_ID>.iam.gserviceaccount.com,storageLocation=gs://<PREFIX>-cdp-data"
  6. Once your environment is created, its status will change to Available and the Data Lake status will change to Running.

Once your environment is running, you can start creating Data Hub clusters.