Cloudera Docs

Setting up Cloudera Iceberg REST Catalog for data sharing

Learn how to perform the preparatory configurations in Cloudera to enable data sharing. These configurations are required for the creation of a data share in Cloudera and allow your clients to access data in Cloudera environments.

Installation and Upgrade Scenarios

Different versions of Cloudera Runtime and Data Lakes require different steps to enable data sharing.

Fresh 7.3.2.x Data Lake Installation

Complete the following steps after a fresh installation of 7.3.2.x Data Lake to enable data sharing.

  1. HMS Rest Catalog configuration

    Configure the Hive Metastore (HMS) service to serve as an Iceberg REST catalog. This allows clients to use REST Catalog APIs to access the required metadata files.

  2. Table 1. Knox topologies
    AWS environment Azure environment

    Editing Knox topologies

    Edit your Knox topologies to define the lifetime of your Knox tokens (lifetime of your Data Shares) and the number of external users.

    Declaring Knox topologies

    Create the Knox topologies to define for Knox how to proxy requests from external users.

  3. Knox configuration

    Configure properties in Knox to set up administrator privileges that allow creation of tokens.

  4. Table 2. Create a Data Share in Cloudera
    AWS environment Azure environment
    Creating a Data Share with CDP CLI or Creating a new Data Share in Cloudera Data Catalog Creating a Data Share

Upgrade from Cloudera Runtime 7.2.18.x with no Cloudera Iceberg REST Catalog configurations

Complete the following configurations to enable Cloudera Data Sharing when upgrading the Data Lake from 7.2.18.x with no REST Catalog or Cloudera Data Sharing configured in that version.

  1. Manual Installation of metering service

    Install and configure the metering server for Cloudera Data Sharing.

  2. HMS Rest Catalog configuration

    Configure the Hive Metastore service to serve as an Iceberg REST catalog. This allows your clients to use REST Catalog APIs to access the required metadata files.

  3. Table 3. Knox topologies
    AWS environment Azure environment

    Editing Knox topologies

    Edit your Knox topologies to define the lifetime of your Knox tokens (lifetime of your Data Shares) and the number of external users.

    Declaring Knox topologies

    Create the Knox topologies to define for Knox how to proxy requests from external users.

  4. Knox configuration

    Configure properties in Knox to set up administrator privileges that allow creation of tokens.

  5. Table 4. Create a Data Share in Cloudera
    AWS environment Azure environment
    Creating a Data Share with CDP CLI or Creating a new Data Share Creating a Data Share

Upgrade from Cloudera Runtime 7.2.18.x with Cloudera Iceberg REST Catalog / Cloudera Manager configuration

Complete the following configurations to enable Data Sharing when upgrading the Data Lake from 7.2.18.x with REST Catalog / Data Sharing configured in that version.

  1. Manual Installation of metering service

    Install and configure the metering server for Cloudera Data Sharing.

  2. HMS Rest Catalog configuration

    Configure the Hive Metastore service to serve as an Iceberg REST catalog. This allows your clients to use REST Catalog APIs to access the required metadata files.

  3. Table 5. Knox topologies
    AWS environment Azure environment

    Editing Knox topologies

    Edit your Knox topologies to define the lifetime of your Knox tokens (lifetime of your Data Shares) and the number of external users.

    Declaring Knox topologies

    Create the Knox topologies to define for Knox how to proxy requests from external users.

  4. Knox configuration

    Configure properties in Knox to set up administrator privileges that allow creation of tokens.

  5. Table 6. Create a Data Share in Cloudera
    AWS environment Azure environment
    Creating a Data Share with CDP CLI or Creating a new Data Share Creating a Data Share

Resizing Cloudera Data Lake from Light Duty Data Lake to Enterprise Data Lake

Complete the following configurations to enable Cloudera Data Sharing after resizing the Data Lake from Light Duty Data Lake to Enterprise Data Lake.

  1. HMS Rest Catalog configuration

    Configure the Hive Metastore service to serve as an Iceberg REST catalog. This allows your clients to use REST Catalog APIs to access the required metadata files.

  2. Table 7. Knox topologies
    AWS environment Azure environment

    Editing Knox topologies

    Edit your Knox topologies to define the lifetime of your Knox tokens (lifetime of your Data Shares) and the number of external users.

    Declaring Knox topologies

    Create the Knox topologies to define for Knox how to proxy requests from external users.

  3. Knox configuration

    Configure properties in Knox to set up administrator privileges that allow creation of tokens.

  4. Manually backing up and restoring Knox tables

    Manually back up and restore the Knox tables to preserve generated tokens, client credentials, and API keys.