Configuring Impala access for Ozone using S3A

Learn about how to configure Impala access for Ozone using S3A.

  • You must import the CA certificate to run Accessing Ozone S3 using S3A FileSystem.
  • Perform the following steps to add the following configurations to S3 configurations for Ozone access:
    1. Sign in to Cloudera Manager.
    2. Navigate to Clusters > Impala > Configurations > Impala Catalog Server Advanced Configuration Snippet (Safety Valve) for core-site.xml.
    3. Add the following properties:
      fs.s3a.endpoint = <Ozone S3 endpoint URL>
      fs.s3a.bucket.probe = 0
      fs.s3a.change.detection.version.required = false
      fs.s3a.path.style.access = true
      fs.s3a.change.detection.mode = none
    4. Navigate to Clusters > Impala > Configurations > Impala Daemon Advanced Configuration Snippet (Safety Valve) for core-site.xml.
    5. Add the following properties:
      fs.s3a.endpoint = <Ozone S3 endpoint URL>
      fs.s3a.bucket.probe = 0
      fs.s3a.change.detection.version.required = false
      fs.s3a.path.style.access = true
      fs.s3a.change.detection.mode = none
  • Provide the required permissions in Ranger to the user running the queries.
    Example: Provide a user with all the permissions:
    • Assign the user with all permissions to the Database, table/udf, and URL resources in a HadoopSQL resource-based policy.
    • Assign the user with S3_VOLUME_POLICY in an Ozone policy.

    You can change the permissions based on your requirements.

  1. Create an Ozone bucket.
    ozone sh bucket create /s3v/[***BUCKET-NAME***]
    Example: The following example shows how you can create a bucket named s3impala:
    ozone sh bucket create /s3v/s3impala
  2. Sign in to the Impala shell and perform the following steps:
    1. Create a table on Ozone using S3A.
      > create external table mytable2(key string, value int) location 's3a://s3impala/mytable2';
    2. Add the data to the table.
      > insert into mytable2 values("cldr",1);
      > insert into mytable2 values("cldr-cdp",1);
    3. View the data added to the table.
      > select * from mytable2;
      Example,
      +----------+-------+
      | key      | value |
      +----------+-------+
      | cldr     | 1       |
      +----------+-------+
      +----------+-------+
      | cldr-cdp | 1       |
      +----------+-------+