Configuring Impala access for S3A

You must configure specific properties for client applications such as Impala to access the Ozone data store using S3A.

  • You must import the CA certificate to run Ozone S3 Gateway from the S3A filesystem.
  • You must configure the following Impala properties using the Cluster-wide Advanced Configuration Snippet (Safety Valve) for core-site.xml:
    fs.s3a.bucket.<<bucketname>>.access.key = <accesskey>
    fs.s3a.bucket.<<bucketname>>.secret.key = <secret>
    fs.s3a.endpoint = <Ozone S3 endpoint url>
    fs.s3a.bucket.probe = 0
    fs.s3a.change.detection.version.required = false
    fs.s3a.path.style.access = true
    fs.s3a.change.detection.mode = none
  • You must provide the required permissions in Ranger to the user running the queries. Consider the following example of providing a user with all permissions. You can change the permissions based on your requirements.
    • Assign the user with all permissions to the Database, table/udf, and URL resources in a HadoopSQL resource-based policy.
    • Assign the user with S3_VOLUME_POLICY in an Ozone policy.
  1. Create an Ozone bucket.
    The following example shows how you can create a bucket named s3impala:
    ozone sh bucket create /s3v/s3impala
  2. Log on to the Impala shell and perform the specified steps.
    1. Create a table on Ozone using S3A.
      bv-hoz-1.bv-hoz.abc.site:25003> create external table mytable2(key string, value int) location 's3a://s3impala/mytable1';
    2. Add data to the table.
      bv-hoz-1.bv-hoz.abc.site:25003> insert into mytable2 values("cldr",1);
      bv-hoz-1.bv-hoz.abc.site:25003> insert into mytable2 values("cldr-cdp",1);
      
    3. View the data added to the table.
      bv-hoz-1.bv-hoz.abc.site:25003> select * from mytable2;