Cloudera Docs

Testing S3 with Hive

Add the following to S3 configurations for Ozone access. Do not add any S3 credentials. For more information, see Configuring Hive access for S3A.

Perform the following steps to add the following to S3 configurations for Ozone access:
  1. Sign in to Cloudera Manager.
  2. Navigate to Clusters > Hive > Configurations tab.
  3. Search for Hive Service Advanced Configuration Snippet (Safety Valve) for hive-site.xml.
  4. Add the following property: 
    <property>
<name>fs.s3a.endpoint</name>
<value>https://ccycloud-5.ela75-chf4.root.comops.site:9879</value>
</property>
<property>
<name>fs.s3a.bucket.probe</name>
<value>0</value>
</property>
<property>
<name>fs.s3a.change.detection.version.required</name>
<value>false</value>
</property>
<property>
<name>fs.s3a.path.style.access</name>
<value>true</value>
</property>
<property>
<name>fs.s3a.change.detection.mode</name>
<value>none</value>
</property>
  5. Navigate to Clusters > Hive_On_Tez > Configurations tab.
  6. Search for Hive Service Advanced Configuration Snippet (Safety Valve) for hive-site.xml.
  7. Add the following property: 
    <property>
<name>fs.s3a.endpoint</name>
<value>https://ccycloud-5.ela75-chf4.root.comops.site:9879</value>
</property>
<property>
<name>fs.s3a.bucket.probe</name>
<value>0</value>
</property>
<property>
<name>fs.s3a.change.detection.version.required</name>
<value>false</value>
</property>
<property>
<name>fs.s3a.path.style.access</name>
<value>true</value>
</property>
<property>
<name>fs.s3a.change.detection.mode</name>
<value>none</value>
</property>
  8. Update the following configurations:
    • Deploy client configuration
    • Restart the cluster
  9. Create, insert, and select Hive tables through beeline. 
    create external table mytable6(key string, value int) location 's3a://obs-bucket/warehouse/hive/external/mytable6';
insert into mytable6 values("cldr",1);

select * from mytable6;

+-------------------+---------------------+
| mytable6.key  | mytable6.value  |
+------------------+----------------------+
| cldr             |     1                        |
+------------------+----------------------+