Configure Data Engineering Data Hub to use with COD

You can configure Hive in your Data Engineering Data Hub cluster to interact with CDP Operational Database (COD). You can use this integration to create and modify HBase tables using Hive. You can also READ and WRITE to existing HBase tables.

Ensure that you have a COD database instance and a Data Engineering template based Data Hub cluster in the same CDP environment.
  1. Create a Data Engineering Datahub cluster in the same CDP environment on which your COD database is launched.
  2. Download the HBase client configuration zip archive file from the COD client connectivity page. For more information see Client connectivity information. Then, extract hbase-site.xml from the zip file.
  3. On each node in the Data Engineering Data Hub cluster that runs HiveServer2, run the mkdir /etc/hbase/cod-conf command, and copy the hbase-site.xml file from Step 2 into /etc/hbase/cod-conf.
    Do not copy all the files from client configuration zip archive. Only copy the hbase-site.xml file.
  4. In Cloudera Manager for the Data Engineering Data Hub cluster, in the Hive-on-Tez service, set the HiveServer2 Environment Safety: HADOOP_CLASSPATH to /etc/hbase/cod-conf.
    1. Copy all the contents of the hbase-site.xml file inside the <configuration> tag.
    2. In the Hive-on-Tez service, go to Configuration > Hive Service Advanced Configuration Snippet (Safety Valve) for hive-site.xml and click View as XML.
    3. Append the copied contents into the XML editor.
    4. Click Save Configurations.
    5. Restart the Hive-on-Tez service.
  5. Go to Ranger service in the Data Lake.
    1. Find HBase Ranger policy cod_[***COD DATABASE NAME***]_hbase. For example, if your COD database name is CODDB, you must modify the Ranger policy cod_CODDB_hbase.
    2. Add the user hive to the existing policy. For example, all - table, column-family, column.
    3. Wait for a few minutes because Ranger policy sync is not immediate to all resources.
  6. Interact with Hive using the hive command from a node in the Data Engineering Data Hub.
    $ hive
    hive> create table test(key int, value string) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties ("hbase.columns.mapping" = ":key,f1:val");
    hive> insert into test values(1, 'a');
    hive> select * from test;
    
  7. Validate data is present in HBase as well. You can also run this from the Data Engineering Data Hub node: the hbase executable is present to invoke. Otherwise, set up the HBase client tarball from COD client connectivity.
    $ HBASE_CONF_DIR=/etc/hbase/cod-conf hbase shell
    hbase> scan 'test'