Configure data engineering Data Hub to use with COD
You can configure Hive in your Data Engineering Data Hub cluster to interact with Cloudera Operational Database (COD). You can use this integration to create and modify HBase tables using Hive. You can also READ and WRITE to existing HBase tables.
- Create a Data Engineering Datahub cluster in the same CDP environment on which your COD database is launched.
Download the HBase client configuration zip archive file from the
COD client connectivity page. For more information
see Client connectivity information. Then,
hbase-site.xmlfrom the zip file.
On each node in the Data Engineering Data Hub cluster that runs HiverServer2,
mkdir /etc/hbase/cod-confcommand, and copy the
hbase-site.xmlfile from Step 2 into
/etc/hbase/cod-conf.Do not copy all the files from client configuration zip archive. Only copy the
In Cloudera Manager for the Data Engineering Data Hub cluster, in the
Hive-on-Tez service, set the
HiveServer2 Environment Safety: HADOOP_CLASSPATHto
Go to Ranger service in the Data Lake.
Find HBase Ranger policy
cod_[***COD DATABASE NAME***]_hbase. For example, if your COD database name is
CODDB, you must modify the Ranger policy
Add the user
hiveto the existing policy. For example,
all - table, column-family, column.
- Wait for a few minutes because Ranger policy sync is not immediate to all resources.
- Find HBase Ranger policy
Interact with Hive using the
hivecommand from a node in the Data Engineering Data Hub.
$ hive hive> create table test(key int, value string) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties ("hbase.columns.mapping" = ":key,f1:val"); hive> insert into test values(1, 'a'); hive> select * from test;
Validate data is present in HBase as well. You can also run this from the Data
Engineering Data Hub node: the hbase executable is present to invoke. Otherwise,
set up the HBase client tarball from COD client connectivity.
$ HBASE_CONF_DIR=/etc/hbase/cod-conf hbase shell hbase> scan 'test'