Ingesting Data into HBase

You can use an Apache NiFi data flow to ingest data into Apache HBase in the CDP Public Cloud by following these steps.

Understand the use case
You can use Apache NiFi to move data from a range of locations into an Operational Database cluster running Apache HBase in CDP Public Cloud.
Meet the prerequisites
Use this checklist to make sure that you meet all the requirements before you start building your data flow.
Create the HBase target table
Before you can ingest data into Apache HBase in CDP Public Cloud, ensure that you have an HBase target table. These steps walk you through creating a simple table with one column family. Modify these instructions based on your ingest target table needs.
Add Ranger policies
Add Ranger policies to ensure that you have write access to your HBase tables.
Obtain HBase connection details
To enable an Apache NiFi data flow to communicate with HBase, you must obtain the HBase connection details by downloading several client configuration files. The NiFi processors require these files to parse the configuration values and use those values to communicate with HBase.
Build the data flow
From the Apache NiFi canvas, set up the elements of your data flow. This involves opening NiFi in CDP Public Cloud, adding processors to your NiFi canvas, and connecting the processors.
Configure the HBase client service
You can add Controller Services to provide shared services to be used by the processors in your data flow. In the case of data ingest into HBase, use a Controller Service to configure the HBase client service.
Configure the processor for your data source
You can set up a data flow to move data from many locations into Apache HBase. This example assumes that you are configuring ConsumeKafkaRecord_2_0. If you are moving data from a location other than Kafka, review Getting Started with Apache NiFi for information about how to build a data flow, and about other data consumption processor options.
Configure the processor for your data target
You can set up a data flow to move data into many locations. This example assumes that you are moving data into Apache HBase using PutHBaseRecord. If you are moving data into another location, review Getting Started with Apache NiFi for information about how to build a data flow, and about other data ingest processor options.
Start your data flow
Start your data flow to verify that you have created a working dataflow and to begin your data ingest process.
Verify your data flow
Once you have configured and started your data flow, you can verify that you are successfully ingesting data into HBase.