Ingesting data into Solr

You can use an Apache NiFi data flow to ingest data into Apache Solr in the CDP Public Cloud following these steps.

Understand the use case
Learn how you can use Apache NiFi to move data to Apache Solr in your Data Discovery and Exploration (DDE) cluster in CDP Public Cloud.
Meet the prerequisites
Use this checklist to make sure that you meet all the requirements before you start building your data flow.
Create Solr target collection
Learn how you can create collections in Solr. Data is stored in collections, which are single indexes distributed across multiple servers.
Build the data flow
Learn how you can create an ingest data flow to move data from Apache Kafka to Apache Solr. This involves adding processors and other data flow elements to the NiFi canvas, configuring them, and connecting the elements to create the data flow.
Create controller services for your data flow
Learn how you can create and configure controller services to provide shared services for the processors in your Solr ingest data flow. You will use these controller services later when you configure your processors.
Configure the processor for your data source
You can set up a data flow to move data to Apache Solr from many different locations. This example assumes that you are transferring data from Kafka using the ConsumeKafkaRecord_2_0 processor.
Configure the processor for your data target
You can set up a data flow to move data to many locations. This example assumes that you are moving data to Apache Solr in a Data Discovery and Exploration cluster in CDP Public Cloud, using the PutSolrRecord processor.
Start the data flow
When you have built and configured your data flow, you can begin ingesting data into Solr. Learn how to start your Solr ingest data flow.
Verify data flow operation
Once you have configured and started your data flow, you can verify that you can write data to Solr. There are a number of ways you can check that data is running through the flow and it actually appears in Solr.