[Technical Preview] Setting up the Hive-Solr connector

Hive-Solr integration allows you to index Hive data into Solr and to query Solr data from Hive. It also enables Kerberos support, securing communication between Hive and Solr.

Prerequisites

  • Deployed Solr

  • Deployed Hive (make sure the hive command line tool works by running ​​the hive command as a 'hive' user.

Workflow

  1. Create a collection in Solr where you want to index data using Hive.
  2. Register the Hive-Solr connector in Hive.
  3. Create a solr-jaas.conf file, using a principal with read and write rights on both Hive and Solr (must be different from the Solr principal).

    For example:

    Client {
      com.sun.security.auth.module.Krb5LoginModule required
      useKeyTab=true
      keyTab="/data/solr-indexer.keytab"
      storeKey=true
      useTicketCache=true
      debug=true
      principal="solr-indexer@EXAMPLE.COM";
    };
    
    Copy the keytab and the solr-jaas.conf to all nodes.
  4. Create an external table in Hive. If Solr ZooKeeper TLS is enabled, include the lww.truststore, lww.truststore.password, lww.jaas.file (Kerberized clusters), and lww.zookeeper.secure table properties. See Enabling SSL for Solr ZooKeeper.
  5. Insert data to Solr through the Hive external table.
  6. Query Solr data from Hive.