Cloudera Docs

Configuring Hive connector with HiveServer as the metadata source

Configure the Apache Hive connector in Octopai Client to extract metadata through HiveServer2 over an ODBC connection, using username and password or Kerberos authentication.

In HiveServer mode, Octopai Client connects to HiveServer2 through an Open Database Connectivity (ODBC) data source name (DSN). You configure the DSN once, choose either username and password or Kerberos authentication, and then add a matching connection in Octopai Client.

Ensure the following are available before you configure the connector:

  • A Hive ODBC driver that matches your Octopai Client architecture (32-bit or 64-bit). You can download the driver from the Hive ODBC driver download page.
  • For Kerberos authentication, MIT Kerberos for Windows. You can download the releveant distribution from MIT Kerberos Distribution. The default installation path is C:\Program Files\MIT\Kerberos\. Verify that C:\Program Files\MIT\Kerberos\bin\kinit.exe exists.
  • For Kerberos authentication, the Kerberos configuration file krb5.ini from your administrator, placed at C:\ProgramData\MIT\Kerberos5\krb5.ini, with your realm and key distribution center (KDC), for example:
    [libdefaults]
default_realm = YOUR.REALM
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true

[realms]
YOUR.REALM = {
  kdc = kdc.example.com
  admin_server = kdc.example.com
}

[domain_realm]
.example.com = YOUR.REALM
  • For Kerberos authentication, a keytab file with the extension .keytab for your service principal, stored securely on the Octopai Client machine, for example C:\Octopai\keytabs\hive.keytab.
  1. Install the Hive ODBC driver, choosing the 32-bit or 64-bit version that matches your environment.
  2. Open the ODBC Data Source Administrator and create a new system DSN.
    • Search for ODBC Data Source in the Start menu, and open the 32-bit or 64-bit version that matches the installed driver.
    • On the System DSN tab, click Add, select the Hive ODBC driver, and click Finish.
  3. Configure the basic DSN settings.
    • Data Source Name: a descriptive name, for example Hive_Prod.
    • Host: the hostname or IP address of the HiveServer2 service, for example hive.example.com.
    • Port: the HiveServer2 port. The default is 10000; confirm the value in your cluster configuration.
    • Database: a default database, for example default.
  4. Configure the authentication for the DSN.
    • For username and password, set the authentication mechanism to User Name and Password and enter the database credentials.
    • For Kerberos, set the authentication mechanism to Kerberos and enter the service name (hive), the realm, the fully qualified domain name (FQDN) of the HiveServer2 host, and the Kerberos configuration path C:\ProgramData\MIT\Kerberos5\krb5.ini.
  5. Optional: If the cluster requires secure connectivity, enable SSL or TLS and configure the truststore path and password according to the security policy of the cluster.
  6. Click Test to test the DSN, and then click OK to save it.
    For a Kerberos DSN, a valid Kerberos ticket might be required for the test to succeed.
  7. In Octopai Client, start the New Metadata Source wizard, select the Hive metadata source type, and on the Metadata Source Details step, under Metadata Source, select Hive Server.
  8. For Authentication method, select the method that matches the DSN, and enter the connection details.
    • For Username and Password, enter a Connection Name, the DSN Name that you created, the UserName, and the Password.
    • For Kerberos, enter a Connection Name, the DSN Name that you created, the Keytab Path (for example, C:\Octopai\keytabs\hive.keytab), and the Kerberos Principal (for example, hive@YOUR.REALM).
    Figure 1. Hive Server metadata source with Username and Password authentication

    New Metadata Source wizard with Hive Server selected, Username and Password authentication, and the Connection Name, DSN Name, UserName, and Password fields.
    Figure 2. Hive Server metadata source with Kerberos authentication

    New Metadata Source wizard with Hive Server selected, Kerberos authentication, and the Connection Name, DSN Name, Keytab Path, and Kerberos Principal fields.
  9. Select the Parse, Analyze, and Tag the Views as ETLs check box to process views as ETLs.
  10. Click Next, test the connection, and save it.
    For Kerberos, Octopai Client acquires a ticket by using kinit and the keytab, connects to Hive, and displays the connection status.

To verify the extracted metadata, navigate to the target folder (TGT) on the server where Octopai Client is installed. The default location is C:\Program Files (x86)\Octopai\Service\TGT. Locate the compressed file for the Hive connector, for example Hive_Metadata_Export.zip, and verify the quantity and quality of the extracted files.

If the connection or extraction fails, verify the following and try again:

  • Permissions on the Hive server and the ODBC connection.
  • The DSN configuration, including the correct hostname, port, and authentication.
  • For Kerberos, the krb5.ini configuration, the keytab, and the file permissions.
  • That the HiveServer2 service is running and reachable.

If the problem persists, send the log file with the connector name and number to Cloudera Support. The log is at C:\Program Files (x86)\Octopai\Service\log.