Data migration to Apache Hive

The Apache Sqoop client is a CLI-based tool bulk transfers of data between relational databases and HDFS or cloud object stores including Amazon S3 and Microsoft ADLS. The Sqoop client is available in CDP Private Cloud Base and in CDP Public Cloud.

In CDP Private Cloud Base, HDFS is typically the source of legacy system data that needs to undergo an extract, transform, and load (ETL) process. In CDP Public Cloud, S3 is typically the source of legacy data. You can also import data in delimited text (default) or SequenceFile format, and then convert data to ORC format recommended for Hive. Generally, for querying the data in Hive, ORC is the preferred format because of the performance enhancements ORC provides.