Importing Data into Microsoft Azure Data Lake Store (Gen1 and Gen2) Using Sqoop
Microsoft Azure Data Lake Store (ADLS) is a cloud object store designed for use as a hyper-scale repository for big data analytic workloads. ADLS acts as a persistent storage layer for CDH clusters running on Azure.
There are two generations of ADLS, Gen1 and Gen2. You can use Apache Sqoop with both
generations of ADLS to efficiently transfer bulk data between these file systems and
structured datastores such as relational databases. For more information on ADLS Gen 1 and Gen
2, see:
You can use Sqoop to import data from any relational database that has a JDBC adaptor such as SQL Server, MySQL, and others, to the ADLS file system.
The major benefits of using Sqoop to move data are:
- It leverages RDBMS metadata to get the column data types
- It ensures fault-tolerant and type-safe data handling
- It enables parallel and efficient data movement
