ADLS Gen 2 is where all future development of Azure Big Data storage is taking place, in the cloud server as well as the client connector. In contrast, ADLS Gen 1 must be considered a maintenance only data store, which is not being rolled out across more Azure regions.
|Feature||Azure Storage||ADLS Gen 1||ADLS Gen 2|
|Broadly supported in applications||Yes||No||Yes in non-hierarchical; growing in hierarchical|
|Scales to many petabytes||No||Yes||Yes|
|Available in all Azure region||Yes||No||Yes|
|Interoperability||ADLS Gen 2 in non-hierarchical||none||Azure Storage|
|Directories with permissions||No||Yes||Yes|
Differences between ADLS Gen 1 and ADLS Gen 2
ADLS Gen 2 should be considered as a reimplementation of the ADLS Gen 1 features, but integrated with the original Azure Storage.
Features of ADLS Gen 2
- Supports the Hadoop FileSystem API, with directories, file and directory permissions, and other key features.
- Reads and writes data stores in the original Azure Storage (which has previously used the wasb:// URL)
- No integration with ADLS Gen1. The adls:// connector must be used there.
- Available in all Azure regions.
- If an ADLS Gen 2 account is created "without hierarchical namespaces" then the wasb:// connector can read/write data stored in ADLS Gen 2.
- Capable of storing many Petabytes of data.