Configuring Cloud Data Access
Also available as:
PDF

Test access to ADLS Gen2

To test access to ADLS, SSH to a cluster node, switch to the hdfs user (by using ‘sudo su hdfs’), and run a few hadoop fs shell commands against your existing storage account.

ADLS access path syntax is:

abfs://file_system@account_name.dfs.core.windows.net/dir/file

Use abfss instead of abfs to connect with a secure socket layer connection.

For more information about syntax refer to Azure documentation.

For example, the following Hadoop FileSystem shell commands demonstrate access to “test container” in a storage account named “myaccount”:

hadoop fs -mkdir abfs://test-container@myaccount.dfs.core.windows.net/testdir
hadoop fs -put testfile abfs://test-container@myaccount.dfs.core.windows.net/testdir/testfile

FilesystemNotFound error

When hierarchical namespace is enabled, you may experience the java.io.FileNotFoundException with an error code of 404 mentioning FilesystemNotFound and the error message “The specified filesystem does not exist”. The common reason for this error is that you are trying to access a blob container created through the Azure Portal, but when hierarchical namespace is enabled, you must not create containers through Azure Portal.

To solve this issue, do one of the following:

  • Delete the blob container through Azure Portal, wait a few minutes, and then try accessing this container.

  • Or use a different container that is not created through Azure Portal.