To protect your credentials from unauthorized users, we recommend that you use the credential provider framework which securely stores your credentials and allows you to securely access them.

To provision the credentials:

% hadoop credential create fs.azure.account.key.youraccount.blob.core.windows.net -value 123
    -provider localjceks://file/home/lmccay/wasb.jceks

Next, configure the following configuration properties, either on the command line or in the core-site.xml configuration file:

<property>
  <name>hadoop.security.credential.provider.path</name>
  <value>localjceks://file/home/lmccay/wasb.jceks</value>
  <description>Path to interrogate for protected credentials.</description>
</property>

You may optionally add the provider path property to the distcp command line instead of adding a job-specific configuration to a generic core-site.xml. The options enclosed in square brackets illustrate this capability.

% hadoop distcp
    [-D hadoop.security.credential.provider.path=localjceks://file/home/lmccay/wasb.jceks]
    hdfs://hostname:9001/user/lmccay/007020615 wasb://yourcontainer@youraccount.blob.core.windows.net/testDir/

You may also protect the Azure credentials within an encrypted file.

Related Links

Credential Provider API