Configuring the Hadoop Security Credential Provider Path Property
The URL to the provider must be set in the configuration property
hadoop.security.credential.provider.path
, either in the
core-site.xml
configuration file or on the command line:
Example: Setting via Configuration File
<property> <name>hadoop.security.credential.provider.path</name> <value>jceks://hdfs@nn1.example.com:9001/user/backup/s3.jceks</value> </property>
Because this property only supplies the path to the secrets file, the configuration option itself is no longer a sensitive item.
Example: Setting via Command Line
hadoop distcp \ -D hadoop.security.credential.provider.path=jceks://hdfs@nn1.example.com:9001/user/backup/s3.jceks \ hdfs://nn1.example.com:9001/user/backup/007020615 s3a://glacier1/ hadoop fs \ -D hadoop.security.credential.provider.path=jceks://hdfs@nn1.example.com:9001/user/backup/s3.jceks \ -ls s3a://glacier1/
Because the provider path is not itself a sensitive secret, there is no risk from placing its declaration on the command line.
Once the provider is set in the Hadoop configuration, hadoop commands work exactly as if the secrets were in an XML file. For example:
hadoop distcp hdfs://nn1.example.com:9001/user/backup/007020615 s3a://glacier1/ hadoop fs -ls s3a://glacier1/