Configuring Apache Hadoop YARN SecurityPDF version

Configure YARN Security for Long-Running Applications

Long-running applications need additional configuration since the default settings only allow the hdfs user's delegation tokens a maximum lifetime of 7 days which is not always sufficient.

Long-running applications such as Spark Streaming jobs will need additional configuration since the default settings only allow the hdfs user's delegation tokens a maximum lifetime of 7 days which is not always sufficient.

You can work around this by configuring the ResourceManager as a proxy user for the corresponding HDFS NameNode so that the ResourceManager can request new tokens when the existing ones are past their maximum lifetime. YARN will then be able to continue performing localization and log-aggregation on behalf of the hdfs user.

  1. In Cloudera Manager, select the YARN service.
  2. Click the Configuration tab.
  3. Search for yarn-site.xml.
  4. Find the ResourceManager Advanced Configuration Snippet (Safety Valve) for yarn-site.xml.
  5. Click the plus icon and add the followoing:
    • Name: yarn.resourcemanager.proxy-user-privileges.enabled
    • Value: true
  6. Click Save Changes.
  7. Go back to the home page by clicking the Cloudera Manager logo.
  8. Select the HDFS service.
  9. Click the Configuration tab.
  10. Search for core-site.xml.
  11. Find the Cluster-wide Advances Configuration Snippet (Safety Valve) for core-site.xml.
  12. Click the plus icon and add the following:
    • Name: hadoop.proxyuser.yarn.hosts
    • Value: *
  13. Click the plus icon again and add the following:
    • Name: hadoop.proxyuser.yarn.groups
    • Value: *
  14. Click Save Changes.
  15. Click the Stale Service Restart icon that is next to the service to invoke the cluster restart wizard.
  16. Click Restart Stale Services.
  17. Select Re-deploy client configuration.
  18. Click Restart Now.