YARN Registry
The YARN registry is a location into which statically and dynamically deployed applications can register service endpoints; client applications can look up these entries to determine the URLs and IPC ports with which to communicate with a service.
It is implemented as a zookeeper tree: services register themselves as
system services
, under the registry path /system
, or
user services
, which are registered under
/users/
where
USERNAME
is the name of the user
registering the service.USERNAME
As the purpose of the mechanism is to allow arbitrary clients to look up a service, the entries are always world readable. No secrets should be added to service entries.
In insecure mode, all registry paths are world readable and writeable: nothing may be trusted.
In a secure cluster, the registry is designed to work as follows:
Kerberos + SASL provides the identification and authentication.
/system
services can only be registered by designated system applications (YARN, HDFS, etc)/User-specific services can only be registered by the user deploying the application.
If a service is registered under a user's path, it may be trusted, and any published public information (such as HTTPS certifications) assumed to have been issued by the user.
All user registry entries should also be registered as world writeable with the list of system accounts defined in
hadoop.registry.system.accounts
; this is a list of ZK SASL-authenticated accounts to be given full access. This is needed to support system administration of the entries, especially automated deletion of old entries after application failures.The default list of system accounts are
yarn
,mapred
,hdfs
, andhadoop
; these are automatically associated with the Kerberos realm of the process interacting with the registry, to create the appropriatesasl:account@REALM ZK
entries.If applications are running from different realms, the configuration option
hadoop.registry.kerberos.realm
must be set to the desired realm, orhadoop.registry.system.accounts
configured with the full realms of the accounts.There is support for ZooKeeper
id:digest
authentication; this is to allow a user's short-lived YARN applications to register service endpoints without needing the Kerberos TGT. This needs active use by the launching application (which must explicitly create a user service node with an id:digest permission, or by settinghadoop.registry.user.accounts
, to the list of credentials to be permitted.System services must not use id:digest authentication —nor should they need to; any long-lived service already needs to have a kerberos keytab.
The per-user path for their user services,
/users/
, is created by the YARN resource manager when users launch services, if the RM is launched with the optionUSERNAME
hadoop.registry.rm.enabled
set totrue
.When
hadoop.registry.rm.enabled
is true, the RM will automatically purge application and container service records when the applications and containers terminate.Communication with ZK is over SASL, using the
java.security.auth.login.config
system property to configure the binding. The specific JAAS context to use can be set inhadoop.registry.jaas.context
if the default value,Client
, is not appropriate.
ZK Paths and Permissions:
All paths are world-readable; permissions are set up when the RM creates the root entry and user paths and hadoop.registry.secure=true.
Path | Role | Permissions |
---|---|---|
/registry | Base registry path | yarn, hdfs, mapred, hadoop : cdrwa |
/registry/system | System services | yarn, hdfs, mapred, hadoop : cdrwa |
/registry/users | Users | yarn, hdfs, mapred, hadoop : cdrwa |
/registry/users/ | The registry tree for the user
USER . |
|
Configuration options for secure registry access
Name | Recommended Value |
---|---|
hadoop.registry.secure | true |
hadoop.registry.rm.enabled | true |
hadoop.registry.system.accounts |
Grants system accounts write access to the root registry paths. A tighter version would be sasl:yarn@ which will only give the RM the right to manipulate these, or explicitly declare a realm, such as sasl:yarn@EXAMPLE |
hadoop.registry.kerberos.realm |
(empty) The Kerberos realm to use when converting the system accounts to full realms. If left empty, uses the realm of the user |
hadoop.registry.user.accounts | (empty) |
hadoop.registry.client.auth |
How to authenticate with ZK. Alternative (insecure) options: anonymous, digest. |
hadoop.registry.jaas.context |
The JAAS context to use for registry clients to authenticate with ZooKeeper. |