Security
Also available as:
PDF
loading table of contents...

Creating Service Principals and Keytab Files for HDP

Each service in HDP must have its own principal. Because services do not login with a password to acquire their tickets, their principal's authentication credentials are stored in a keytab file, which is extracted from the Kerberos database and stored locally with the service principal.

First create the principal, using mandatory naming conventions. Then create the keytab file with that principal's information, and copy the file to the keytab directory on the appropriate service host.

  1. To create a service principal you will use the kadmin utility. This is a command-line driven utility into which you enter Kerberos commands to manipulate the central database. To start kadmin, enter:

    'kadmin $USER/admin@REALM'

    To create a service principal, enter the following:

    kadmin: addprinc -randkey $principal_name/$service-host-FQDN@$hadoop.realm

    You must have a principal with administrative permissions to use this command. The randkey is used to generate the password.

    The $principal_name part of the name must match the values in the following table.

    In the example each service principal's name has appended to it the fully qualified domain name of the host on which it is running. This is to provide a unique principal name for services that run on multiple hosts, like DataNodes and TaskTrackers. The addition of the hostname serves to distinguish, for example, a request from DataNode A from a request from DataNode B.

    This is important for two reasons:

    1. If the Kerberos credentials for one DataNode are compromised, it does not automatically lead to all DataNodes being compromised

    2. If multiple DataNodes have exactly the same principal and are simultaneously connecting to the NameNode, and if the Kerberos authenticator being sent happens to have same timestamp, then the authentication would be rejected as a replay request.

    Note: The NameNode, Secondary NameNode, and Oozie require two principals each.

    If you are configuring High Availability (HA) for a Quorom-based NameNode, you must also generate a principle (jn/$FQDN) and keytab (jn.service.keytab) for each JournalNode. JournalNode also requires the keytab for its HTTP service. If the JournalNode is deployed on the same host as a NameNode, the same keytab file (spnego.service.keytab) can be used for both. In addition, HA requires two NameNodes. Both the active and standby NameNodes require their own principle and keytab files. The service principles of the two NameNodes can share the same name, specified with the dfs.namenode.kerberos.principal property in hdfs-site.xml, but the NameNodes still have different fully qualified domain names.

    Table 2.8. Service Principals

    Service

    Component

    Mandatory Principal Name

    HDFS

    NameNode

    nn/$FQDN

    HDFS

    NameNode HTTP

    HTTP/$FQDN

    HDFS

    SecondaryNameNode

    nn/$FQDN

    HDFS

    SecondaryNameNode HTTP

    HTTP/$FQDN

    HDFS

    DataNode

    dn/$FQDN

    MR2

    History Server

    jhs/$FQDN

    MR2

    History Server HTTP

    HTTP/$FQDN

    YARN

    ResourceManager

    rm/$FQDN

    YARN

    NodeManager

    nm/$FQDN

    Oozie

    Oozie Server

    oozie/$FQDN

    Oozie

    Oozie HTTP

    HTTP/$FQDN

    Hive

    Hive Metastore

    HiveServer2

    hive/$FQDN

    Hive

    WebHCat

    HTTP/$FQDN

    HBase

    MasterServer

    hbase/$FQDN

    HBase

    RegionServer

    hbase/$FQDN

    Storm

    Nimbus server

    DRPC daemon

    nimbus/$FQDN **
    Storm

    Storm UI daemon

    Storm Logviewer daemon

    Nodes running process controller (such as Supervisor)

    storm/$FQDN **
    Kafka

    KafkaServer

    kafka/$FQDN
    ZeppelinZeppelin Serverzeppelin/$FQDN

    Hue

    Hue Interface

    hue/$FQDN

    ZooKeeper

    ZooKeeper

    zookeeper/$FQDN

    JournalNode Server*

    JournalNode

    jn/$FQDN

    Gateway

    Knox

    knox/$FQDN


    * Only required if you are setting up NameNode HA.

    ** For more information, see Configure Kerberos Authentication for Storm.

    For example: To create the principal for a DataNode service, issue this command:

    kadmin: addprinc -randkey dn/$datanode-host@$hadoop.realm
  2. Extract the related keytab file and place it in the keytab directory of the appropriate respective components. The default directory is /etc/krb5.keytab.

    kadmin: xst -k $keytab_file_name $principal_name/fully.qualified.domain.name

    You must use the mandatory names for the $keytab_file_name variable shown in the following table.

    Table 2.9. Service Keytab File Names

    Component

    Principal Name

    Mandatory Keytab File Name

    NameNode

    nn/$FQDN

    nn.service.keytab

    NameNode HTTP

    HTTP/$FQDN

    spnego.service.keytab

    SecondaryNameNode

    nn/$FQDN

    nn.service.keytab

    SecondaryNameNode HTTP

    HTTP/$FQDN

    spnego.service.keytab

    DataNode

    dn/$FQDN

    dn.service.keytab

    MR2 History Server

    jhs/$FQDN

    nm.service.keytab

    MR2 History Server HTTP

    HTTP/$FQDN

    spnego.service.keytab

    YARN

    rm/$FQDN

    rm.service.keytab

    YARN

    nm/$FQDN

    nm.service.keytab

    Oozie Server

    oozie/$FQDN

    oozie.service.keytab

    Oozie HTTP

    HTTP/$FQDN

    spnego.service.keytab

    Hive Metastore

    HiveServer2

    hive/$FQDN

    hive.service.keytab

    WebHCat

    HTTP/$FQDN

    spnego.service.keytab

    HBase Master Server

    hbase/$FQDN

    hbase.service.keytab

    HBase RegionServer

    hbase/$FQDN

    hbase.service.keytab

    Stormstorm/$FQDNstorm.service.keytab
    Kafkakafka/$FQDNkafka.service.keytab
    Zeppelin Serverzeppelin/$FQDNzeppelin.server.kerberos.keytab

    Hue

    hue/$FQDN

    hue.service.keytab

    ZooKeeper

    zookeeper/$FQDN

    zk.service.keytab

    Journal Server*

    jn/$FQDN

    jn.service.keytab

    Knox Gateway**

    knox/$FQDN

    knox.service.keytab


    * Only required if you are setting up NameNode HA.

    ** Only required if you are using a Knox Gateway.

    For example: To create the keytab files for the NameNode, issue these commands:

    kadmin: xst -k nn.service.keytab nn/$namenode-host kadmin: xst -k spnego.service.keytab HTTP/$namenode-host

    When you have created the keytab files, copy them to the keytab directory of the respective service hosts.

  3. Verify that the correct keytab files and principals are associated with the correct service using the klist command. For example, on the NameNode:

    klist –k -t /etc/security/nn.service.keytab

    Do this on each respective service in your cluster.