Managing Data Operating System
Also available as:
PDF
loading table of contents...

Run the YARN service on a Kerberized cluster

You must create a Kerberos principal and a keytab, and upload the latter to HDFS. In addition, you must specify the principal in your service definition.

  1. Create a Kerberos principal of the format <username>/<hostname>@<realm>.
  2. Create a keytab for the principal and upload it to HDFS.
    Note
    Note
    You must ensure that the user for which you are creating the principal has write permissions to the HDFS home directory.
    The following example shows the creation of a keytab for the principal user1/host1.example.com@EXAMPLE.COM, and the command to upload the keytab to HDFS:
    kadmin.local
    >addprinc user1/host1.example.com@EXAMPLE.COM
    ...
    >xst -k user1_host1.keytab user1/host1.example.com@EXAMPLE.COM
    ...
    >exit
    hadoop fs -put user1_host1.keytab hdfs:/user/user1/
    hadoop fs -chown user1 hdfs:/user/user1/
  3. Create a YARN service definition JSON using the REST API.
    Note
    Note
    Ensure that the service definition is unique in the cluster.
    The following example shows a YARN service definition added to a JSON file named yarnservice.json:
    {
      "name": "redis-service",
      "version": "1.0.0",
      "description": "redis example",
      "components" :
        [
          {
            "name": "redis",
            "number_of_containers": 1,
            "artifact": {
              "id": "library/redis",
              "type": "DOCKER"
            },
            "launch_command": "",
            "resource": {
              "cpus": 1,
              "memory": "256"
            },
            "configuration": {
              "env": {
                "YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE": "true"
              }
            }
          }
        ],
        "kerberos_principal": {
          "principal_name": "user1/host1.example.com@EXAMPLE.COM",
          "keytab": "hdfs:/user/user1/user1_host1.keytab"
        }
    }
  4. Submit the service definition as specified.
    YARN responds with the Application ID.
    The following example shows the curl command to submit the service definition:
    curl --negotiate -u : -X POST -H "Content-Type: application/json" http://<resource manager>:8088/app/v1/services -d @yarnservice.json
  5. Optional: Track the status of the service through the YARN UI or by using the REST APIs.
    The following example shows the curl command to read the status of the service:
    curl --negotiate -u : http://<resource manager>:8088/app/v1/services/redis-service | python -m json.tool