4.3. Add YARN/MR2 and Update Configurations

  1. Start the Ambari Server. On the Server host:

    ambari-server start
  2. IF using a local repository, update the URL for the local repository. On the Server host:

    curl -H "X-Requested-By: ambari" -X PUT -u admin:$ADMIN_PASSWD http://$AMBARI_SERVER_HOST:8080/api/v1/stacks2/HDP/versions/$HDP_VERSION/operatingSystems/$OS_TYPE/repositories/$HDP_STACK_ID -d '{"Repositories": {"base_url": "$LOCAL_REPO_URL", "verify_base_url": true}}'

    For example,

    curl -H "X-Requested-By: ambari" -X PUT -u admin:admin http://localhost:8080/api/v1/stacks2/HDP/versions/2.0.6/operatingSystems/centos6/repositories/HDP-2.0.6 -d '{"Repositories": {"base_url": "http://private-repo.xyz.com/HDP/centos6/2.x/updates/2.0.6.1", "verify_base_url": true}}'
    [Note]Note

    This update verifies the base URL, by default. Optionally, to update this URL without verification, set "verify_base_url" to false.

  3. Start each Ambari Agent. On all Agent hosts:

    ambari-agent start
  4. After the Server and all Agents are running, log into Ambari Web. Do a hard refresh on your browser to make sure you are displaying the updated GUI. Make sure all hosts are healthy and all services are in Stopped state.

  5. Add YARN and MapReduce2 services:

    1. If you are not already there, go to the Upgrade Folder.

    2. Execute the add-yarn-mr2 action:

      python UpgradeHelper_HDP2.py --hostname $HOSTNAME --user $USERNAME --password $PASSWORD --clustername $CLUSTERNAME add-yarn-mr2

      If desired, you can use the -n option to see the API calls as they are being made so that you can verify them.

  6. Update the respective configurations:

    1. If you are not already there, go to the Upgrade Folder.

    2. Execute the update-configs action:

      python UpgradeHelper_HDP2.py --hostname $HOSTNAME --user $USERNAME --password $PASSWORD --clustername $CLUSTERNAME update-configs
  7.  Update individual configuration settings as needed. On the Ambari Server, use /var/lib/ambari-server/resources/scripts/configs.sh to inspect and update the configuration properties.

    [Note]Note

    configs.sh creates temporary files. We recommended that you run configs.sh as root or as a user having write permission on the local folder.

    1. Get configuration details:

      configs.sh get $HOSTNAME $CLUSTERNAME $CONFIGURATION-TYPE
      
      For example:
      configs.sh get localhost myclustername global
    2. Evaluate each property value returned and modify as needed:

      configs.sh set $HOSTNAME $CLUSTERNAME $CONFIGURATION-TYPE "property name" "new value"
      
      For example:
      configs.sh set localhost myclustername global yarn_log_dir_prefix “/apps/logs/yarn”
    3. Remove properties that are not needed:

      configs.sh delete $HOSTNAME $CLUSTERNAME $CONFIGURATION-TYPE "property name"
      
      For example:
      configs.sh delete localhost myclustername global dfs.client-write-packet-size 
      Table II.6.1. Key Properties to Check
      Configuration Type Property Description
      global yarn_log_dir_prefix The location for the YARN logs
      global yarn_pid_dir_prefix The location for the YARN pid files
      global yarn_user The YARN user
      - - -
      yarn-site yarn.nodemanager.local-dirs The location for container logs
      yarn-site yarn.nodemanager.log-dirs The directories for localized files
      [Note]Note

      Make any necessary modifications before starting the services.

    4. Install the YARN and MapReduce2 services:

      1. If you are not already there, go to the Upgrade Folder.

      2. Execute the install-yarn-mr2 action:

        python UpgradeHelper_HDP2.py --hostname $HOSTNAME --user $USERNAME --password $PASSWORD --clustername $CLUSTERNAME install-yarn-mr2
        [Note]Note

        This is a two step process. You can use the Ambari Web GUI to monitor the progress. Both steps must be complete before you continue to the next step.


loading table of contents...