HDP Reference Guide
Also available as:
PDF

Option I - Mirror server has no access to the Internet

Complete the following instructions to set up a mirror server that has no access to the Internet:

  1. Check Your Prerequisites.

    Select a mirror server host with the following characteristics:

    • This server runs on either CentOS (v5.x, v6.x), RHEL (v5.x, v6.x), Oracle Linux(v5.x, v6.x), SLES 11, or Ubuntu 12, and has several GB of storage available.

    • This server and the cluster nodes are all running the same OS.

      [Note]Note

      To support repository mirroring for heterogeneous clusters requires a more complex procedure than the one documented here.

    • The firewall lets all cluster nodes (the servers on which you want to install HDP) access this serve

  2. Install the Repos.

    1. Use a workstation with access to the Internet and download the tarball image of the appropriate Hortonworks yum repository.

      Table 3.2. Hortonworks Yum Repositories

      Cluster OSHDP Repository Tarballs
      RHEL/CentOs/Oracle Linux 6.xwget http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.2.0/HDP-2.3.2.0-centos6-rpm.tar.gz wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6/HDP-UTILS-1.1.0.20-centos6.tar.gz
      RHEL/CentOs/Oracle Linux 7.xwget http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.3.2.0/HDP-2.3.2.0-centos7-rpm.tar.gz wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos7/HDP-UTILS-1.1.0.20-centos7.tar.gz
      SLES 11 SP3/SP4wget http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/updates/2.3.2.0/HDP-2.3.2.0-suse11sp3-rpm.tar.gz wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/suse11sp3/HDP-UTILS-1.1.0.20-suse11sp3.tar.gz
      Ubuntu 12.04wget http://public-repo-1.hortonworks.com/HDP/ubuntu12/2.x/updates/2.3.2.0/HDP-2.3.2.0-ubuntu12-deb.tar.gz wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/ubuntu12/HDP-UTILS-1.1.0.20-ubuntu12.tar.gz
      Ubuntu 14wget http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.3.2.0/HDP-2.3.2.0-ubuntu14-deb.tar.gz wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/ubuntu14/HDP-UTILS-1.1.0.20-ubuntu14.tar.gz
      Debian 6wget http://public-repo-1.hortonworks.com/HDP/debian6/2.x/updates/2.3.2.0/HDP-2.3.2.0-debian6-deb.tar.gz wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/debian6/HDP-UTILS-1.1.0.20-debian6.tar.gz
      Debian 7wget http://public-repo-1.hortonworks.com/HDP/debian7/2.x/updates/2.3.2.0/HDP-2.3.2.0-debian7-deb.tar.gz wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/debian7/HDP-UTILS-1.1.0.20-debian7.tar.gz

    2. Create an HTTP server.

      • On the mirror server, install an HTTP server (such as Apache httpd) using the instructions provided here.

      • Activate this web server.

      • Ensure that the firewall settings (if any) allow inbound HTTP access from your cluster nodes to your mirror server.

        [Note]Note

        If you are using EC2, make sure that SELinux is disabled.

        If you are using EC2, make sure that SELinux is disabled.

    3. On your mirror server, create a directory for your web server.

      • For example, from a shell window, type:

        • ForRHEL/CentOS/Oracle:

          mkdir –p /var/www/html/hdp/
        • For SLES:

          mkdir –p /srv/www/htdocs/rpms
        • For Ubuntu:

          mkdir –p /var/www/html/hdp/
      • If you are using a symlink, enable thefollowsymlinks on your web server.

    4. Copy the HDP Repository Tarball to the directory created in step 3, and untar it.

    5. Verify the configuration.

      • The configuration is successful, if you can access the above directory through your web browser.

        To test this out, browse to the following location: http://$yourwebserver/hdp/$os/HDP-2.3.2.0/.

        You should see directory listing for all the HDP components along with the RPMs at: $os/HDP-2.3.2.0.

        [Note]Note

        If you are installing a 2.x.0 release, use: http://$yourwebserver /hdp/ $os /2.x/GA

        If you are installing a 2.x.x release, use: http://$yourwebserver /hdp/ $os /2.x/updates

        where

        • $os can be centos5, centos6, suse11, or ubnuntu12. Use the following options table for $osparameter:

          Table 3.3. HDP Component Options

          Operating SystemValue
          CentOS 5centos5
          RHEL 5 
          Oracle Linux 5 
          CentOS 6centos6
          RHEL 6 
          Oracle Linux 
          SLES 11suse11
          Ubuntu 12ubuntu12


    6. Configure the yum clients on all the nodes in your cluster.

      • Fetch the yum configuration file from your mirror server.

        http://$yourwebserver /hdp/$os/2.x/updates/2.3.2.0/hdp.repo
      • Store the hdp.repo file to a temporary location.

      • Edit the hdp.repo file changing the value of the base url property to point to your local repositories based on your cluster OS.

        where

        • $yourwebserver is the FQDN of your local mirror server.

        • $os can be centos5, centos6, suse11, or ubuntu12. Use the following options table for $os parameter:

          Table 3.4. Yum Client Options

          Operating SystemValue
          CentOS 5centos5
          RHEL 5 
          Oracle Linux 5 
          CentOS 6centos6
          RHEL 6 
          Oracle Linux 6 
          Ubuntu 12ubuntu12


        • Use scp or pdsh to copy the client yum configuration file to /etc/yum.repos.d/ directory on every node in the cluster.

      • [Conditional]: If you have multiple repositories configured in your environment, deploy the following plugin on all the nodes in your cluster.

        • Install the plugin.

          • For RHEL and CentOs v5.x

            yum install yum-priorities
          • For RHEL and CentOs v6.x

            yum install yum-plugin-priorities
        • Edit the /etc/yum/pluginconf.d/priorities.conf file to add the following:

          [main]
          enabled=1
          gpgcheck=0