7. Set up a trusted proxy server

Complete the following instructions to set up a trusted proxy server:

  1. Check Your Prerequisites.

    Select a mirror server host with the following characteristics:

    • This server runs on either CentOS/RHEL/Oracle Linux (5.x or 6.x), SLES 11, or Ubuntu 12, and has several GB of storage available.

    • The firewall allows all cluster nodes (the servers on which you want to install HDP) to access this server, and allows this server to access the Internet (at least those Internet servers for the repositories to be proxied)Install the Repos.

  2. Create a caching HTTP Proxy server on the selected host.

    It is beyond the scope of this document to show how to set up an HTTP PROXY server, given the many variations that may be required, depending on your data center’s network security policy. If you choose to use the Apache HTTPD server, it starts by installing httpd, using the instructions provided in the Apache HTTP Server Project page, and then adding the mod_proxy and mod_cache modules, as stated in the Apache mod_proxy module documentation. Please engage your network security specialists to set up the proxy server.

    After the proxy server is set up:

    • Activate this proxy server and configure its cache storage location.

    • Ensure that the firewall settings (if any) allow inbound HTTP access from your cluster nodes to your mirror server, and outbound access to the desired repo sites, including: public-repo-1.hortonworks.com.

      If you are using EC2, make sure that SELinux is disabled.

    • Depending on your cluster OS, configure the yum clients on all the nodes in your cluster.

      The following description is taken from the CentOS documentation. On each cluster node, add the following lines to the /etc/yum.conf file. (As an example, the settings below will enable yum to use the proxy server mycache.mydomain.com, connecting to port 3128, with the following credentials: yum-user/query.

    • # proxy server:port number
      proxy=http://mycache.mydomain.com:3128 
      # account details for secure yum proxy connections
      proxy_username=yum-user
      proxy_password=qwerty
    • Once all nodes have their /etc/yum.conf file updated with appropriate configuration info, you can proceed with the HDP installation just as though the nodes had direct access to the Internet repositories.

    • If this proxy configuration does not seem to work, try adding a / at the end of the proxy URL. For example:

      proxy=http://mycache.mydomain.com:3128/