To deploy your HDP instance, you need to prepare your deploy environment:
The clocks of all the nodes in your cluster must be able to synchronize with each other. If your system does not have access to the Internet, set up a master node as an NTP xserver. Use the following instructions to enable NTP for your cluster:
Configure NTP clients. Execute the following command on all the nodes in your cluster:
yum install ntp
Enable the service. Execute the following command on all the nodes in your cluster:
chkconfig ntpd on
Start the NTP. Execute the following command on all the nodes in your cluster:
/etc/init.d/ntpd start
For using existing NTP server in your environment. Configure firewall on local NTP server to enable UDP input traffic on port
123
. See the following sample rule:-A RH-Firewall-1-INPUT -s 192.168.1.0/24 -m state --state NEW -p udp --dport 123 -j ACCEPT
Restart iptables. Execute the following command on all the nodes in your cluster:
service iptables restart
Configure clients to use the local NTP server. Edit the
/etc/ntp.conf
and add the following line:server $LOCAL_SERVER_IP OR HOSTNAME
All hosts in your system must be configured for DNS and Reverse DNS.
Note | |
---|---|
If you are unable to configure DNS and Reverse DNS, you must edit the hosts file on every host in your cluster to contain each of your hosts. |
Use the following instructions to check DNS for all the host machines in your cluster:
Forward lookup checking.
For example, for domain
localdomain
that contains host with namehost01
and IP address192.168.0.10
, execute the following command:nslookup host01
You should see a message similar to the following:
Name: host01.localdomain Address: 192.168.0.10
Reverse lookup checking.
For example, for domain
localdomain
that contains host with namehost01
and IP address192.168.0.10
, execute the following command:nslookup 192.168.0.10
You should see a message similar to the following:
10.0.168.192.in-addr.arpa name = host01.localdomain.
If you do not receive valid responses (as shown above), you should set up DNS zone in your cluster or configure host files on each host of the cluster using one of the following options:
Option I: Configure hosts file on each node of the cluster.
For all nodes of cluster, add to the
/etc/hosts
file key-value pairs like the following:192.168.0.11 host01
Option II: Configuring DNS using BIND nameserver.
The following instructions, use the example values given below:
Example values: domain name: “localdomain” nameserver: “host01”/192.168.0.11 hosts: “host02”/192.168.0.12, “host02”/192.168.0.12
Install BIND packages:
yum install bind yum install bind-libs yum install bind-utils
Initiate service
chkconfig named on
Configure files. Add the following lines for the example values given above (ensure that you modify these for your environment) :
Edit the
/etc/resolv.conf
(for all nodes in cluster) and add the following lines:domain localdomain search localdomain nameserver 192.168.0.11
Edit the
/etc/named.conf
(for all nodes in cluster) and add the following lines:listen-on port 53 { any; };//by default it is opened only for localhost ... zone "localdomain" { type master; notify no; allow-query { any; }; file "named-forw.zone"; }; zone "0.168.192.in-addr.arpa" { type master; notify no; allow-query { any; }; file "named-rev.zone"; };
Edit the
named-forw.zone
as shown in the following sample forward zone configuration file:$TTL 3D @ SOA host01.localdomain.root.localdomain (201306030;3600;3600;3600;3600) NS host01 ; Nameserver Address localhost IN A 127.0.0.1 host01 IN A 192.168.0.11 host02 IN A 192.168.0.12 host03 IN A 192.168.0.13
Edit the
named-rev.zone
as shown in the following sample reverse zone configuration file:$TTL 3D @ SOA host01.localdomain.root.localdomain. (201306031;28800;2H;4W;1D); NS host01.localdomain.; Nameserver Address 11 IN PTR host01.localdomain. 12 IN PTR host02.localdomain. 13 IN PTR host03.localdomain.
Restart bind service.
/etc/init.d/named restart
Add rules to firewall.
iptables -A INPUT -p udp -m state --state NEW --dport 53 -j ACCEPT iptables -A INPUT -p tcp -m state --state NEW --dport 53 -j ACCEPT service iptables save service iptables restart
Alternatively, you can also allow traffic over DNS port (
53
) usingsystem-config-firewall
utility.
Security-Enhanced (SE) Linux feature should be disabled during installation process.
Check state of SELinux. On all the host machines, execute the following command:
getenforce
If the result is
permissive
ordisabled
, no further actions are required. Else, proceed to step 2.Disable SELinux either temporarily for each session or permanently.
Option I: Disable SELinux temporarily by executing the following command:
setenforce 0
Option II: Disable SELinux permanently in the
/etc/sysconfig/selinux
file by changing the value ofSELINUX
field topermissive
ordisabled
. Restart your system.
To have automatically deploy HDP in all your cluster hosts, you must set up password-less SSH connections between the master installation host and all other machines.
Note | |
---|---|
You can choose to install the HDP on each cluster host manually. In this case you do not need to setup SSH. |
Generate public and private SSH keys on the master install machine:
ssh-keygen
Copy the SSH Public Key
id_rsa.pub
to the root account on your target hosts..ssh/id_rsa .ssh/id_rsa.pub
Depending on your version of SSH, you may need to set permissions on your
.ssh
directory (to700
) and theauthorized_keys
file in that directory (to600
).chmod 700 ~/.ssh chmod 600 ~/. ssh /authorized_keys
Add the SSH Public Key to the
authorized_keys
file.cat id_rsa.pub >> authorized_keys
From the master install host machine, make sure you can connect to each host in the cluster using SSH.
ssh root@{$remote.target.host}
You may see this warning. This happens on your first connection and is normal.
Are you sure you want to continue connecting (yes/no)?
Retain a copy of the SSH Private Key on the machine from which you will run HDP Installer.