Adding an HDP cluster (CCMv1)

You must register clusters with CDP before you can use those clusters with CDP services and components. To ensure optimum security, clusters within the customer environment are not accessible for communication. They have private IP addresses and cannot be accessed outside the firewall. However, to add your cluster to the CDP, a communication line needs to be established.

A reverseSSH tunnel solves the problem by establishing a tunnel from the cluster to CDP. You must download and install AutoSSH and the connectivity install scripts to establish a secure two-way communication channel. The AutoSSH ensures that the connectivity is stable. The connectivity scripts and their installation ensure safe connectivity and communication.

Before you begin

HDP clusters must be managed by Ambari. Clusters that are not managed by Ambari cannot be registered to CDP.
HDP clusters must include Knox.
HDP clusters must include Ranger policy settings
LDAP/AD must be set up and synced in Ambari
LDAP settings are automatically detected from the default topology setup in Knox. If the default topology does not have the LDAP setup, you will be asked to provide another topology name where you have configured the LDAP. If that topology has LDAP the setup continues. If the LDAP is not configured, you will receive an error message.
Kerberos must be enabled on the HDP cluster and the LDAP/AD must be set up in the Kerberos authentication so that the same set of LDAP/AD credentials can be used to access Ambari APIs as well as Beacon APIs
All clusters must meet the requirements identified in Prerequisites for adding classic clusters.

Steps

The process to register an HDP cluster using a reverseSSH tunnel is as follows:

Log in to CDP and navigate to the Management Console.
Click Classic Clusters in the left navigation panel.
Click Add Cluster.
Classic Cluster displays the Cluster Details dialog box.
If you are a first time user, under Step 1 in the Register Classic Cluster wizard, click GET STARTED. Classic Cluster then displays the Cluster Details dialog box.
If you are not a first time user, click the ADD CLUSTER button on the right side of the listing page.
Click HDP.
Provide the following connectivity information for your new cluster, then click CONNECT:
- IP Address
- Port
- Data center
note
Establishing connection might take more than twenty minutes if you are adding the cluster for the first time.

After Classic Cluster establishes connection, it will highlight Step 2 in the Register Classic Cluster wizard.
Start the download and installation process for the SSH connectivity files by clicking the Files button in Step 2 of the wizard.
Follow the instructions in the Setup Connectivity Client dialog box: Download and install the ccm-autosssh-client, and the ssh_tunnel_setup_files or cluster_connectivity_setup_files onto your new cluster.
If the installation of CCM autossh-client rpm fails with “No package autossh available”, refer to Troubleshooting: Install autossh package.
Copy the files to your Knox node or the Knox proxy host in the cluster.
Run the install.sh script.
./install.sh
Enter the following information as the install script prompts for it:
- Enter Ambari URL (http(s)://host:[port]):
- Enter Ambari Username:
- Enter Ambari Password:
If Knox is not installed on a proxy server, proceed to Step 13.
Classic Cluster sets up the topology for the Knox server and establishes the reverseSSH tunnel.
If Knox is installed on a proxy server, Classic Cluster displays the following message:
We discovered that your Knox is installed in HA mode. Please confirm if this node is your proxy node (yes/no):
Enter yes.
Classic Cluster generates XML content that you will need to add to your Knox hosts. Classic Cluster also displays three steps you must perform on all of your Knox hosts:
1. Copy the generated XML to /usr/hdp/current/knox-server/conf/topologies/cdp_default.xml.
  For example:
```
<?xml version ='1.0' encoding='utf8'?>
<topology> 
  <gateway>       
    <provider>  
      <role>authentication</role>     
      <name>ShiroProvider</name>   
      <enabled>true</enabled>   
      <param>   
        <name>sessionTimeout</name>  
        <value>30</value>
        .
        .
        .
</topology>>
              
```
2. Run chown knox:knox cdp_default.xml.
3. Check the Knox logs/deployment directory to verify that the cdp_default topology is deployed.
4. After you have completed Steps a through c on all of your Knox hosts, press Enter to continue.
  This sets up the topology for the Knox server or Knox proxy host and establishes the reverseSSH tunnel.
Classic Cluster starts checking the connectivity with the cluster. When the connectivity is successful, proceed to Step 3 in the wizard.
If the connection attempts fail or if there is an error in the connectivity, Classic Cluster displays troubleshooting information in Step 2 of the Registration wizard. Follow the troubleshooting information to fix the connectivity error, then click Test connection.

note
After you download the files, the ssh files download is disabled. At this point, you can regenerate the ssh files using the regenerate files option. This option comes in handy if you lose the files previously downloaded before you can set the cluster connectivity in the cluster.
Click Register in Step 3 of the Registration wizard.
In the Cluster Details dialog box, provide the username and password to access the cluster, then click CONNECT.
The user should have Admin access to the customer cluster services.
Finish registering the cluster by providing the following information.
- Cluster Location
- Data Center
- Tags (optional)
- Description (optional)
If LDAP is not set up on the default topology, the system will ask for the following additional information:
Enter knox topology name that contains LDAP setup:
Click ADD.