Adding an HDP cluster
You must register clusters with CDP before you can use those clusters with CDP services and components. To ensure optimum security, clusters within the customer environment are not accessible for communication. They have private IP addresses and cannot be accessed outside the firewall. However, to add your cluster to the CDP, a communication line needs to be established.
A reverseSSH tunnel solves the problem by establishing a tunnel from the cluster to CDP. You must download and install AutoSSH and the connectivity install scripts to establish a secure two-way communication channel. The AutoSSH ensures that the connectivity is stable. The connectivity scripts and their installation ensure safe connectivity and communication.
Before you begin
- HDP clusters must be managed by Ambari. Clusters that are not managed by Ambari cannot be registered to CDP.
- HDP clusters must include Knox.
- HDP clusters must include Ranger policy settings
- LDAP/AD must be set up and synced in Ambari
LDAP settings are automatically detected from the default topology setup in Knox. If the default topology does not have the LDAP setup, you will be asked to provide another topology name where you have configured the LDAP. If that topology has LDAP the setup continues. If the LDAP is not configured, you will receive an error message.
- Kerberos must be enabled on the HDP cluster and the LDAP/AD must be set up in the Kerberos authentication so that the same set of LDAP/AD credentials can be used to access Ambari APIs as well as Beacon APIs
- All clusters must meet the requirements identified in Prerequisites for adding classic clusters.
The process to register an HDP cluster using a reverseSSH tunnel is as follows:
- Log in to CDP and navigate to the Management Console.
- Click Classic Clusters in the left navigation panel.
- Click Add Cluster.
Classic Cluster displays the Cluster Details dialog box.
- If you are a first time user, under Step 1 in the Register Classic
Cluster wizard, click GET STARTED. Classic Cluster then displays the
Cluster Details dialog box.
If you are not a first time user, click the ADD CLUSTER button on the right side of the listing page.
- Click HDP.
- Provide the following connectivity information for your new cluster, then click
- IP Address
- Data center
After Classic Cluster establishes connection, it will highlight Step 2 in the Register Classic Cluster wizard.
- Start the download and installation process for the SSH connectivity files by clicking the Files button in Step 2 of the wizard.
- Follow the instructions in the Setup Connectivity Client dialog
box, downloading and installing the
ccm-autosssh-client, and the
cluster_connectivity_setup_filesonto your new cluster.
- Copy the files to your Knox node or the Knox proxy host in the cluster.
- Run the
- Enter the following information as the install script prompts for it:
- Enter Ambari URL (http(s)://host:[port]):
- Enter Ambari Username:
- Enter Ambari Password:
- If Knox is not installed on a proxy server, proceed to Step 13.
Classic Cluster sets up the topology for the Knox server and establishes the reverseSSH tunnel.
- If Knox is installed on a proxy server, Classic Cluster displays the following
We discovered that your Knox is installed in HA mode. Please confirm if this node is your proxy node (yes/no):
Classic Cluster generates XML content that you will need to add to your Knox hosts. Classic Cluster also displays three steps you must perform on all of your Knox hosts:
- Copy the generated XML to
<?xml version ='1.0' encoding='utf8'?> <topology> <gateway> <provider> <role>authentication</role> <name>ShiroProvider</name> <enabled>true</enabled> <param> <name>sessionTimeout</name> <value>30</value> . . . </topology>>
chown knox:knox cdp_default.xml.
- Check the Knox
logs/deploymentdirectory to verify that the
cdp_defaulttopology is deployed.
- After you have completed Steps a through c on all of your Knox hosts, press
Enter to continue.
This sets up the topology for the Knox server or Knox proxy host and establishes the reverseSSH tunnel.
- Copy the generated XML to
- Classic Cluster starts checking the connectivity with the cluster. When the connectivity
is successful, proceed to Step 3 in the wizard.
If the connection attempts fail or if there is an error in the connectivity, Classic Cluster displays troubleshooting information in Step 2 of the Registration wizard. Follow the troubleshooting information to fix the connectivity error, then click Test connection.
- Click Register in Step 3 of the Registration wizard.
- In the Cluster Details dialog box, provide the username and
password to access the cluster, then click CONNECT.
The user should have Admin access to the customer cluster services.
- Finish registering the cluster by providing the following information.
- Cluster Location
- Data Center
- Tags (optional)
- Description (optional)
If LDAP is not set up on the default topology, the system will ask for the following additional information:
Enter knox topology name that contains LDAP setup:
- Click ADD.