Install, Configure, and Deploy a HDP Cluster with CCP

You can use the Ambari Install wizard running in your browser to install, configure, and deploy your cluster.

1. Open Ambari Web using a web browser.
   1. Point your browser to http://<your.ambari.server>:8080, where <your.ambari.server> is the name of your ambari server host.
      For example, a default Ambari server host is located at http://c6401.ambari.apache.org:8080.
   2. Log in to the Ambari Server using the default user name/password: admin/admin.
      You can change these credentials later.
      For a new cluster, the Ambari install wizard displays a Welcome page from which you launch the Ambari Install wizard.
2. For an existing cluster, select Choose Services from the Actions/Add Service Wizard menu and skip to Step 7.
3. From the Ambari Welcome page, choose Launch Install Wizard.
4. In Name your cluster, type a name for the cluster you want to create, and then click NEXT.
   Avoid white spaces or special characters in the name.
5. Select the HDP stack you want to run.
6. Remove the base URL operating system information for any OSs you will not be running, and then click NEXT.
7. Use the Target Hosts text box to enter your list of host names, one per line.
   You can use ranges inside brackets to indicate larger sets of hosts. For example, for host-01.domain through host-10.domain use host-[01-10].domain
8. If you want to let Ambari automatically install the Ambari Agent on all your hosts using SSH,in the Host Registration Information section, select Provide your SSH Private Key to automatically register hosts and either use the Choose File button to find the private key file that matches the public key you installed earlier on all your hosts or cut and paste the key into the text box manually.
9. Click Register and Confirm to continue then click OK to confirm the host name pattern expressions.
10. Click NEXT to move to the Choose Services dialog box that lists the services that Ambari can install onto the cluster.
11. Choose the services to install onto the cluster, and then click Next.
    You can ignore any limited functionality warnings and click PROCEED ANYWAY.

    
    
    

    CCP requires the following services:
    • HDFS
    • Yarn + MapReduce2
    • Tez
    • Hive
    • HBase
    • ZooKeeper
    • Storm
    • Ambari Metrics
    • Kafka
    • Spark2
    • Zeppelin Notebook
    • Elasticsearch or Solr

      Elasticsearch can be installed either manually or by Ambari. We recommend installing Elasticsearch by Ambari.

    • Kibana (Can be installed either manually or by Ambari. Hortonworks recommends installing Kibana by Ambari.)
    • Metron
    • HDFS
    Ambari displays the Assign Masters window.
12. Use the Assign Masters window to assign the Master components to the appropriate hosts in your cluster.
    • All Metron components must reside on the same node.
    • The node containing the Metron components must have a Kafka broker.
    • The node containing the Metron components must have a ZooKeeper server.

    
    
    

    If Ambari detects any errors in your master component assignments, it will indicate the error in red.
    1. To change the host assignment for a service, select a host name from the drop-down menu for that service.
    2. To remove a ZooKeeper instance, click the green minus icon next to the host address you want to remove.
    3. When you are satisfied with the assignments, click Next.
       Ambari displays the Assign Slaves and Clients window.
13. Use the Assign Slaves and Clients window to assign cluster nodes (DataNodes, NodeManagers, and RegionServers) to run with worker processes such as Elasticsearch.
    The node containing the Metron host must have a Supervisor.
    1. Use all or none to select all of the hosts in the column or none of the hosts, respectively.

       If a host has an asterisk next to it, that host is also running one or more master components. Hover your mouse over the asterisk to see which master components are on that host.

    2. Select a minimum of one Elasticsearch data node.
       The data node cannot be on same host as the master.
    3. Select Supervisor for all data nodes.
       Supervisor is used by Storm to run the topologies, so selecting Supervisor on all of the nodes enables you to distribute the workload among the nodes.
    4. Fine-tune your selections by using the check boxes next to specific hosts.
       
    5. When you are satisfied with your assignments, click NEXT.
14. Assign a username and password for any services that require credentials.
    
15. No changes are necessary for the DATABASES, DIRECTORIES, and ACCOUNTS windows, so click NEXT to proceed to the next window.
    Ambari displays the SETTINGS window.
16. Click at the top right of the SETTINGS window to list the configuration properties you must set to proceed with the installation:
    
17. Click EDIT to display the appropriate window for each item in the list and provide the required information.
18. After you have completed configuring the required properties, click the BACK button to display the ALL CONFIGURATIONS window, then click METRON to display the Metron configurations.
    You might need to click the arrow button to see METRON:
    
19. If you are using Elasticsearch, you must provide the Elasticsearch Host name under the INDEX SETTINGS tab in the Elasticsearch Hosts field.
20. You must either set up your JDBC information or enable LDAP prior to deploying METRON:
    1. To set up your JDBC information, provide the required information under the REST tab:
       

       Metron REST port

       Use 8082.

       Metron JDBC URL

       jdbc:mysql://mysql_host:3306/metronrest

       Metron JDBC Driver

       com.mysql.jdbc.Driver

       You can choose between the following databases for the REST configuration.

       • PostgreSQL
       • MySQL
       • H2
       • Oracle

       Metron JDBC Username

       Metron REST user name

       Metron JDBC Password

       Metron REST password

       Metron JDBC client path

       <MYSQL_JAVA_CONNECTOR_PATH>/mysql-connector-java-5.1.41-bin.jar

    2. To switch to using LDAP to define access privileges see Switch to LDAP Access Privileges.
       Now, when you go to Swagger or the UIs, you should be able to view your assigned roles and permissions.
21. Browse through each service tab to determine if you want to provide any additional configuration information.
    Most of the fields in the Advanced tab are auto populated and should not be modified.
22. OPTIONAL: Configure the PCAP topology by setting your PCAP properties in the PCAP tab.
    
23. Check the assignments displayed by Ambari to ensure that everything is correct, and then click Deploy.
    If you need to make changes, use the BACK button to return to the appropriate screen.
    
    The progress of the install displays on the screen. Ambari installs, starts, and runs a simple test on each component. Overall status of the process displays in a progress bar at the top of the screen and host-by-host status displays in the main section. Do not refresh your browser during this process. Refreshing the browser might interrupt the progress indicators.
24. OPTIONAL: To see specific information on what tasks have been completed per host, click the link in the Message column for the appropriate host. In the Tasks pop-up, click the individual task to see the related log files. You can select filter conditions by using the Show drop-down list. To see a larger version of the log contents, click the Open icon or, to copy the contents to the clipboard, use the Copy icon.
25. When Successfully installed and started the services appears, click NEXT.