Install and Configure Databases

Cloudera Manager Server, Cloudera Management Service, and the Hive Metastore data is stored in a database. Install and configure required databases following the instructions in Cloudera Manager and Managed Service Databases.

(CDH 5 only) On RHEL and CentOS 5, Install Python 2.6 or 2.7

Python 2.6 or 2.7 is required to run Hue. RHEL 5 and CentOS 5, in particular, require the EPEL repository package.

$ su -c 'rpm -Uvh http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm'
...
$ yum install python26

Install the Oracle Java Development Kit (JDK) on the Cloudera Manager Server host.

$ sudo yum install oracle-j2sdk1.7

$ sudo zypper install oracle-j2sdk1.7

$ sudo apt-get install oracle-j2sdk1.7

Install the Oracle JDK

Install Cloudera Manager Agent Packages

Install CDH and Managed Service Packages

For more information about manually installing CDH packages, see CDH 4 Installation Guide or CDH 5 Installation Guide.

1. Choose a repository strategy:
   • Standard Cloudera repositories. For this method, ensure you have added the required repository information to your systems.
   • Internally hosted repositories. You might use internal repositories for environments where hosts do not have access to the Internet. In such a case, ensure your environment is properly prepared. For more information, see Understanding Custom Installation Solutions.
2. Install packages:

   CDH Version	Procedure
   CDH 5	Red Hat Download and install the "1-click Install" package Download the CDH 5 "1-click Install" package. Click the entry in the table below that matches your Red Hat or CentOS system, choose Save File, and save the file to a directory to which you have write access (it can be your home directory). OS Version Click this Link Red Hat/CentOS/Oracle 5 Red Hat/CentOS/Oracle 5 link Red Hat/CentOS/Oracle 6 Red Hat/CentOS/Oracle 6 link Install the RPM: Red Hat/CentOS/Oracle 5 $ sudo yum --nogpgcheck localinstall cloudera-cdh-5-0.x86_64.rpm Red Hat/CentOS/Oracle 6 $ sudo yum --nogpgcheck localinstall cloudera-cdh-5-0.x86_64.rpm (Optionally) add a repository key: Red Hat/CentOS/Oracle 5 $ sudo rpm --import http://archive.cloudera.com/cdh5/redhat/5/x86_64/cdh/RPM-GPG-KEY-cloudera Red Hat/CentOS/Oracle 6 $ sudo rpm --import http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera Install the CDH packages: $ sudo yum clean all $ sudo yum install avro-tools crunch flume-ng hadoop-hdfs-fuse hadoop-hdfs-nfs3 hadoop-httpfs hbase-solr hive-hbase hive-webhcat hue-beeswax hue-hbase hue-impala hue-pig hue-plugins hue-rdbms hue-search hue-spark hue-sqoop hue-zookeeper impala impala-shell kite llama mahout oozie pig pig-udf-datafu search sentry solr-mapreduce spark-python sqoop sqoop2 whirr   Note: Installing these packages will also install all the other CDH packages that are needed for a full CDH 5 installation. SLES Download and install the "1-click Install" package. Download the CDH 5 "1-click Install" package. Click this link, choose Save File, and save it to a directory to which you have write access (it can be your home directory). Install the RPM: $ sudo rpm -i cloudera-cdh-5-0.x86_64.rpm Update your system package index by running: $ sudo zypper refresh (Optionally) add a repository key: $ sudo rpm --import http://archive.cloudera.com/cdh5/sles/11/x86_64/cdh/RPM-GPG-KEY-cloudera Install the CDH packages: $ sudo zypper clean --all $ sudo zypper install avro-tools crunch flume-ng hadoop-hdfs-fuse hadoop-hdfs-nfs3 hadoop-httpfs hbase-solr hive-hbase hive-webhcat hue-beeswax hue-hbase hue-impala hue-pig hue-plugins hue-rdbms hue-search hue-spark hue-sqoop hue-zookeeper impala impala-shell kite llama mahout oozie pig pig-udf-datafu search sentry solr-mapreduce spark-python sqoop sqoop2 whirr   Note: Installing these packages will also install all the other CDH packages that are needed for a full CDH 5 installation. Ubuntu and Debian Download and install the "1-click Install" package Download the CDH 5 "1-click Install" package: OS Version Click this Link Wheezy Wheezy link Precise Precise link Install the package. Do one of the following: Choose Open with in the download window to use the package manager. Choose Save File, save the package to a directory to which you have write access (it can be your home directory) and install it from the command line, for example: sudo dpkg -i cdh5-repository_1.0_all.deb (Optionally) add a repository key: Debian Wheezy $ curl -s http://archive.cloudera.com/cdh5/debian/wheezy/amd64/cdh/archive.key | sudo apt-key add - Ubuntu Precise $ curl -s http://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add - Install the CDH packages: $ sudo apt-get update $ sudo apt-get install avro-tools crunch flume-ng hadoop-hdfs-fuse hadoop-hdfs-nfs3 hadoop-httpfs hbase-solr hive-hbase hive-webhcat hue-beeswax hue-hbase hue-impala hue-pig hue-plugins hue-rdbms hue-search hue-spark hue-sqoop hue-zookeeper impala impala-shell kite llama mahout oozie pig pig-udf-datafu search sentry solr-mapreduce spark-python sqoop sqoop2 whirr   Note: Installing these packages will also install all the other CDH packages that are needed for a full CDH 5 installation.
   CDH 4, Impala, and Solr	Red Hat-compatible Click the entry in the table at CDH Download Information that matches your Red Hat or CentOS system. Navigate to the repo file (cloudera-cdh4.repo) for your system and save it in the /etc/yum.repos.d/ directory. Optionally add a repository key: Red Hat/CentOS/Oracle 5 $ sudo rpm --import http://archive.cloudera.com/cdh4/redhat/5/x86_64/cdh/RPM-GPG-KEY-cloudera Red Hat/CentOS 6 $ sudo rpm --import http://archive.cloudera.com/cdh4/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera Install packages on every host in your cluster: Install CDH 4 packages: $ sudo yum -y install bigtop-utils bigtop-jsvc bigtop-tomcat hadoop hadoop-hdfs hadoop-httpfs hadoop-mapreduce hadoop-yarn hadoop-client hadoop-0.20-mapreduce hue-plugins hbase hive oozie oozie-client pig zookeeper To install the hue-common package and all Hue applications on the Hue host, install the hue meta-package: $ sudo yum install hue (Requires CDH 4.2 or later) Install Impala Click the entry in the table at Cloudera Impala Version and Download Information that matches your Red Hat or CentOS system. Navigate to the repo file for your system and save it in the /etc/yum.repos.d/ directory. Install Impala and the Impala Shell on Impala machines: $ sudo yum -y install impala impala-shell (Requires CDH 4.3 or later) Install Search Click the entry in the table at Cloudera Search Version and Download Information that matches your Red Hat or CentOS system. Navigate to the repo file for your system and save it in the /etc/yum.repos.d/ directory. Install the Solr Server on machines where you want Cloudera Search. $ sudo yum -y install solr-server SLES Run the following command: $ sudo zypper addrepo -f http://archive.cloudera.com/cdh4/sles/11/x86_64/cdh/cloudera-cdh4.repo Update your system package index by running: $ sudo zypper refresh Optionally add a repository key: $ sudo rpm --import http://archive.cloudera.com/cdh4/sles/11/x86_64/cdh/RPM-GPG-KEY-cloudera Install packages on every host in your cluster: Install CDH 4 packages: $ sudo zypper install bigtop-utils bigtop-jsvc bigtop-tomcat hadoop hadoop-hdfs hadoop-httpfs hadoop-mapreduce hadoop-yarn hadoop-client hadoop-0.20-mapreduce hue-plugins hbase hive oozie oozie-client pig zookeeper To install the hue-common package and all Hue applications on the Hue host, install the hue meta-package: $ sudo zypper install hue (Requires CDH 4.2 or later) Install Impala Run the following command: $ sudo zypper addrepo -f http://archive.cloudera.com/impala/sles/11/x86_64/impala/cloudera-impala.repo Install Impala and the Impala Shell on Impala machines: $ sudo zypper install impala impala-shell (Requires CDH 4.3 or later) Install Search Run the following command: $ sudo zypper addrepo -f http://archive.cloudera.com/search/sles/11/x86_64/search/cloudera-search.repo Install the Solr Server on machines where you want Cloudera Search. $ sudo zypper install solr-server Ubuntu or Debian Click the entry in the table at CDH Version and Packaging Information that matches your Ubuntu or Debian system. Navigate to the list file (cloudera.list) for your system and save it in the /etc/apt/sources.list.d/ directory. For example, to install CDH 4 for 64-bit Ubuntu Lucid, your cloudera.list file should look like: deb [arch=amd64] http://archive.cloudera.com/cdh4/ubuntu/lucid/amd64/cdh lucid-cdh4 contrib deb-src http://archive.cloudera.com/cdh4/ubuntu/lucid/amd64/cdh lucid-cdh4 contrib Optionally add a repository key: Ubuntu Lucid $ curl -s http://archive.cloudera.com/cdh4/ubuntu/lucid/amd64/cdh/archive.key | sudo apt-key add - Ubuntu Precise $ curl -s http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add - Debian Squeeze $ curl -s http://archive.cloudera.com/cdh4/debian/squeeze/amd64/cdh/archive.key | sudo apt-key add - Install packages on every host in your cluster: Install CDH 4 packages: $ sudo apt-get install bigtop-utils bigtop-jsvc bigtop-tomcat hadoop hadoop-hdfs hadoop-httpfs hadoop-mapreduce hadoop-yarn hadoop-client hadoop-0.20-mapreduce hue-plugins hbase hive oozie oozie-client pig zookeeper To install the hue-common package and all Hue applications on the Hue host, install the hue meta-package: $ sudo apt-get install hue (Requires CDH 4.2 or later) Install Impala Click the entry in the table at Cloudera Impala Version and Download Information and that matches your Ubuntu or Debian system. Navigate to the list file for your system and save it in the /etc/apt/sources.list.d/ directory. Install Impala and the Impala Shell on Impala machines: $ sudo apt-get install impala impala-shell (Requires CDH 4.3 or later) Install Search Click the entry in the table at Cloudera Search Version and Download Information that matches your Ubuntu or Debian system. Install Solr Server on machines where you want Cloudera Search: $ sudo apt-get install solr-server

1. Wait several minutes for the Cloudera Manager Server to complete its startup. To observe the startup process you can perform tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log on the Cloudera Manager Server host. If the Cloudera Manager Server does not start, see Troubleshooting Installation and Upgrade Problems.
2. In a web browser, enter http://Server host:7180, where Server host is the fully-qualified domain name or IP address of the host where you installed the Cloudera Manager Server. The login screen for Cloudera Manager Admin Console displays.
3. Log into Cloudera Manager Admin Console. The default credentials are: Username: admin Password: admin. Cloudera Manager does not support changing the admin username for the installed account. You can change the password using Cloudera Manager after you run the installation wizard. While you cannot change the admin username, you can add a new user, assign administrative privileges to the new user, and then delete the default admin account.

1. In the first page of the Add Services wizard you choose the combination of services to install and whether to install Cloudera Navigator:
   • Click the radio button next to the combination of services to install:

     CDH 4	CDH 5
     Core Hadoop - HDFS, MapReduce, ZooKeeper, Oozie, Hive, and Hue Core with HBase Core with Impala All Services - HDFS, MapReduce, ZooKeeper, HBase, Impala, Oozie, Hive, Hue, and Sqoop Custom Services - Any combination of services.	Core Hadoop - HDFS, YARN (includes MapReduce 2), ZooKeeper, Oozie, Hive, Hue, and Sqoop Core with HBase Core with Impala Core with Search Core with Spark All Services - HDFS, YARN (includes MapReduce 2), ZooKeeper, Oozie, Hive, Hue, Sqoop, HBase, Impala, Solr, Spark, and Key-Value Store Indexer Custom Services - Any combination of services.

     As you select the services, keep the following in mind:
     • Some services depend on other services; for example, HBase requires HDFS and ZooKeeper. Cloudera Manager tracks dependencies and installs the correct combination of services.
     • In a CDH 4 cluster, the MapReduce service is the default MapReduce computation framework. Choose Custom Services to install YARN or use the Add Service functionality to add YARN after installation completes.
         Important: You can create a YARN service in a CDH 4 cluster, but it is not considered production ready.
     • In a CDH 5 cluster, the YARN service is the default MapReduce computation framework. Choose Custom Services to install MapReduce or use the Add Service functionality to add MapReduce after installation completes.
         Important: In CDH 5 the MapReduce service has been deprecated. However, the MapReduce service is fully supported for backward compatibility through the CDH 5 life cycle.
     • The Flume service can be added only after your cluster has been set up.
   • If you have chosen Data Hub Edition Trial or Cloudera Enterprise, optionally check the Include Cloudera Navigator checkbox to enable Cloudera Navigator. See the Cloudera Navigator Documentation.
   Click Continue. The Customize Role Assignments page displays.
2. Customize the assignment of role instances to hosts. The wizard evaluates the hardware configurations of the hosts to determine the best hosts for each role. The wizard assigns all worker roles to the same set of hosts to which the HDFS DataNode role is assigned. These assignments are typically acceptable, but you can reassign role instances to hosts of your choosing, if desired.

   Click a field below a role to display a dialog containing a pageable list of hosts. If you click a field containing multiple hosts, you can also select All Hosts to assign the role to all hosts or Custom to display the pageable hosts dialog.

   The following shortcuts for specifying hostname patterns are supported:

   • Range of hostnames (without the domain portion)

     Range Definition	Matching Hosts
     10.1.1.[1-4]	10.1.1.1, 10.1.1.2, 10.1.1.3, 10.1.1.4
     host[1-3].company.com	host1.company.com, host2.company.com, host3.company.com
     host[07-10].company.com	host07.company.com, host08.company.com, host09.company.com, host10.company.com

   • IP addresses
   • Rack name

   Click the View By Host button for an overview of the role assignment by hostname ranges.

3. When you are satisfied with the assignments, click Continue. The Database Setup page displays.
4. On the Database Setup page, configure settings for required databases:
   1. Enter the database host, database type, database name, username, and password for the database that you created when you set up the database.
   2. Click Test Connection to confirm that Cloudera Manager can communicate with the database using the information you have supplied. If the test succeeds in all cases, click Continue; otherwise check and correct the information you have provided for the database and then try the test again. (For some servers, if you are using the embedded database, you will see a message saying the database will be created at a later step in the installation process.) The Review Changes page displays.
5. Review the configuration changes to be applied. Confirm the settings entered for file system paths. The file paths required vary based on the services to be installed.
     Warning: DataNode data directories should not be placed on NAS devices.
   Click Continue. The wizard starts the services.
6. When all of the services are started, click Continue. You will see a success message indicating that your cluster has been successfully started.
7. Click Finish to proceed to the Home Page.