The Isilon Service
EMC Isilon is a storage service with a distributed file system that can used in place of HDFS to provide storage for CDH services.
The following versions of Cloudera and Isilon products are supported:
- Cloudera Manager 5.2
- CDH 5.1.3
- Isilon OneFS 7.2.0
Preliminary Steps on the Isilon Service
Before installing a Cloudera Manager cluster to use Isilon storage, perform the following steps on the Isilon OneFS system. For detailed information on setting up Isilon OneFS for Cloudera Manager, see the Isilon documentation at https://community.emc.com/docs/DOC-39522.
- Create an Isilon access zone with HDFS support.
- Create two directories that will be used by all CDH services:
- Create a tmp directory in the access zone.
- Create supergroup group and hdfs user.
- Create a tmp directory and set ownership to hdfs:supergroup, and permissions to 1777.
Example: cd hdfs_root_directory isi_run -z zone_id mkdir tmp isi_run -z zone_id chown hdfs:supergroup tmp isi_run -z zone_id chmod 1777 tmp
- Create a user directory in the access zone and set ownership to hdfs:supergroup, and permissions to 755
Example: cd hdfs_root_directory isi_run -z zone_id mkdir user isi_run -z zone_id chown hdfs:supergroup user isi_run -z zone_id chmod 755 user
- Create a tmp directory in the access zone.
- Create the service-specific users, groups, or directories for each CDH service you plan to use. Create the directories under the access zone you have created.
- ZooKeeper: nothing required.
- Create hbase group with hbase user.
- Create the root directory for HBase:
Example: hdfs_root_directory/hbase hbase:hbase 755
- YARN (MR2)
- Create mapred group with mapred user.
- Create history directory for YARN:
Example: hdfs_root_directory/user/history mapred:hadoop 777
- Create the remote application log directory for YARN:
Example: hdfs_root_directory/tmp/logs mapred:hadoop 775
- Create oozie group with oozie user.
- Create the user directory for Oozie:
Example: hdfs_root_directory/user/oozie oozie:oozie 775
- Create hive group with hive user.
- Create the user directory for Hive:
Example: hdfs_root_directory/user/hive hive:hive 775
- Create the warehouse directory for Hive:
Example: hdfs_root_directory/user/hive/warehouse hive:hive 1777
- Create solr group with solr user.
- Create the data directory for Solr:
Example: hdfs_root_directory/solr solr:solr 775
- Create sqoop group with sqoop2 user.
- Create the user directory for Sqoop:
Example: hdfs_root_directory/user/sqoop2 sqoop2:sqoop 775
- Create hue group with hue user.
- Create sample group with sample user.
- Create spark group with spark user.
- Create the user directory for Spark:
Example: hdfs_root_directory/user/spark spark:spark 751
- Create application history directory for Spark:
Example: hdfs_root_directory/user/spark/applicationHistory spark:spark 1777
Once the users, groups, and directories are created in Isilon OneFS, you are ready to install Cloudera Manager.
Installing Cloudera Manager with Isilon
- The simplest installation procedure, suitable for development or proof of concept, is Installation Path A, which uses embedded databases that are installed as part of the Cloudera Manager installation process.
- For production environments, Installation Path B - Manual Installation Using Cloudera Manager Packages describes configuring external databases for Cloudera Manager and CDH storage needs.
If you choose parcel installation on the Cluster Installation screen, the installation wizard will point to the latest parcels of CDH available. To specify CDH 5.1.3 as an option for parcel installation, click More Options and add the following repository URL to the list of parcel repositories: https://archive.cloudera.com/cdh5/parcels/5.1.3/. The screen will refresh after a few seconds, and you can choose CDH 5.1.3.
On the installation wizard's Cluster Setup page, choose Custom Services, and choose the services you want installed in the cluster. Be sure to choose Isilon among the selected services, do not select the HDFS or Impala services, and do not check Include Cloudera Navigator at the bottom of the Cluster Setup page. Also, on the Role Assignments page, be sure to specify the hosts that will serve as gateway roles for the Isilon service. You can add gateway roles to one, some, or all nodes in the cluster.
Installing a Secure Cluster with Isilon
- Create an unsecure Cloudera Manager cluster as described above in Installing Cloudera Manager with the Isilon Storage Service.
- Follow the Isilon documentation to enable Kerberos for your access zone: https://community.emc.com/docs/DOC-39522. This includes adding a Kerberos authentication provider to your Isilon access zone.
- Add the following proxy users in Isilon if your Cloudera Manager cluster includes the corresponding CDH services. The procedure for configuring proxy users is described in the Isilon
- proxy user hdfs for hdfs user.
- proxy user mapred for mapred user.
- proxy user hive for hive user.
- proxy user oozie for oozie user
- proxy user flume for flume user
- proxy user hue for hue user
- Follow the Cloudera Manager documentation for information on configuring a secure cluster with Kerberos: Configuring Authentication in Cloudera Manager.