Cloudera Director and Cloudera Manager Usage
Cloudera Director works with Cloudera Manager and the cloud service provider to provide centralized and programmatic administration of clusters in the cloud, including deployment, configuration, and maintenance of CDH clusters. With Cloudera Director, you can monitor and manage multiple Cloudera Manager and CDH deployments, across different cloud environments.
When you use Cloudera Director to deploy CDH, you can perform administrative tasks either in Cloudera Director or in Cloudera Manager. However, to avoid conflicts and inconsistencies, you must use the appropriate management tool to perform each task. In some cases, changes made directly in Cloudera Manager cannot be detected by Cloudera Director and can result in differences in cluster information stored in the applications.
Cloudera Director Usage
Use Cloudera Director when you want to perform the following types of tasks:
- Deploying Cloudera Manager and CDH clusters for prototyping.
- Deploying Cloudera Manager and CDH clusters when you have finalized the topology and configuration.
- Growing or shrinking a cluster. If you have made changes to the cluster using Cloudera Manager, update Cloudera Director with the changes and redeploy the cluster before you grow or shrink the cluster.
- Setting up clusters with Kerberos authentication or high availability.
Cloudera Manager Usage
- Testing and iterating on the topology and configuration of clusters.
Use Cloudera Director to create the cluster when you have finalized the topology and configuration.
- Setting up TLS and wire encryption.
When encryption is enabled, Cloudera Director cannot communicate with Cloudera Manager.
CDH Cluster Management Tasks
When you deploy CDH and Cloudera Manager through Cloudera Director, use Cloudera Director and Cloudera Manager to manage the clusters appropriately. In some cases, if you perform administrative tasks in Cloudera Manager, Cloudera Director and Cloudera Manager can become out of sync.
When Cloudera Director and Cloudera Manager are out of sync, Cloudera Director cannot grow or shrink the cluster or perform other updates to the cluster. You can still use Cloudera Director to deploy new Cloudera Managers and clusters. Cloudera Managers that are out of sync with Cloudera Director will continue to function independently of Cloudera Director.
The following table lists cloud administrative tasks and the application where you must perform them to avoid inconsistencies in Cloudera Director and Cloudera Manager:
| Task | Application | Description |
|---|---|---|
| Host configuration changes that must be propagated to additional hosts, such as changes to parcel or agent configuration or setting the instance type | Cloudera Director | If you make the changes in Cloudera Manager, the changes will affect only the current deployment. Cloudera Director cannot apply the changes to new hosts or roles that you add to the cluster. |
| Role assignment or migration | Cloudera Director | |
| Host decommission | Cloudera Director | |
| Addition of host to a cluster or addition of cluster to Cloudera Manager | Cloudera Director | |
| Cloudera Manager username and password change | Cloudera Manager and Cloudera Director | Change the username and password in Cloudera Manager. After you change the username and password in Cloudera Manager, you must update the information in Cloudera Director. If you do not update the information in Cloudera Director, Cloudera Director will not be able to monitor or modify the cluster. |
| Cluster setup | Cloudera Director | Cloudera Director cannot manage clusters that are set up directly in Cloudera Manager. |
| Enabling Kerberos authentication during cluster setup | Cloudera Director |
Kerberos setup is supported in Cloudera Director version 2.0 or higher. Use the configuration file to enable Kerberos. Do not use the Cloudera Director web UI. If you use Cloudera Director to deploy a cluster but use Cloudera Manager to enable Kerberos authentication, Cloudera Director and Cloudera Manager will become out of sync. |
| Enabling high availability during cluster setup. | Cloudera Director |
High availability is supported in Cloudera Director version 2.0 or higher. Use the configuration file to enable high availability. Do not use the Cloudera Director web UI. If you use Cloudera Director to deploy a cluster but use Cloudera Manager to enable high availability, Cloudera Director and Cloudera Manager will become out of sync. |
| Modifying a cluster in a highly available deployment | Cloudera Director |
If you enable high availability in Cloudera Manager, you can run modify operations only on clusters in instance groups that do not contain highly available master roles. |
| Upgrading a Cloudera Manager license. | Cloudera Manager |
Use Cloudera Manager to upgrade from Cloudera Express to Cloudera Enterprise. Cloudera Director will not display the state of the updated license, but will not prevent Cloudera Enterprise functionality. |
| Maintenance version upgrade to Cloudera Manager or CDH. | Cloudera Manager |
You must upgrade Cloudera Manager manually, and then use the upgraded Cloudera Manager to upgrade CDH. You can perform only maintenance upgrades. If you perform major or minor upgrades to Cloudera Manager or CDH, Cloudera Director will be out of sync with Cloudera Manager. |
| Changes to the configuration of a service or role | Cloudera Manager | Cloudera Director cannot detect service and role configuration changes made in Cloudera Manager. However, the changes will not affect Cloudera Director functionality. |
| Adding a service | Adding a service to a cluster is not supported by Cloudera Director. If you add a service to a cluster in Cloudera Manager, Cloudera Director will be out of sync with Cloudera Manager. Cloudera Director will not be able to modify the cluster. | |
| Removing a service | Removing a service from a cluster is not supported by Cloudera Director. If you remove a service from a cluster in Cloudera Manager, Cloudera
Director will be out of sync with Cloudera Manager.
You can stop a service instead of removing it from a cluster. You can also use the grow and shrink feature of Cloudera Director to create hosts that do not have that service’s roles. |
CDH Cluster Management Guidelines for Cloudera Director
When you use Cloudera Director to deploy Cloudera Manager and CDH, the cluster information is saved in the Cloudera Director database. If you make changes to the cluster using the cloud provider management console, the changes cannot be detected by Cloudera Director. As a result, Cloudera Director will have incorrect information about the configuration and state of the cluster.
Use the following guidelines when you manage CDH clusters deployed through Cloudera Director:
- You cannot update the AMI of an instance in Cloudera Director. When an AMI is scheduled for retirement, you must migrate the nodes in the instance that uses the AMI to a new instance with a new AMI before the AMI is retired. The clusters that use the AMI will not be affected by the AMI retirement. However, growing an instance group that uses the retired AMI will fail because the AMI is no longer available. Use Cloudera Director to migrate the cluster nodes to a new instance with a new AMI.
- Terminating an instance using the cloud provider management console will result in poor health of the hosts and services in Cloudera Director. If the health of an instance turns bad or the instance fails, you can migrate to a new instance. Use the Cloudera Director web UI to shrink and grow the worker nodes and migrate the master node to a new instance.
- Cloudera Director does not support resizing or changing the instance type of a deployed instance using the cloud provider management console. Instead, use Cloudera Director’s grow and shrink functionality to migrate to a new instance with the appropriate instance type.
For information about growing or shrinking a cluster, see Modifying the Number of Instances in an Existing Cluster.
For information about migrating HDFS master roles to a new instance, see Using Role Migration to Repair HDFS Master Role Instances.