New Features and Changes in Cloudera Director

New Features and Changes in Cloudera Director 2

What's New in Cloudera Director 2.8.0

  • Simplified cluster templates. Cloudera Director now includes a simple setup procedure for launching CDH clusters, available through the UI, CLI, or API. When you create a cluster with the simple setup procedure, the installation process is streamlined because Cloudera Director selects default settings for you wherever possible. The results is fewer installation steps, and less risk of errors. You choose from a range of cluster types that are based on common CDH workloads, including basic, data engineering, analytic database, operational database, and EDH.
  • Export cluster configuration files. You can now export a cluster configuration file from any running CDH cluster, which can be used as-is or with changes for launching other clusters. Sensitive information, such as cloud provider credentials and database passwords, is redacted in the exported configuration file.
  • The command line client's standalone functionality has been deprecated. The standalone CLI commands will be removed in a future release (bootstrap, status, terminate, update, and validate), and the Cloudera Director client will be used only for command line interactions with a Cloudera Director server. When the validate command is removed, a new CLI command will be provided for validating a configuration file.
  • Support for RHEL 7.4 and CentOS 7.4 in Azure.
  • Support for AWS instance types P3 and G3.
  • Support for the AWS region EU (Paris), eu-west-3.
  • A number of issues have been fixed. See Issues Fixed in Cloudera Director 2.8.0 for details.

What's New in Cloudera Director 2.7.1

What's New in Cloudera Director 2.7.0

  • LDAP and Active Directory support for Cloudera Director users. You can now configure Cloudera Director to authenticate users against an LDAP or AD server and use groups on the server to control user roles in Cloudera Director.
  • Improved Java 8 support for clusters. Cloudera Director can assume direct control of JDK installation to make it easier to install a single JDK, including Java 8, on Cloudera Manager and cluster instances.
  • Instance level control of normalization. Cloudera Director previously allowed the disabling of instance normalization steps through global configuration. With Cloudera Director 2.7.0 and higher, instance template configuration can override the global configuration of those steps, so that normalization can be enabled and disabled differently for different instances.
  • AWS plugin improvements:
    • New EC2 instance types M5, H1, and C5 are now supported.
    • A configuration file can now supply EC2 instance user data in its raw form, and Cloudera Director will base64 encode it appropriately before submitting it to EC2, so that users do not need to do the encoding themselves.
  • A number of issues have been fixed. See Issues Fixed in Cloudera Director 2.7.0 for details.

What's New in Cloudera Director 2.6.1

What's New in Cloudera Director 2.6.0

  • Expanded support for TLS:
    • Automatic TLS for Cloudera Manager and CDH: Cloudera Director now provides the ability to automatically configure Cloudera Manager 5.13 or later and CDH 5.13 or later with TLS using a bespoke certificate authority based on the Cloudera Manager instance. See Automatic TLS (Auto-TLS).
    • TLS Support for Cloudera Manager: Cloudera Director is now able to communicate with Cloudera Manager 5.13 or later after it has been configured manually to communicate over TLS. See Manual TLS.
    • TLS Support for Cloudera Director clients: The Java and Python client libraries and the Cloudera Director client are now able to communicate with the Cloudera Director server after the server has been configured to communicate over TLS. See TLS for Cloudera Director Itself.
  • SSH Host Key Support: Cloudera Director can now retrieve SSH host keys from instances it allocates to provide extra security. See SSH Host Key Retrieval and Verification.
  • Support for including Cloudera Data Science Workbench when installing CDH on AWS with a Cloudera Director configuration file.
  • Starting with Cloudera Manager 5.13, Cloudera Director is able to delete Kerberos principals automatically created for a cluster when the cluster is terminated.
  • AWS EBS volumes with provisioned IOPS are now supported.
  • An instance template now supports specification of a rack ID for instances created from it. The rack ID can be used by Cloudera Manager and CDH to guide data distribution and increase durability.
  • New features and changes with Microsoft Azure:
    • Simplified Azure Environment and instance template configurations. Configurations from previous releases of Cloudera Director will continue to be supported.
    • Support for Azure Managed Disks. Azure Managed Disks are an improvement over Azure Storage Accounts. They simplify storage, provide better reliability for availability sets, and improve scalability. For more information on Managed Disks, see Azure Managed Disks Overview.
    • The instance prefix requirements have been relaxed to support prefixes of one and two characters.
    • Virtual machines can now be provisioned without an Availability Set.
    • Support for new disk sizes:
      • P/S40 (2048 GB)
      • P/S50 (4095 GB)
    • Some web UI menu items have been moved from the Advanced section to the Basic section.
    • Referencing Azure Marketplace Images inline in the Image field is now supported using this format: /publisher/<publisher>/offer/<offer>/sku/<sku>/version/<version>.
    • Deploying clusters with custom images is now supported when using Azure Managed Disks. See Deploying Clusters with Custom Images.
  • A beta of version 2 of the Cloudera Director plugin SPI is available, and the plugins that ship with Cloudera Director are updated to use it.
  • A number of issues have been fixed. See Issues Fixed in Cloudera Director 2.6.0 for details.

What's New in Cloudera Director 2.5.1

What's New in Cloudera Director 2.5.0

  • Automatic repair: Cloudera Director can now automatically repair instances that get terminated unexpectedly for clusters that have auto-repair enabled.
  • Significant improvement in resilience to unexpected instance terminations during bootstrap and grow operations. This improves support for AWS spot instances, as well as resilience of clusters using on-demand instances.
  • Cloudera Director can now configure and create external accounts in Cloudera Manager that include cloud provider login credentials in order to manage the use of additional cloud provider functionality, including object stores, such as Amazon S3 in AWS.
  • Cloudera Director now supports S3Guard, which guarantees a consistent view of data stored in Amazon S3.
  • Cloudera Director can now recognize changes made directly from the cloud provider console to provisioned instances such as instance type and AMI.
  • Cluster details page in the web UI now shows cluster history, displaying the most recent updates, including instances added, removed, or replaced in the cluster.
  • Logging improvements, including additional information about service and role configuration errors returned by the Cloudera Manager API.
  • A number of issues have been fixed. See Issues Fixed in Cloudera Director 2.5.0 for details.

What's New in Cloudera Director 2.4.1

What's New in Cloudera Director 2.4.0

  • Improved support for long-running clusters: Cloudera Director 2.4 will support upgrades, change of services, and configurations from Cloudera Manager 5.11.
  • Support for Spark 2 and Kudu.
  • Support for multiple bootstrap scripts. This feature allows you to combine functionality required during bootstrap while maintaining modularity. Using multiple scripts is beneficial when different parts of the customized bootstrap have different retry semantics, because retry upon failure is done only for failed scripts.
  • Configurable AWS instance tag names and values.
  • Support for passing of AWS user data to instances while they are being launched.
  • Sample code using the Java SDK to grow, shrink and repair the cluster. See Cloudera Director Java Client Samples.
  • For Azure clusters, support for Azure Data Lake Store via Cloudera Manager and CDH 5.11 deployments.
  • Support for the following Microsoft Azure Clouds and Regions:
  • Added RHEL 6.8 to the default OS list for Azure.
  • Improved error logging for Azure clusters.
  • A number of issues have been fixed. See Issues Fixed in Cloudera Director 2.4.0 for details.

What's New in Cloudera Director 2.3.0

  • Support for AWS Spot Blocks. Spot blocks can be used to ensure the duration of instances during bootstraps so that the bootstrap process is resilient to fluctuations in AWS Spot instance availability. For more information, see Specifying a Duration for Your Spot Instances in the Amazon AWS documentation.
  • More control of cluster configuration with the introduction of instance-level and deployment-level post-creation scripts, in addition to existing cluster-level post-creation scripts.
  • Updated look and feel of the Cloudera Director web user interface.
  • Support for Cloudera Enterprise 5.10 out of the box.
  • Support for RHEL 6.8 and RHEL 7.3 for Cloudera Director and CDH cluster instances.
  • Support for the following Microsoft Azure instance types:
    • Standard_DS15_v2
    • Standard_GS4
    • Standard_GS5
    Refer to the Cloudera Reference Architecture on Azure for supported configurations using these Storage Account types.
  • On Microsoft Azure, support for the CentOS 6.8 image published by Cloudera.
  • A number of issues have been fixed. See Issues Fixed in Cloudera Director 2.3.0 for details.

What's New in Cloudera Director 2.2.0

What's New in Cloudera Director 2.1.1

  • Cloudera Director now includes support for the following Microsoft Azure instance types:
    • Standard_DS12_v2
    • Standard_DS13_v2
    • Standard_DS14_v2
  • On Microsoft Azure, Cloudera Director now supports the RHEL 6.7 image published by Red Hat in partnership with Microsoft.
  • A number of issues have been fixed. See Issues Fixed in Cloudera Director 2.1.1 for details.

What's New in Cloudera Director 2.1.0

What's New in Cloudera Director 2.0.0

  • AWS Spot instances and Google Cloud Platform preemptible virtual machine instances are supported.
  • Setup of clusters that are highly available and authenticated through Kerberos is automated.
  • You can automate submission of jobs to clusters with dynamic creation and termination of clusters.
  • You can run custom scripts after cluster setup and before cluster termination.
  • The user interface is enhanced, with deeper insights into cluster health.
  • Reliability of cluster modifications is increased, including rollback in some failure scenarios.
  • RHEL 7.1 is supported.
  • A number of issues have been fixed. See Issues Fixed in Cloudera Director 2.0.0 for details.

New Features and Changes in Cloudera Director 1

The following sections describe what’s new and changed in each Cloudera Director 1 release.

What's New in Cloudera Director 1.5.2

What's New in Cloudera Director 1.5.1

What's New in Cloudera Director 1.5.0

  • Cloudera Director now supports multiple cloud providers through an open-source plugin interface, the Cloudera Director Service Provider Interface (Cloudera Director SPI).
  • Google Cloud Platform is now supported through an open-source implementation of the Cloudera Director SPI, the Cloudera Director Google Plugin.
  • Database servers set up by Cloudera Director can now be managed from the web UI.
  • You can now specify custom scripts to be run after cluster creation. Example scripts for enabling HDFS high availability and Kerberos are available on the Cloudera GitHub site.
  • The Cloudera Director database can now be encrypted. Encryption is enabled by default for new installations.
  • Cluster and Cloudera Manager configurations can now be set through the web UI.
  • A number of issues have been fixed. See Issues Fixed in Cloudera Director 1.5.0 for details.

What's New in Cloudera Director 1.1.3

  • A number of issues have been fixed. See Issues Fixed in Cloudera Director 1.1.3 for details.
  • The Cloudera Director disk preparation method now supports RHEL 6.6, which is supported by Cloudera Manager 5.4.
  • Custom endpoints for AWS Identity and Access Management (IAM) are now supported.
  • To ensure version compatibility between Cloudera Manager and CDH, Cloudera Director now defaults to installing the latest 5.3 version of Cloudera Manager and CDH, rather than installing the latest post-5.3 version.

What's New in Cloudera Director 1.1.2

What's New in Cloudera Director 1.1.1

What's New in Cloudera Director 1.1.0

  • Support for demand-based shrinking of clusters
  • Integration with Amazon RDS to enable end-to-end setup of clusters as well as related databases
  • Native client bindings for Cloudera Director API in Java and Python
  • Faster bootstrap of Cloudera Manager and clusters
  • Improved User Interface of Cloudera Director server including display of health of clusters and ability to customize cluster setups
  • Improvements to usability and documentation