Setting up Azure Resources

This topic describes how to set up various resources required by Microsoft Azure:

Setting Up Resource Groups

Cloudera Director requires you to set up resource groups that house the following cluster resources::
  • Azure virtual machines (VMs)
  • Azure virtual network (VNet)
  • Azure network security group (NSG)
These resources typically have different lifecycles, so you may want to locate each in a separate resource group for convenience. For simplicity, you can locate them in the same resource group instead.

Creating a New Resource Group

To create a new resource group, perform the following steps:
  1. In the left pane, click New.
  2. Type resource group in the search box.
  3. Click Resource Group in the search result.
  4. Click Create.
  5. Type in a name for the resource group.
Repeat these steps to create multiple resource groups for Azure resources.

Setting Up a Network Security Group

This section explains how to create a new network security group (NSG) in Azure.

Creating a New Network Security Group

To create a new network security group:
  1. In the left pane, click New.
  2. Type Network security group in the search box.
  3. Click Network security group in the search result.
  4. Click Create.
  5. Type in a name for the network security group.
  6. Type in a name for new resource group or select an existing resource group.
  7. Click Create.
  8. Once created, see How to manage NSGs using the Azure portal in the Microsoft Azure documentation for instructions on creating the rules in the network security group.


Setting Up a Virtual Network (VNet) and Subnet

Cloudera Director requires a virtual network and subnet to implement its networking environment. The networking environment must be set up for forward and reverse hostname resolution. For a basic example for setting up forward and reverse hostname resolution, see Setting Up Dynamic DNS on Azure.

For an overview of virtual networks on Azure, see Virtual networks.

To set up a new virtual network and its subnets, follow the steps below. Skip these steps if you are using an existing virtual network and subnet.
  1. In the left pane, click New.
  2. Type Virtual Network in the search box.
  3. Click Virtual Network in the search result.
  4. Click Create.
  5. Type in a name for the virtual network and subnet
  6. Type in a name for new resource group, or select an existing resource group.
  7. Click Create.




Setting Up Availability Sets for Master Nodes and Worker Nodes

Azure uses availability sets to manage the availability of virtual machines. For best practices, in a CDH cluster, Cloudera recommends using one availability set for the master nodes and one availability set for the worker nodes. An availability set should not be shared by more than one CDH cluster.

Read this Azure document for an overview of availability sets on Azure.

To create an availability set:
  1. In the left pane, click New.
  2. Type Availability Set in the search box.
  3. Click Availability Set in the search result.
  4. Click Create.
  5. Type in a name for the availability set.
  6. Type in a name for new resource group or select an existing resource group.
  7. Increase the fault domain and update domain to as large a size as possible.
  8. Click Create.

After creating the availability set for master nodes, repeat the steps to create an availability set for worker nodes.




When creating the Availability Set, make sure to select No (Classic) for the Use managed disks option. No is the default option.