Cloudera Docs

Checking the node count on your Cloud Service Provider's website

Learn about how to get real-time node count information on Amazon Web Services (AWS) and on Microsoft Azure.

To filter the nodes in the AWS Auto scaling groups (ASGs) or in Azure, obtain one of these identifiers:

  • Provisioner ID
  • Cluster name
  • Cluster ID

To obtain the provisioner ID (for example: liftie-xyz), in Cloudera Data Engineering, navigate to:

Administration > Service-Details > Logs and search for provisionerID: liftie.

Navigate to your Cloud Service Provider's website.
  1. In the AWS Management Console, log in as an IAM user, root user, or a similar role.
  2. At the top-right corner, select the region in which your cluster is located.
  3. In the AWS Management Console Search bar, type EC2, and from the displayed list, select EC2.

    The EC2 Dashboard for the selected region is displayed.

  4. In the EC2 Dashboard, in the left navigation pane, click Auto Scaling and select Auto Scaling Groups.

    On the Auto Scaling Groups page, you can see a list of all ASGs in the selected region.

  5. In the search bar, filter the ASGs using one of the following:
    • Cluster name
    • Cluster ID
    • provisioner ID
  6. To calculate the number of nodes in your Virtual Cluster (VC), add all the values displayed in the Instances column.
  7. Optional: If your cluster operates in multiple regions, switch to another region at the top-right corner of the AWS Management Console and repeat the steps described in this procedure.
  1. Login to Azure Portal.
  2. In the top-right corner, select the Directory and the Subscription where your VC is hosted.
  3. At the top of the portal, in the Search bar, type Virtual machine scale sets and select it from the displayed list.

    The Virtual machine scale sets page is displayed, where all Virtual Machine Scale Sets (VMSS) in your subscription are listed.

  4. To narrow down the list, perform one of these options:
    • If your cluster resources are grouped within a specific Resource group, use the Resource group filter to narrow down the list.
    • To search by the Cluster Name or Cluster ID tags associated with your Cluster or node pools, use the Add filter option. If you add a tag filter, the list of VMSS associated with the queried Cluster ID or Cluster Name is displayed.
    • To filter by the Provisioner ID, enter the Provisioner ID in the Search field on the VMSS page.
  5. To calculate the number of nodes in your VC, add up all the values displayed in the Instances column.
  6. Optional: If your VC spans through multiple regions, make sure that you select the correct Resource group, or use the global search to find resources in the specific region.