Cloudera Docs

Planning Capacity Using WXM

You can generate a capacity plan for your target environment using WXM if you have it deployed in your environment. To build a custom cloud environment that meets your capacity requirements you must analyze your existing CDH architecture, understand your business needs, and generate a capacity plan.

There might be differences in how you should size your impala compute clusters (either in Datahub or in CDW service) because the compute node sizes (CPU and RAM) are different than what you are currently using in CDH. If you are currently using a 20 node CDH cluster, it does not necessarily mean that you will need a 20 node Datahub cluster or a 20 node Impala virtual warehouse in CDW.

Using WXM Functionality to Generate a Capacity Plan

Benefits of using WXM

  • You can explore your cluster and analyze your workload before migrating the data. You can also identify Impala workloads that are good candidates for cloud migration.
  • You also have the provision to optimize your workload before migrating them to CDP Public Cloud. This mitigates the risk of run-away costs in the cloud due to suboptimal workloads.
  • You can generate a cloud-friendliness score for your workload to be migrated.
  • You have an option to auto-generate capacity for your target environment.
  • WXM in conjunction with the Replication manager automates the replication plan.

Prerequisites to Use WXM

Before you set up Cloudera Manager Telemetry Publisher service to send diagnostic data to Workload XM, you must ensure you have the correct versions of Cloudera Manager and CDH.

To use Workload XM with CDH clusters managed by Cloudera Manager, you must have the following versions:
  • For CDH 5.x clusters:
    • Cloudera Manager version 5.15.1 or later
    • CDH version 5.8 or later
  • For CDH 6.x clusters:
    • Cloudera Manager version 6.1 or later
    • CDH version 6.1 or later

After you have verified that you have the correct versions of Cloudera Manager and CDH, you must configure data redaction and your firewall.

For information on configuring firewall, see Configuring a Firewall for Workload XM.

For information on redacting data before sending to WXM, see Redaction Capabilities for Diagnostic Data

Steps to Auto-generate Capacity Plan

If you have a Cloudera workload manager deployed in your environment, follow the high level steps to generate the capacity plan and migrate the Impala workload to the cloud.

  1. On the Cloudera workload manager page, choose a cluster to analyze your data warehouse workloads. The Summary page for your workload view contains several graphs and tabs you can view to analyze. Using the Workloads View feature you can analyze workloads with much finer granularity. For example, you can analyze how queries that access a particular database or that use a specific resource pool are performing against SLAs. Or you can examine how all the queries are performing that a specific user sends to your cluster.
  2. On the Data Warehouse Workloads View page you can choose an Auto-generated Workload Views by clicking Define New and choosing Select recommended views from the drop-down menu. Review the Criteria that are used to create the workload views, select the one from the auto-generated workload views that aligns with your requirements.
  3. You can custom build the workload to be migrated by clicking on Define New and choosing Manually define view from the drop-down menu. You have the option to define a set of criteria that enables you to analyze a specific set of your workloads.
  4. If you chose to custom build, once the custom build workload is generated, you are returned to the Data Warehouse Workloads page, where your workload appears in the list. Use the search bar to search for your workload and click on the workload to view the workload details.
  5. The detail page for your workload view contains several graphs and tabs you can view to analyze. Review the workload and make sure that this is the workload you want to migrate to the cloud.
  6. After you are satisfied with the workload you want to burst, click the Burst to Cloud option and select View Performance Rating Details.
  7. Review the cloud performance rating details and make the call to proceed with the migration to cloud by clicking Start Burst to Cloud Wizard.
  8. Burst to Cloud wizard walks you through the steps to generate the capacity plan and to replicate the workload you selected to the destination on the cloud.