Deployment Architecture

Describes the components and architecture of a basic Workload XM environment.

A Workload XM environment comprises the following:
  • Cloudera Environment, which is a secure and governed cloud service platform. The Workload XM service and all its main component services run in the Workload XM framework. Users access the Workload XM web user interface from the web host server in this framework.
  • Working Environment, which contains your Workload Clusters in your Workload environments, such as Production, Development, and Staging.
  • Workload Cluster, which is one or more CDH, CDP, or HDP clusters managed by Cloudera Manager. Each cluster is associated with Telemetry Publisher through Cloudera Manager and runs your workload processes.

The below diagram shows the communication between Workload XM and your workload clusters through Cloudera Manager's Telemetry Publisher. Where, the Workload XM service, including its main component services, runs in the Cloudera Environment, and the area on the right, behind your firewall, is your Working Environment that contains the clusters and services required to run your workload processes.

Cloudera Manager (not shown) manages one or more clusters in each of your working environments. Telemetry Publisher is enabled and configured for Workload XM from each Cloudera Manager instance in your working environment.

For example, as shown in the diagram below, Customer A's Production environment contains two clusters that are both managed by one instance of Cloudera Manager, whereas the Development environment, which also contains two clusters, is managed by two instances of Cloudera Manager, one for each cluster. In this case, only one Telemetry Publisher service is enabled for the Production environment and two Telemetry Publisher services are enabled for the Development environment.

Telemetry Publisher collects and sends diagnostic information about job and query processes from your Workload Clusters to Workload XM and its services. To ensure that all data transfer is secure between your Workload Clusters and Workload XM, its services, and its S3 bucket, Telemetry Publisher communicates with Network Load Balancer, which is inside the Cloudera Environment's demilitarized zone, through the secure Transport Layer Security (TLS) protocol, as follows:

  1. When a job is completed, Telemetry Publisher requests access to Workload XM and its services through Network Load Balancer, which is inside the Cloudera Environment's demilitarized zone. When received and processed, a signed Workload XM S3 URL is returned to Telemetry Publisher.
  2. When the URL is received, Telemetry Publisher performs a secure and direct protocol test using the Workload XM S3 URL, before sending any diagnostic data.