Cloudera Observability deployment architecture for Cloudera on cloud
Describes the components and architecture of a basic Cloudera Observability environment deployed in Cloudera on cloud.
- Cloudera Environment, which is a secure and governed cloud service platform. The Cloudera Observability component services run in the Cloudera Control Plane of the Cloudera Observability framework. Users access the Cloudera Observability web user interface from the web host server in this framework.
- Working Environment, which contains your Workload Clusters in your Workload environments, such as Production, Development, and Staging.
- Workload Cluster, which is one or more Cloudera clusters managed by Cloudera. Depending on your environment's Cloudera data service, Telemetry Publisher is associated with a cluster, virtual warehouse, or virtual cluster in a data service by either Cloudera Manager for a Data Hub service or by Databus WXM Client for Cloudera Data Engineering and Cloudera Data Warehouse services.
The following Cloudera Observability Architecture for Cloudera on cloud diagram shows the communication between Cloudera Observability and your workload clusters through either Telemetry Publisher or Databus WXM Client. Where, the Cloudera Observability service, including its main component services, run in the Cloudera Control Plane and the area on the right is your Working Environment that contains the clusters and services required to run your workload jobs and queries.
Cloudera Management Console (not shown) manages the clusters and services in each of your working environments.
Telemetry Publisher and Databus WXM Client collect and send diagnostic information about jobs and queries from your Workload Clusters to Cloudera Observability. To ensure that all data transfers are secure between your Workload Clusters and Cloudera Observability, Telemetry Publisher and Databus WXM Client communicate with Cloudera Observability’s endpoints using Transport Layer Security (TLS), as follows:
- When a job or query is completed, the Telemetry Publisher or Databus WXM Client connects to the Cloudera Observability service and asks to upload the workload diagnostic information. Once the request is authorized and verified, Cloudera Observability replies with a signed S3 URL that can then be used to upload the workload diagnostic information by Telemetry Publisher or Databus WXM Client.
- When the URL is received, Telemetry Publisher and Databus WXM Client perform a secure and direct protocol test using the Cloudera Observability S3 URL, before sending any diagnostic data.
-
Figure 3. Cloudera Data Warehouse Environment 
-
Figure 4. Cloudera Data Engineering Environment 
