Deployment architecture for CDP Private Cloud
Describes the components and architecture of a basic Cloudera Observability environment deployed in CDP Private Cloud.
- Cloudera Environment, which is a secure and governed cloud service platform. The Cloudera Observability component services all run in the Control Plane of the Cloudera Observability framework. Users access the Cloudera Observability web user interface from the web host server in this framework.
- Working Environment, which contains your Workload Clusters in your Workload environments, such as Production, Development, and Staging.
- Workload Cluster, which is one or more CDP clusters managed by Cloudera Manager. Telemetry Publisher is associated with a cluster by Cloudera Manager.
The below diagram shows the communication between Cloudera Observability and your workload clusters through Telemetry Publisher. Where, the Cloudera Observability service, including its main component services, run in the Cloudera Control Plane and the area on the right, behind your firewall, is your Working Environment that contains the clusters and services required to run your workload jobs and queries.
Cloudera Manager (not shown) manages one or more clusters in each of your working environments. Telemetry Publisher is enabled and configured for Cloudera Observability from each Cloudera Manager instance in your working environment.
For example, as shown in the diagram below, Customer A's Production environment contains two clusters that are both managed by one instance of Cloudera Manager, whereas the Development environment, which also contains two clusters, is managed by two instances of Cloudera Manager, one for each cluster. In this case, only one Telemetry Publisher service is enabled for the Production environment and two Telemetry Publisher services are enabled for the Development environment.
Telemetry Publisher collects and sends diagnostic information about jobs and queries from your Workload Clusters to Cloudera Observability. To ensure that all data transfer is secure between your Workload Clusters and Cloudera Observability, Telemetry Publisher communicates with Cloudera Observability’s endpoints using Transport Layer Security (TLS), as follows:
- When a job or query is completed, Telemetry Publisher connects to the Cloudera Observability service and asks to upload the workload diagnostic information. Once the request is authorized and verified, Cloudera Observability replies with a signed S3 URL that can then be used to upload the workload diagnostic information by Telemetry Publisher.
- When the URL is received, Telemetry Publisher performs a secure and direct protocol test using the Cloudera Observability S3 URL, before sending any diagnostic data.
