Understanding Cloudera Observability Financial Governance feature
The Cloudera Observability Financial Governance feature collects CPU, GPU, memory, data read, data written, and resource usage data from your environment, allocates those charges to your custom cost centers, and visually displays the results. It provides in-depth visibility into the resource usage of your workloads and services, as well as the costs of your environment’s infrastructure, which can be used for planning, budgeting, and forecasting.
The Cloudera Observability Financial Governance feature allows you to track costs at the environment level and cost center level as created by the administrators, providing a view of infrastructure costs. You can compare the Provisioned and Utilized costs for a cluster, which includes CPU, GPU, Memory, Data Read, Data Write and the average costs, allowing you to resize the cluster based on the previous usage using the Date Range and Granularity drop-downs.
The Cloudera Observability Financial Governance feature allows you to track costs at the environment level and cost center across various organizations and departments. You can create a cost center for your organization, department, region, or any other by defining the costs using the chargeback criteria.
The Financial Governance feature collects data on CPU, GPU, memory, data read, data written, and resource usage from your environment, allocates these charges to custom cost centers, and visually presents the results. This feature provides detailed insights into resource usage of your workloads and services, supporting effective planning, budgeting, and forecasting.
For instance, you can create three cost centers based on regions - Asia-Pacific, America, and European Union and define the costs using the chargeback criteria. Based on the provisioned budget for these three regions, you can track the utilized budget for these regions, say for six months. This provides an understanding of the budgeting and forecasting based on the historical data and helps you to effectively plan for the next year.
About the Cloudera Observability Cost Centers feature
This feature measures and records the resource usage of your workloads and allocates them to the users who consume them. For resources that are shared, such as multi-tenant clusters that are shared between different organizations and departments, it also enables you to measure and record those shared costs and charge those users based on their actual consumption. This feature helps you plan and forecast budgets, ensuring costs align with business requirements. The cost center reports can be used to raise cost awareness and set limits to control your overall costs.
About cost centers and their criteria
- Users or Pool usage, for instance, YARN pools for YARN-based offerings, enables you to separate user and resource pool costs.
- CPU, GPU, and Memory unit costs are based on actual CPU, GPU, and memory usage using your internal pricing or cost model.
- Data read and written unit costs are charged per gigabyte (GB) for the data read and written by your application.
Using the Cost Centers criteria that you have set, charges for CPU, GPU, memory, data read, and data written consumption are calculated and assigned by Cloudera Observability to a cost center that is created by you. Cost centers separate costs across users or pools and track their workload resource consumption costs. They can be divided and/or grouped into members associated with an organization or group to help you assign the charges to a user’s department.
When you create a Cloudera Observability Cost Center, detailed summary reports of the costs and resource usage for the environment are generated. After a job has run, the tracked resource costs that are associated with the cost center’s environment, service, or cluster are visually displayed. You can drill down for more detailed reports, such as viewing the costs incurred by a specific user or pool or viewing the top 500 users or pools whose jobs created the highest costs or the top 500 jobs or queries that created the highest costs.
Over time, as more jobs and queries are run, you can view and compare historical trends by selecting specific periods from the Date Range and Granularity lists. By default, data is retained for six months.
The Cloudera Observability Cost Centers feature uses usage-based metrics for CPU and GPU utilization and memory consumption that have an aggregation.
The cost centers costs are calculated based on a cost per resource unit, where:
- The CPU and GPU costs are expressed as the amount of time a job process uses the CPU and GPU within an hour.
- The Memory costs are expressed as an allocation cost per gigabyte.
Considerations and limitations
-
Cost centers aggregate the charges, where a cost center can be for one individual user, multiple users, or pools. To avoid cost duplication, users and pools must only be assigned one cost center.
-
When viewing the cost center reports, the costs are adjusted to a user's local timezone. Therefore, total costs, such as daily charges, may differ across time zones.
Assumptions and prerequisites
- Your organization has an internal pricing or cost model.
- You have created Cloudera Observability users or resource pools and assigned them to your workloads.
