Resource efficiency and potential savings metrics
The Query Cost and Efficiency Analysis metrics dashboard in Cloudera Observability provides detailed insights into the efficiency of your queries or jobs, including usage percentages and potential savings.
- Query Cost: Displays the total cost of a specific query with a detailed breakdown by CPU, Memory, Data Input, and Data Output, measured in USD.
- CPU: Displays the percentage of CPU resources utilized by the job
or query execution. If spare CPU resources are available on the YARN cluster, YARN may
allocate additional CPU cycles to the job, leading to higher CPU usage than originally
allocated.
Potential savings are displayed by multiplying
the CPU cost defined at chargeback setup * unused CPU core hours. - Memory: Indicates the total memory consumed by the job or query,
represented as a percentage. This is calculated using the peak memory used during
execution, measured in gigabytes multiplied by milliseconds.
Potential savings are displayed by multiplying
the memory cost defined at chargeback setup * unused Memory GB hours. - Overall: Displays the average usage for both CPU and memory. The
percentage is calculated as
(CPU percentage + Memory percentage) / 2.Potential savings are calculated by adding potential savings from all resources:
(CPU Potential Savings + Memory Potential Savings).
These metrics help you identify the under-utilization of resources. High CPU or memory wastage may suggest the need to reallocate resources, optimize usage, or adjust configurations to allocate fewer resources.
Dashboard color indicators:
- Green: Usage is between 75% and 100%—indicating efficient utilization.
- Orange: Usage is between 25% and 74%—indicating moderate utilization.
- Blue: Usage is between 0% and 24%—indicating low utilization.
