Cloudera Docs

Query and job resource optimization using resource efficiency analysis

You can evaluate the efficiency of individual jobs or queries based on the insights related to the resources requested versus the actual consumption. Queries or jobs with excess resource allocation waste cluster resources that can be better utilized for other tasks. Therefore, it is important to identify and analyze these inefficient jobs or queries.

Resource efficiency analysis feature availability

  • The Cloudera Observability On-Premises Resource efficiency analysis feature is available only for CDP Private Cloud Base and CDP Data Hub environments.
  • Currently, the feature is available for Hive on Tez when using YARN as a scheduler.

Key benefits

  • Identifies inefficiencies: Highlight jobs or queries that over-allocate resources such as CPU and memory.
  • Optimizes resource allocation: Ensure resources are allocated more efficiently, reducing waste and improving overall system performance.
  • Cost analysis: Provides insights into costs associated with different resources used by the query.