Cloudera Docs

Displaying your costs associated with a cost center

When a Cloudera Observability cost center is created, detailed summary reports of the costs and resource usage for the environment are also generated, which enable you to analyze the costs and the cost breakdown associated with the cost center. You can view the current and historical costs and the resource usage associated with your cost centers.

Steps on how to view the detailed summary reports associated with a cost center.

The reports visually display the tracked resource consumption and usage costs associated with the cost center for a specific time-period that you select from the time-range list.

Within each report you can to drill down further:
  • To view by users, resource pools, jobs, queries, projects, teams, and application types with the highest costs. Note that projects and teams are only available in the Cloudera AI environment, and resource pools are not applicable for Cloudera AI.
  • To view the health of a job or query of interest. To optimize costs by using the Cloudera Observability prescriptive guidance and recommendations that enable you to improve performance and resource usage.
  • Enabling the Show Resource Usage checkbox allows you to track resource usage, including CPU, GPU, memory, and data read/write. For example, CPU and GPU usage can be viewed in core-hours.
  1. Verify that you are logged in to the Cloudera Observability on premises web UI.
    1. In the URL field of a supported web browser, enter the Cloudera Observability on premises URL that you were given by your system administrator and press Enter.
    2. When the Cloudera Observability on premises Log in page opens, enter your Cloudera Observability on premises user name and password access credentials.
    3. Click Log in.
      The Cloudera Observability on premises landing page opens.
  2. From the Cloudera Observability Main navigation panel, select Financial Governance > Cost Centers.
    The Cloudera Observability Cost Centers page opens, which displays:
    • The provisioned cost, and the provisioned cost of CPU, memory, and GPU cost for all of your environments. Data read and written values for all of your cost centers.

      Provisioned cost = Provisioned CPU + Provisioned Memory + Provisioned GPU + Data read + Data write Data read and written values for all of your cost centers.

    • Lists your existing cost centers that use the current criteria settings and displays the provisioned costs, CPU, GPU, and Memory usage and data read and written values associated with each cost center.
    • The provisioned cost, CPU, GPU, and memory usage, and data read and written values for users and pools that are not yet assigned to a cost center in the uncategorized section.
  3. To display a cost center's detailed report that includes costs for each chosen environment, do the following:
    1. From the Date Range and Granularity list, select a time-period that meets your requirements.
    2. From the Cost Centers page, click inside the cost center row that requires analysis.
      The cost center's report page opens, which displays the following:
      • The provisioned costs, CPU, GPU, Memory usage, and Data Read and Data Written cost per GB for the cost center.
      • Lists the environments that are associated with the cost center and displays their provisioned costs, CPU, GPU, and Memory usage, and data read and written cost.
    3. To view more details about an environment, click on the environment name.
  4. To display the users, pools, jobs, queries, projects, teams, and application types that created the highest costs on a specific cluster of interest, do the following:
    1. Click inside the cluster row that requires more analysis.
      The cluster report overview page opens, which displays the following:
      • The top 500 users whose jobs created the highest costs.
      • The top 500 pools whose jobs created the highest costs.
      • The top 500 jobs or queries that created the highest costs.
    2. To gain more insights on the health of a job or query, click the name of the job or query listed in the Top 500 Jobs panel that requires more investigation.

      The job or query's summary page opens.

    3. Select the Health Checks and Execution Details tabs for more insights, and if available, read the optimization recommendations.
    4. Enabling the Show Resource Usage checkbox allows you to track resource usage, including CPU, GPU, memory, and data read/write. For example, CPU and GPU usage can be viewed in core-hours.
  5. To view a full list of users, including their job costs, CPU, GPU, and Memory usage, select the Users tab.
    The Users report opens, which displays the following:
    • The name of the user.
    • The total cost that the user incurred.
    • The number of jobs that the user ran.
    • The CPU, GPU, and Memory usage.
    • The cost per GB for the data read and written by your application.
    • The total job costs that the Cloudera Observability engines incurred: AI Jobs, Impala, Hive, Spark, MapReduce, and Oozie.
  6. To view a full list of pools, including their job costs, CPU, GPU, and Memory usage, select the Pools tab.
    The Pools report opens, which displays the following:
    • The name of the pool.
    • The environment that the pool is associated with.
    • The total cost that the pool incurred.
    • The number of jobs that the pool ran.
    • The CPU, GPU, and Memory usage.
    • The cost per GB for the data read and written by your application.
    • The total job costs that the Cloudera Observability incurred: AI Jobs, Impala, Hive, Spark, MapReduce, and Oozie.
  7. To view the list of Jobs & Queries, their CPU, GPU, and Memory usage, select the Jobs tab.
    The Top 500 Jobs report opens, which displays the following:
    • The name of the job or query.
    • The total cost that the job or query incurred.
    • The CPU, GPU, and Memory cost and resource usage.
    • The cost per GB and resource usage for the data read and written by your job or query.
    • The total job costs that the Cloudera Observability engines incurred: AI Jobs, Impala, Hive, Spark, MapReduce, and Oozie.
  8. To view the list of Projects, their CPU, GPU, and Memory usage, select the Projects tab.
    The Projects report opens, which displays the following:
    • The name of the Project.
    • The number of jobs for that project.
    • The total cost that the project incurred.
    • The CPU, GPU, and Memory usage.
    • The cost per GB for the data read and written by your application.
  9. To view the list of Teams, their CPU, GPU, and Memory usage, select the Teams tab.
    The Teams report opens, which displays the following:
    • The name of the Team.
    • The number of jobs for that team.
    • The total cost that the project incurred.
    • The CPU, GPU, and Memory usage.
    • The cost per GB for the data read and written by your application.
  10. To view the list of Application types, their CPU, GPU, and Memory usage, select the Application Types tab.
    The Application Types open, displaying the Top Application Types, which include the engines Impala, Hive, Spark, MapReduce, and Oozie. However, if the environment is Cloudera AI, the Top Application Types will display Application, Session, Job, and Model.
    • The name of the Application Type.
    • The number of jobs for the application type.
    • The total cost that the user incurred.
    • The CPU, GPU, and Memory cost and resource usage.
    • The cost per GB and resource usage for the data read and written by your application.
    • The total job costs that the Application Type incurred.