Cloudera Docs

Jobs and queries metrics

From the Cloudera Observability Real-time monitoring (RTM) Jobs and Queries page, you can monitor your current workload job, the query, and the associated users. You can gain insights into workloads, response times, resource utilization and queued or failed operations. The data helps you immediately identify workloads with resource consumption, latency, or run time issues.

Job and query performance metrics

The top section's cards on the Jobs and Queries page provide instant insights about your job and query activity:
  • RUNNING JOBS AND QUERIES BY ENGINES: Displays the number of currently running jobs or queries and the engines in which they are running.
  • WAITING JOBS AND QUERIES BY ENGINES: Displays the number of jobs or queries waiting in the queue to run and the engines in which they will run.
  • TOP 5 USERS BY JOBS AND QUERIES COUNT: Displays the top five users running workloads and whether the job or query is running or waiting to run.

The Jobs and Queries page is automatically refreshed every minute to update the job and query real-time metrics.

Filter job and query data

You can use the following filters to minimize the list of jobs and queries and focus on specific anomalies:
  • Search: Search for a specific job or query.
  • Status: Select the current state of the workload job or query and click Apply.
  • User: Select the user or users who run jobs and queries and click Apply.
  • Engine: Select the engine in which jobs and queries are run and click Apply.
  • Pool: Select the pool that the user is part of and click Apply.

Jobs and query metrics

The Jobs and Queries table lists the following queries and information about their status.

  • Completed

    The completed queries remain available in the Jobs and Queries table for 15 minutes.

  • Time out
  • Canceled
  • Failed

If you run Hive, Impala, and Spark jobs simultaneously, the data for all three jobs are displayed on the Jobs and Queries page.

  • Job/Query: The name of the workload job or query. You can hover over a job or query for more information.
  • User: The user who submitted the job or query.
  • Engine: The engine in which the job or query is run.
  • Pool: The name of the Pool that the user is part of.

    To avoid spikes and out-of-memory conditions, Cloudera recommends balancing your resource usage and throughput by dividing your resources into pools that run specific workloads.

  • Start: The actual start time that the current or last job or query ran.
  • Submitted: The point in time when the job or query was submitted for processing.
  • Duration: The duration for which the query has been running or waiting.
  • CPU: The CPU usage of the current or most recent execution of a job or query.
  • Memory: The total memory usage consumed by a job or query.