Debug Web UI for impalad
The impalad Web UI includes information about configuration settings, running and completed queries, and associated performance and resource usage for queries.
To debug and troubleshoot an impalad using a
web-based interface, open the URL
in a browser. (For secure clusters, use the prefix
https:// instead of
Because each Impala node produces its own set of debug information, you should choose a specific node that you want to investigate an issue on.
- Main Page
impaladWeb UI page at / lists the following information about the
- The version of the impalad daemon
The Version section also contains other information, such as when Impala was built and what build flags were used.
- Process start time
- Hardware information
- OS information
- Process information
- CGroup information
- The version of the impalad daemon
- Admission Controller Page
The Admission Controller
impaladdebug Web UI is at /admission page under the main
Use the /admission page to troubleshoot queued queries and the admission control.
The admission page provides the following information about each resource pool to which queries have been submitted at least once:
- Time since the
statestoredreceived the last update
- A warning if this
impaladis considered disconnected from the
statestoredand thus the information on this page could be stale.
- Pool configuration
- Queued queries submitted to this coordinator, in the order of submission
- Running queries submitted to this coordinator
- Pool stats
- Average of time in queue: An exponential moving average which represents the average time in queue over the last 10 to 12 queries. If a query is admitted immediately, the wait time of 0 is used in calculating this average wait time.
- Histogram of the distribution of peak memory used by queries
admitted to the pool
Use the histogram to figure out settings for the minimum and maximum query
MEM_LIMITranges for this pool.
The histogram displays data for all queries admitted to the pool, including the queries that finished, got canceled, or hit an error.
Click on the pool name to only display information relevant to that pool. You can then refresh the debug page to see only the information for that specific pool.
Click Reset informational stats for all pools to reset the stats that keep track of historical data, such as Totals stats, Time in queue (exponential moving average), and the histogram.The above information is also available as a JSON object from the following HTTP endpoint:
- Time since the
- Known Backends Page
The Known backends page of the
impaladdebug Web UI is at /backends under the main
This page lists the following info for each of the impalad nodes in the cluster. Because each impalad daemon knows about every other impalad daemon through the StateStore, this information should be the same regardless of which node you select.
- Address of the node: Host name and port
- KRPC address: The KRPC address of the
impalad. Use this address when you issue the
SHUTDOWNcommand for this
- Whether acting as a coordinator
- Whether acting as an executor
- Quiescing status: Specify whether the graceful shutdown process
has been initiated on this
- Memory limit for admission: The amount of memory that can be admitted to this backend by the admission controller.
- Memory reserved: The amount of memory reserved by queries that
are active, either currently executing or finished but not yet
closed, on this backend.
The memory reserved for a query that is currently executing is its memory limit, if set. Otherwise, if the query has no limit or if the query finished executing, the current consumption is used.
- Memory of the queries admitted to this coordinator: The memory submitted to this particular host by the queries admitted by this coordinator.
- Catalog Page
The Catalog page of the
impaladdebug Web UI is at /catalog under the main
This page displays a list of databases and associated tables recognized by this instance of impalad. You can use this page to locate which database a table is in, check the exact spelling of a database or table name, look for identical table names in multiple databases. The primary debugging use case would be to check if an
impaladinstance has knowledge of a particular table that someone expects to be in a particular database.
- Hadoop Configuration
The Hadoop Configuration page of the
impaladdebug Web UI is at /hadoop-varz under the main
This page displays the Hadoop common configurations that Impala is running with.
The JMX page of the
impaladdebug Web UI is at /jmx under the main
This page displays monitoring information about various JVM subsystems, such as memory pools, thread management, runtime. etc.
- Java Log Level
The Change log level page of the
impaladdebug Web UI is at /log_level under the main
This page displays the current Java and backend log levels, and it allows you to change the log levels dynamically without having to restart the
- Logs Page
The INFO logs page of the
impaladdebug Web UI is at /logs under the main
This page shows the last portion of the
impalad.INFOlog file, including the info, warning, and error logs for the impalad. You can see the details of the most recent operations, whether the operations succeeded or encountered errors. This page provides one central place for the log files and saves you from looking around the filesystem for the log files, which could be in different locations on clusters that use cluster management software.
- Memz Page
The Memory Usage page of the
impaladdebug Web UI is at /memz under the main
This page displays the summary and detailed information about memory usage by the
- Metrics Page
The Metrics page of the
impaladdebug Web UI is at /metrics under the main
This page displays the current set of metrics, counters and flags representing various aspects of
- Queries Page
The Queries page of the
impaladdebug Web UI is at /queries under the main
This page lists:
- Currently running queries
- Queries that have completed their execution, but have not been closed yet
- Completed queries whose details still reside in memory
The queries are listed in reverse chronological order, with the most recent at the top. You can control the amount of memory devoted to completed queries by specifying the
-‑-‑query_log_sizestartup option for
impalad.This page provides:
- How many SQL statements are failing (
- How large the result sets are (
# rows fetched)
- How long each statement took (
Click the Details link for a query to display the detailed performance characteristics of that query, such as the profile output.
On the query detail page, in the Profile tab, you have options to export the query profile output to the Thrift, text, or Json format.
The Queries page also includes the Query Locations section that lists the number of running queries with fragments on this host.
- RPC Services Page
The RPC durations page of the
impaladdebug Web UI is at /rpcz under the main
This page displays information, such as the duration, about the RPC communications of this
impaladwith other Impala daemons.
- Sessions Page
The Sessions page of the
impaladdebug Web UI is at /session under the main
This page displays information about the sessions currently connected to this impalad instance. For example, sessions could include connections from the
impala-shellcommand, JDBC or ODBC applications, or the Impala Query UI in the Hue web interface.
- Threadz Page
The Threads page of the
impaladdebug Web UI is at /threadz under the main
This page displays information about the threads used by this instance of
impalad, and it shows which categories they are grouped into. Making use of this information requires substantial knowledge about Impala internals.
- Varz Page
- The Varz page of the
impaladdebug Web UI is at /varz under the main
This page shows the configuration settings in effect when this instance of
impaladcommunicates with other Hadoop components such as HDFS and YARN. These settings are collected from a set of configuration files.
The bottom of this page also lists all the command-line settings in effect for this instance of
- Prometheus Metrics Page
At /metrics_prometheus, under the main
impaladWeb UI, the metrics are generated in Prometheus exposition format that Prometheus can consume for event monitoring and alerting.
The /metrics_prometheus is not shown in the Web UI list of pages.