Telemetry data exposed to OTel collector
HiveServer2 transmits telemetry data related to live queries, completed queries, task-level details, JVM metrics related to memory usage and thread count, and Operating System level statistics.
Query and Task related insights
- Live queries
- The following metrics related to live HiveServer2 queries are transmitted to the OTel collector:
Metrics Description QueryId Represents the unique identifier for the query. QueryString The SQL statement of the query. UserName The user who submitted the query. ExecutionEngine The query execution engine used to process the query, typically Tez. ErrorMessage Errors encountered during query execution. - Completed queries
- The following metrics related to completed HiveServer2 queries are transmitted to the
OTel collector:
Metrics Description QueryId Represents the unique identifier for the query. QueryStartTime Timestamp when the query execution started. EndTime Timestamp when the query execution completed. OperationId Unique identifier for tasks related to query execution. For example, a0fe8acc-6b9a-4f54-8537-f7b3bf7dea72 OperationLogLocation Path to the local log file for additional query log entries. ErrorMessage Errors encountered during query execution. ExplainPlan Output showing how the engine executed the query. FullLogLocation Location of the complete application logs related to the query. Running Indicates whether the query is currently in progress. Runtime Duration of the query execution in seconds or minutes. UserName The user who submitted the query. ExecutionEngine The query execution engine used to process the query, typically Tez. State Current state of the query (e.g., RUNNING, SUCCESS, ABORTED, or FAILED). SessionId Identifier for the session in which the query was executed. - Task-level details
- The following metrics related to tasks are transmitted to the OTel collector:
Metrics Description TaskId Unique identifier for each task. Name Task types such as MAPRED (Mapper/Reducer tasks), DEPENDENCY_COLLECTION, or STATS TASK. If multiple tasks exist, integers are appended. TaskType Representation with unique values while execution such as MAPRED, DEPENDENCY_COLLECTION, MOVE, or STATS. Status Current task status (e.g. Success). StatusMessage Status with detailed task information ExternalHandle DAGID, used for query optimization and execution. ErrorMsg Error details if the task failed. ReturnValue Task result. A value of 0 indicates success, while negative integers indicate an issue. BeginTime Time when the task execution started. ElapsedTime Time spent executing the task. EndTime Time when the task execution finished.
Java Virtual Machine (JVM) metrics
HiveServer2 collects various JVM metrics, including:
- Memory usage: Data such as heap and non-heap memory usage.
- Thread count: Counts of threads in different states (for example, runnable, waiting).
- OS-Level statistics: CPU load, memory size, and swap space details.
- Metrics related to memory usage
- The following metrics related to JVM heap and non-heap memory usage are transmitted to
the OTEL collector:
Metrics Description memNonHeapUsedMGauge Size (in MB) of non-heap memory currently used by the JVM. memNonHeapCommittedM Amount of non-heap memory (in MB) reserved by the JVM for internal use. memNonHeapMaxM Maximum allowable size (in MB) for non-heap memory. memHeapUsedM Size (in MB) of heap memory currently used by the JVM. memHeapCommittedM Size (in MB) of heap memory reserved by the JVM. memHeapMaxM Maximum allowable size (in MB) for heap memory. memHeapMaxM Maximum memory available (in MB) to the JVM. - Metrics related to thread count
- The following metrics related to the JVM threads in different states (runnable,
waiting) are transmitted to the OTel collector:
Metrics Description threadsNew Number of threads currently in the "NEW" state within a Java application. threadsRunnable Number of threads ready to run or currently executing on the CPU. threadsBlocked Number of threads waiting to acquire a lock. threadsWaiting Number of threads waiting indefinitely for a signal from another thread. threadsTimedWaiting Number of threads waiting for a specific duration before proceeding. threadsTerminated Number of threads where execution is completed. - OS-level statistics
- The following OS related metrics, such as CPU load, memory size, swap space details
are transmitted to the OTel collector:
Metrics Description systemCpuLoad Measures the CPU load of the host machine, container, or pod. committedVirtualMemorySize Amount of allocated memory, including both physical RAM and virtual memory reserved by the operating system. processCpuTime Total CPU time consumed by a specific process or thread. freePhysicalMemorySize Amount of unused physical memory (RAM) available to processes and the operating system, container, or pod. freeSwapSpaceSize Amount of swap space currently available. totalPhysicalMemorySize Total installed physical memory (RAM) in the system. processCpuLoad Percentage of CPU load consumed by a specific process.
