Trigger selection information
This section provides additional information on trigger selection.
Trigger list
Scope: All services except Impala Query
| Triggers | Description |
|---|---|
| Allocated Cores | The total number of virtual cores allocated to the application's running containers. |
| Allocated Memory | The total memory, in MB, allocated to the application's running containers. |
| Duration | The time elapsed since the application started, measured in milliseconds. |
| Number of Containers | The current number of containers running for the application. |
| Total Core Duration (virtual core-seconds) | The CPU resources allocated to the application, measured in virtual core-seconds. |
| Total Memory Duration (MB-seconds) | The memory allocated to the application, measured in megabyte-seconds. |
Scope: Spark only
| Triggers | Description |
|---|---|
| Input Data Read | Total bytes read from storage |
| Output Data Written | Total bytes successfully written to storage by an active Spark job |
|
Input Records |
The complete count of individual object rows loaded into a running Spark context |
|
Output Records |
The complete count of individual object rows output by the final tasks |
| Output to Input Record Ratio |
A derived evaluation representing the mathematical ratio of: Output Records/Input Records This is applicable when Input Records is greater than 0. |
| Shuffle Read | Total bytes pulled remotely or locally by executors during a shuffle dependency stage |
| Shuffle Written | Total bytes written out to local blocks by upstream tasks to be consumed during a downstream reduce stage |
| Memory Spilled | The total uncompressed footprint of active Java objects in memory before serialization and eviction to disk |
| Disk Spilled | The compressed, serialized footprint of data actively written to the local disk of the executor during a memory eviction |
| Requested Containers | The total number of containers the application requests. |
| Requested Memory | The total amount of memory the application requests across all containers. |
| Requested VCores | The total number of virtual cores (vCores) the application requests across all containers. |
Scope: Impala Query only
| Triggers | Description |
|---|---|
| Input Data Read | Total bytes read from underlying storage layers (S3, Apache Ozone, GCS, ADLS, or HDFS) |
| Output Data Written | Total bytes written to storage |
|
Input Records |
Total physical row count extracted from the underlying storage layer. |
|
Output Records |
Total physical row count written to the target storage destination |
| Shuffle Read | Total network bytes received by query fragments over the network from other hosts by using internal EXCHANGE_NODE buffers |
| Shuffle Written | Total network bytes transmitted by query fragments to neighboring hosts by using the internal DataStreamSender |
| Memory Spilled | Total raw data size evicted to the local scratch disk because processing execution (such as joins or sorts) exceeded the available query memory allocation |
| Memory Used | The amount of memory used |
| CPU Usage | The amount of CPU time used |
| Duration | The duration of a ReservationRequest is the consecutive time for which the requested resource allocation must be available. |
Scope: MapReduce (MR) only
| Triggers | Description |
|---|---|
| Input Data Read | Total bytes parsed from HDFS or cloud storage targets during the initial Map Phase |
| Output Data Written | Total bytes committed to final storage locations after completing the Reduce Phase |
| Shuffle Read | Network bytes moved between Mapper outputs and downstream Reducer tasks |
| Shuffle Written | Total intermediate bytes cached to the local disk partition of a task host for subsequent reducer pickup |
| Memory Spilled | Transferred intermediate bytes forced to local disk allocations because the circular in-memory buffer reached capacity |
| Disk Spill Records | Total record count tracking objects moved to local disk storage during map and reduce phases |
