Calcite planner in Impala [Preview]
The Apache Calcite planner is a cost-based query optimizer for Impala that evaluates alternative plans, such as different join orders, to choose a more CPU- and memory-efficient execution path.
This feature is available in Cloudera Runtime 7.3.2.10000 and higher. Impala compiles each query with either its original planner or the Apache Calcite planner. The Calcite planner applies cost-based optimization to select an execution plan. It is disabled by default, and you enable it per session with a query option.
How the Calcite planner works
When you set the PLANNER query option to calcite, Impala compiles queries with the Calcite planner. By default, both the PLANNER and FALLBACK_PLANNER query options are set to original, so queries run on the original planner. When you change PLANNER to calcite and leave FALLBACK_PLANNER at its default, Impala falls back to the original planner whenever the Calcite planner cannot compile a query.
If the Calcite planner cannot compile a query, the FALLBACK_PLANNER query option and the cause of the failure together determine whether Impala falls back to the original planner.
Impala always falls back to the original planner in the following cases, regardless of the FALLBACK_PLANNER value:
- Data Definition Language (DDL) statements and other statements that are not SELECT queries.
- Queries that use features the Calcite planner does not yet support, such as complex types.
For all other compilation failures, the FALLBACK_PLANNER value determines the behavior. The following table describes the supported values:
| Value | Description |
|---|---|
| original | (default) Impala falls back to the original planner when the Calcite planner cannot compile a query. |
| none | Impala does not fall back to a different planner. This value behaves as though FALLBACK_PLANNER matches PLANNER. When PLANNER is calcite, the none and calcite values produce the same result. |
| calcite | Impala does not fall back for unexpected compilation failures, and the affected query fails instead. |
When to use the Calcite planner
Consider enabling it in the following cases:
- Complex queries, such as those with many joins, large multi-table joins, or deeply nested subqueries, where cost-based join ordering is most likely to improve performance.
- Queries that use constructs the original planner does not support, such as certain correlated subqueries in the SELECT list.
- Short or simple queries might not benefit from the Calcite planner and can continue to use the original planner, which is the default.
