Caching Codegen Functions

In Impala, "codegen" refers to code generation, utilizing query-specific information to generate specialized machine code for each query.

When executing a standard query, the query optimizer generates an optimized query plan, passing it to the executor for processing. The codegen capability converts query-specific information into machine code for faster execution. However, for sub-second queries, codegen may consume excessive time; for instance, the compilation time for codegen could take 300ms for a sub-second query. To mitigate this, Cloudera introduces support for memory caches for codegen functions, enhancing the performance of sub-second queries. Users can specify the total cache size for all codegen functions. The configuration parameter 'codegen_cache_capacity' in Impala daemons controls the total codegen cache size. For example, when set to "1GB," it restricts the codegen cache size for the particular Impala daemon to one gigabyte.

The cache is a singleton instance for each daemon, housing multiple cache entries. Each cache entry operates at the fragment level and retains all codegen functions associated with a fragment. Upon reaching maximum capacity, the cache employs an LRU (Least Recently Used) eviction policy. Storing codegen functions in the cache enables their reuse when appropriate, reducing the need for repetitive codegen compilation, which may otherwise consume hundreds of milliseconds.

The LLVM module bitcode serves as the cache key, generated prior to LLVM module optimization and final compilation. When codegen_cache_mode is set to NORMAL, the complete bitcode string is stored as the key by default. Conversely, if codegen_cache_mode is set to OPTIMAL, only a key containing the hash code derived from the NORMAL mode key is stored, reducing memory consumption.

The memory allocated for the codegen cache operates independently of the admission memory reserved for query execution. While the codegen cache memory is initially limited to a certain percentage of the total process memory, adjustments can be made during startup if it exceeds that percentage.

You can deactivate the codegen capability by setting the query option disable_codegen_cache to true or by setting the flag file option 'codegen_cache_capacity' to 0. It is important to note that even if disable_codegen_cache is set to 'false', caching will still be disabled if you set codegen_cache_capacity to 0.