Configuring YARN queue for SQL jobs
You can configure the YARN application queue with a custom value for a SQL job using the Streaming SQL Console.
With YARN queues, you can deploy applications to a specific subset of nodes with separate or limited resources, according to configuration. This enables you to have a separate execution environment with limited resources to experiment with new applications without impacting any production operation.
The YARN application queue can only be configured in a per-job execution target using the production mode of SQL Stream Builder. This means that the default value of the YARN queue does not change, and you can only customize it to the specific job that is executed in the production mode.
The default value of YARN application queue can be configured in Cloudera Manager. After
accessing the SQL Stream Builder service on your cluster, you need to open the configurations,
and search for YARN application queue where you can change the default value. This default
value is overwritten when using the
SET statement in production mode for
specifying the YARN queue for a SQL job.
For more information about running jobs in production mode, see the Executing SQL jobs in production mode section.
SETstatement in Streaming SQL Console as the following code example shows:
--PROD set 'execution.target' = 'yarn-per-job'; set 'yarn.application.queue' = 'my-queue'; select * from datagen_table_1648801198