Cloudera Docs

Executing SQL jobs in production mode

As by default the SQL jobs are running in a session cluster, there is a risk in case of a cluster failure that every job is affected within that cluster. However, you can set a per-job production mode in SQL Stream Builder to create a dedicated environment for your production jobs.

Production mode means that the deployed SQL job (Flink job) runs in a separate SQL session and on a dedicated YARN cluster configured specifically for that particular job. Set production (per-job) execution mode in the Job Settings window; do not use a SET statement for execution.target in the SQL Editor.

Setting production mode in Job Settings

To set the production mode for your SQL jobs in the Jobs Setting page, you need to select the execution mode with the following steps:
  1. Click Job Settings at the SQL Editor.
  2. Select PROD - Job runs in isolated Cluster at Execution Mode.
  3. Add a SQL statement to the SQL Editor you want to execute.
  4. Click Execute.

Setting the runtime mode in Job Settings

Select the desired runtime mode from the Runtime Mode dropwdown menu.

  • STREAMING: the default streaming execution mode.
  • BATCH: batch-stlye execution for bounded application.
  • AUTOMATIC: the system decides, based on the boundedness of sources.