Configuring SQL Client for session mode

Every Flink SQL query is an independent Flink job. As with other Flink applications, you must decide on how you want to run them. The queries can run as standalone (per-job) YARN applications, as the default mode for all Flink jobs, or you can run them on a Flink session cluster.

While the per-job mode ensures better resource isolation and production characteristics, this mode requires more time to startup that can be inconvenient when exploring the SQL API. If you decide to stay in the per-job mode, you only need to start the SQL client without any additional parameters.

If you decide to run a Flink session cluster, perform the following steps.

1. Start a Flink session

You can start a Flink YARN session from the command line:

flink-yarn-session -tm 2048 -s 2 -d

This command starts a new session and specifies the Taskmanager size for future executors. You do not need to set the size of the cluster, because Flink does this automatically. The cluster starts without any TaskManager containers, and will grow and shrink in size as the number of deployed queries change.

2. Configure the SQL Client for session mode

To configure the SQL Client for the session mode, you need to create an environment YAML file (sql-env.yaml), and add the following configuration:

configuration:
  execution.target: yarn-session

This overrides the Flink configuration of the execution target to yarn-session mode.

The SQL Client start command looks like this:

flink-sql-client embedded -e sql-env.yaml