Configuring SQL Client for session mode
Every Flink SQL query is an independent Flink job. As with other Flink applications, you must decide on how you want to run them. The queries can run as standalone (per-job) YARN applications, as the default mode for all Flink jobs, or you can run them on a Flink session cluster.
While the per-job mode ensures better resource isolation and production characteristics, this mode requires more time to startup that can be inconvenient when exploring the SQL API. If you decide to stay in the per-job mode, you only need to start the SQL client without any additional parameters.
If you decide to run a Flink session cluster, perform the following steps.
1. Start a Flink session
flink-yarn-session -tm 2048 -s 2 -d
This command starts a new session and specifies the
Taskmanager size for
future executors. You do not need to set the size of the cluster, because Flink does this
automatically. The cluster starts without any
TaskManager containers, and
will grow and shrink in size as the number of deployed queries change.
2. Configure the SQL Client for session mode
sql-env.yaml), and add the following configuration:
configuration: execution.target: yarn-session
This overrides the Flink configuration of the execution target to
flink-sql-client embedded -e sql-env.yaml