Managing SQL jobs

You can reuse an already submitted SQL query from the Console page or the History page of Streaming SQL Console.

When you need to reuse a SQL query for a new job, you can load it and execute it in the SQL Editor. You can use the Query History button on the Console page or you can select the History page from the main menu of Streaming SQL Console.

• Console page
• History page

You can view the DDL of the submitted query by hovering over the items. You can load the query to the editor from the Console page by clicking on the query in the Query History window. From the History page, you need to click on the query, and select Paste to Active Job.

You can reload the already executed jobs regardless of success to the SQL Editor to make furhter and additional changes, and execute them again.

The details of the job are reloaded to the SQL Editor with the latest saved configuration and Materialized Views.

As a SQL Stream job processes streaming data, you need to stop the job to finish the process, and not let the job run for an unlimited time. You can stop a running job either on the Console page or the SQL Jobs page.

Stopping job from Console page

If the job is currently loaded to the SQL Editor, you need to click on the Stop button to terminate the job.

If the job is not loaded to the SQL Editor, you need to click on the SQL Jobs button to open the list of SQL jobs window. Without selecting the running job, click on the button next to the job status, and select Stop.

Stopping job from SQL Jobs page

On the SQL Jobs page, you need to click on the button next to the job status, and select Stop.

You can filter down the list of job only to the running jobs by selecting Running from the filter.

For changing the order, click on the name of the column, the direction of the ordering is indicated next to the column name by arrows. For further filtering, you can either click the button next to the column name or under the row of the column name. The filtering value can be changed by typing the values directly to the filtering row, the conditions need to be changed using the filtering arrow.

You can restart an already running job when it is active in the SQL Editor. You can also restart a stopped job by executing it again either from the Console page or the SQL Jobs page.

Restarting a running SQL job

If the job is currently loaded to the SQL Editor, you need to click on the Restart button to restart the job.

You can edit every detail of your SQL jobs after stopping and reloading it to the SQL Editor on the Console page.

You can delete running and stopped jobs from the Console and SQL Jobs page. The deleted job no longer appears on the Streaming SQL Console.

Deleting job from Console page

You need to click on the SQL Jobs button to open the list of SQL jobs window. Without selecting the stopped job, click on the button next to the job status, and select Delete.

Deleting job from SQL Jobs page

On the SQL Jobs page, you need to click on the button next to the job status, and select Delete Job.

You can filter down the list of jobs by selecting Stopped or Running from the filters.

For changing the order, click on the name of the column, the direction of the ordering is indicated next to the column name by arrows. For further filtering, you can either click the button next to the column name or under the row of the column name. The filtering value can be changed by typing the values directly to the filtering row, the conditions need to be changed using the filtering arrow.

If you need to further customize your SQL Stream job, you can add more advanced features to configure the job restarting method and time, threads for parallelism, sample behavior, exatly once processing and restoring from savepoint.

Before running a SQL query, you can configure advanced features by clicking on the Settings tabs at the SQL window.

Job parallelism (threads)

The number of threads to start to process the job. Each thread consumes a slot on the cluster. When the Job Parallelism is set to 1, the job consumes the least resources. If the data provired supports parallel reads, increasing the parallelism can raise the maximum throughput. For example, when using Kafka as a data provider, setting the parallelism to the equal number as the partitions of the topic can be a starting point for performance tuning.

Sample Count

The number of sample entries shown under the Results tab. To have an unlimited number of sample entries, add 0 to the Sample Count value.

Sample Window Size

The number of sample entries to keep in under the Results tab. To have an unlimited number of sample entries, add 0 to the Sample Window Size value.

Sample Behavior

You have the following options to choose the behavior of the sampled data under the Results tab:

• Sample all messages
• Sample one message every second
• Sample one message every five seconds

Restore From Savepoint

You can enable or disable restoring a SQL job from a Flink savepoint after stopping it. The savepoint is saved under hdfs:///user/flink/savepoints by default.

Enable Checkpointing

You can enable and disable checkpointing for a SQL job. By default the checkpointing is enabled.

Checkpoint Mode

Switching between checkpointing modes. You can choose between At Least Once or Exactly Once.

Checkpoint Interval

The time in milliseconds between checkpointing attempts.

Checkpoint Timeout

The maximum time in milliseconds until a checkpointing attempt is timed out.

Tolerable Checkpoint Failures

The number of checkpointing attempts until the job is aborted.

Failure Restart Strategy

Switching between restarting strategies when checkpointing is failed. You can choose between enabling and disabling auto recovery. By default auto recovery is enabled for checkpointing.

You can use the REST API to monitor, manage and configure the SQL Stream jobs with GET, POST and DELETE HTTP methods. You can use the SQL Stream Builder (SSB) REST API in command line, import them to REST API Tools or by accessing the Swagger UI.

The REST API Reference document contains the available endpoints for SQL Stream Builder.

You can also reach the SSB REST API reference document from on the main menu of Streaming SQL Console using the API Explorer.