Adjusting logging configuration in Advanced Settings

You can customize the logging configurations for SQL Stream Builder (SSB) jobs on the Streaming SQL Console globally or at job level. Adjusting the log configuration enables you to control the log levels of all the underlying libraries: Flink, Hadoop, Kafka, Zookeeper, other common libraries, and connectors to get more or less information in your job’s log.

The customization of the log configuration works differently whether it’s applied globally or at job level based on the job deployment mode.

Global logging configuration

important

Only users with admin rights can access the global configuration. To grant admin rights, the user needs to be added to the ssb.admin property in Cloudera Manager.

Navigate to the Streaming SQL Console.
1. Go to your cluster in Cloudera Manager.
  1. Select SQL Stream Builder from the list of services.
  2. Click SQLStreamBuilder Console.
The Streaming SQL Console opens in a new window.
Click the Configuration tab.
Find the ssb.admins property.
tip
You can type ssb.admins into the filter bar to narrow the list of configuration properties.
Click the + icon on the right hand side of the last username.
Add the username to the new row.
Write a reason for change and click Save changes.
Restart the SSB instance for the changes to be applied.

To set the global logging configuration:

Navigate to the Streaming SQL Console.
1. Go to your cluster in Cloudera Manager.
  1. Select SQL Stream Builder from the list of services.
  2. Click SQLStreamBuilder Console.
The Streaming SQL Console opens in a new window.
In the left hand navigation click Configuration.
If no global configuration is present, acknowledge the system notice by clicking Set global configuration.
In the editor, set the configuration.
Click Save.

Session mode

The execution.target is set to yarn-session mode, this is the default execution mode.

The log configuration is set at the start time if the Flink YARN session is applied to every job execution. For example, the current log configuration is applied if and only if the Flink YARN session is not set on the Session tab of the Compose page.

Per-job mode

The execution.target is set to yarn-per-job mode.

When you change the default execution mode to per-job, the currently applied log configuration is going to be used for the job. To configure the execution mode, you need to start the SQL query with the following line:

SET 'execution.target'='yarn-per-job';

Click New Job or select a previous job on the Jobs page.
You are redirected to the SQL Editor of the job.
Click Job Settings.
Click Logging.
Click Set Log Configuration.
Modify the settings based on your requirements.
Click Save. (You can restore the default logging configuration through clicking Delete and Set Log Configuration again.)
Close the Job Settings window.
Add and execute a SQL statement.
Click SQL Jobs.
Search for the job you have executed previously.
Click Flink Dashboard.
The Flink Dashboard opens in a new window.
Click Task Managers > Logs.
The log information appears in the log window based on your custom configurations.