Cloudera Docs

Adjusting logging configuration in Advanced Settings

You can customize the logging configurations for the SQL Stream Builder (SSB) job on the Streaming SQL Console in per-job mode or session mode. Adjusting the log configuration enables you to control the log levels of all the underlying libraries: Flink, Hadoop, Kafka, Zookeeper, other common libraries, and connectors to get more or less information in your job’s log.

The customization of the log configuration works differently based on the job deployment mode:
Session mode
The execution.target is set to yarn-session mode, this is the default execution mode.
The log configuration is set at the start time if the Flink YARN session is applied to every job execution. For example, the current log configuration is applied if and only if the Flink YARN session is not set on the Session tab of the Compose page.
Per-job mode
The execution.target is set to yarn-per-job mode.
When you change the default execution mode to per-job, the currently applied log configuration is going to be used for the job. To configure the execution mode, you need to start the SQL query with the following line:
SET 'execution.target'='yarn-per-job';
  1. Click New Job or select a previous job on the Jobs page.
    You are redirected to the SQL Editor of the job.
  2. Click Job Settings.
  3. Click Advanced.
  4. Modify the settings based on your requirements.
  5. Close the Job Settings window.
  6. Click Save to save the job settings.
  7. Add and execute a SQL statement.
  8. Click SQL Jobs.
  9. Search for the job you have executed previously.
  10. Click Flink Dashboard.
    The Flink Dashboard opens in a new window.
  11. Click Task Managers > Logs.
    The log information appears in the log window based on your custom configurations.