Cloudera Docs

Using SQL job notification

You can set up notifications for SQL jobs using Streaming SQL Console to alert individuals and teams about job failures. The notifications include email and webhook alerts, which you can also group together for easier organization.

You can add job notifications to SQL jobs to send alerts and information when the status of the job is changed. The information in the notification can include the job name, job ID, cluster ID, last exception, and Flink job ID.

You can create the following notification actions for SQL jobs:
Email
Sending notification email to the provided email address
Webhook
Sending notification to a webhook address using POST and PUT

You can also collect the Email and Webhook notifications into a Group to organize the notifications. These groups do not trigger any action, but serve as an organizational element.

After creating and adding the notification to a SQL job, anytime a change occurs for that job, a notification is sent to the specified address.

Enabling job notifications

Before you are able to use the job notifications, you need to enable it in Cloudera Manager and based on the notification type configure the email and webhook parameters.

  1. Navigate to Management Console > Environments, and select the environment where you have created your cluster.
  2. Select Cloudera Manager from the list of services.
  3. Select SQL Stream Builder from the list of services.
  4. Click Configuration.
  5. Search for Enable job notifications functionality.
  6. Enable the job notifications by checking the checkbox.
The following parameters also need to be configured for the SQL Stream Builder (SSB) service in Cloudera Manager based on the type of the notification:
Configuration Description
Job notifications monitoring interval Sets the interval of the job monitoring in seconds.
Mail server host for job notifications The host of the SMTP server for job failure notifications.
Mail server username for job notifications The username to access the SMTP server for job failure notifications.
Mail server password for job notifications The password to access the SMTP server for job failure notifications.
SMTP authentication for job notifications Enable SMTP authentication for job notifications.
StartTLS for job notifications Use the StartTLS command to establish a secure connection to the SMTP server for job notifications.
Job notifications sender mail address Sender mail address for job notifications.
Mail server port for job notifications The port of the SMTP server for job failure notifications.
Job notifications webhook sender parallelism Number of threads used by the job notification task to call user-specified webhooks when notifying about a failed or missing job.

Creating job notifications

After enabling the job notification function, you can create email, webhook and group notifications with Streaming SQL Console.

  1. Navigate to Streaming SQL Console.
    1. Navigate to Management Console > Environments, and select the environment where you have created your cluster.
    2. Select the Streaming Analytics cluster from the list of Data Hub clusters.
    3. Select SQL Stream Builder from the list of services.
    4. Click Streaming SQL Console.

      The Streaming SQL Console opens in a new window.

  2. Open a project from the Projects page.
    1. Select an already existing project from the list by clicking the Open button or Switch button.
    2. Create a new project by clicking the New Project button.
    3. Import a project by clicking the Import button.

      You are redirected to the Explorer view of the project.

  3. Click next to Job Notifications.
  4. Select from the following options based on which notification type you want to use:
    1. Click New Email Notification.

      The Create Email Notification window appears.

    2. Provide a Name for the notification.
    3. Add the Email address you want to send the notification.
    4. Provide a Subject.
    5. Provide a Message.

      The message of the Webhook notification must be a valid JSON.

      You can add placeholder items to the payload that will be converted into the specific job information. You can add the placeholders in ${placeholder} format, and the following placeholders are available:
      • jobName
      • jobStatus
      • jobStatusDescription
      • ssbJobId
      • flinkJobId
      • clusterId
      • lastException
    6. Click Save.
    1. Click New Webhook Notification.

      The Create Webhook Notification window appears.

    2. Provide a Name for the notification.
    3. Select between POST or PUT method.
    4. Provide a Webhook address.
    5. Provide a Payload to the webhook.
      You can add placeholder items to the payload that will be converted into the specific job information. You can add the placeholders in ${placeholder} format, and the following placeholders are available:
      • jobName
      • jobStatus
      • jobStatusDescription
      • ssbJobId
      • flinkJobId
      • clusterId
      • lastException
    6. Add HTTP Headers if needed by specifying the Header Name and Value.
    7. Click Save.
    1. Click New Group Notification.

      The Create Group Notification window appears.

    2. Select from the already created Email and Webhook notifications that you would like to add to this group.
    3. Click Save.

Enabling notifications for a SQL job

After creating a job notification, you can enable it for a selected job from the Job Settings window.

  1. Select Job Settings from the SQL Editor.
  2. Click Job Notifications.
  3. Search for the name that you have provided when creating the notification.
  4. Click on the notification.

    The selected notification is added to the job and the notification is sent with the information provided in the message template.