Cloudera Docs

Using SQL job notification

You can set up notifications for SQL jobs using Streaming SQL Console to alert individuals and teams about job status changes. The notifications include email and webhook alerts, which you can also group together for easier organization.

You can add job notifications to SQL jobs to send alerts and information when the status of the job is changed. The information in the notification can include the job name, job ID, cluster ID, last exception, and Flink job ID.

You can create the following notification actions for SQL jobs:
Email
Sending notification email to the provided email address
Webhook
Sending notification to a webhook address using POST and PUT
Group
Grouping email and webhook notifications into one group

After creating and adding the notification to a SQL job, anytime a change occurs for that job, a notification is sent to the specified address.

Enabling job notifications

Before you are able to use the job notifications, you need to enable it in Cloudera Manager and based on the notification type configure the email and webhook parameters.

  1. Go to your cluster in Cloudera Manager.
  2. Select SQL Stream Builder from the list of services.
  3. Click Configuration.
  4. Search for Enable job notifications functionality.
  5. Enable the job notifications by checking the checkbox.
The following parameters also need to be configured for the SQL Stream Builder (SSB) service in Cloudera Manager based on the type of the notification:
Configuration Description
Job notifications monitoring interval Sets the interval of the job monitoring in seconds.
Mail server host for job notifications The host of the SMTP server for job failure notifications.
Mail server username for job notifications The username to access the SMTP server for job failure notifications.
Mail server password for job notifications The password to access the SMTP server for job failure notifications.
SMTP authentication for job notifications Enable SMTP authentication for job notifications.
StartTLS for job notifications Use the StartTLS command to establish a secure connection to the SMTP server for job notifications.
Job notifications sender mail address Sender mail address for job notifications.
Mail server port for job notifications The port of the SMTP server for job failure notifications.
Job notifications webhook sender parallelism Number of threads used by the job notification task to call user-specified webhooks when notifying about a failed or missing job.

Creating job notifications

After enabling the job notification function, you can create email, webhook and group notifications with Streaming SQL Console.

  1. Navigate to Streaming SQL Console.
    1. Go to your cluster in Cloudera Manager.
    2. Select SQL Stream Builder from the list of services.
    3. Click Streaming SQL Console.

      The Streaming SQL Console opens in a new window.

  2. Click Job Notifications from the main menu on Streaming SQL Console.
  3. Click Create > Email.

    The Create Email Notification window appears.

  4. Provide a Name for the notification.
  5. Add the Email address you want to send the notification.
  6. Provide a Subject.
  7. Provide a Message.
    You can add placeholder items to the payload that will be converted into the specific job information. You can add the placeholders in ${placeholder} format, and the following placeholders are available:
    • jobName
    • jobStatus
    • jobStatusDescription
    • ssbJobId
    • flinkJobId
    • clusterId
    • lastException
  8. Click Save.
  1. Navigate to Streaming SQL Console.
    1. Go to your cluster in Cloudera Manager.
    2. Select SQL Stream Builder from the list of services.
    3. Click Streaming SQL Console.

      The Streaming SQL Console opens in a new window.

  2. Click Job Notifications from the main menu on Streaming SQL Console.
  3. Click Create > Webhook.

    The Create Webhook Notification window appears.

  4. Provide a Name for the notification.
  5. Select between POST or PUT method.
  6. Provide a Webhook address.
  7. Provide a Payload to the webhook.
    You can add placeholder items to the payload that will be converted into the specific job information. You can add the placeholders in ${placeholder} format, and the following placeholders are available:
    • jobName
    • jobStatus
    • jobStatusDescription
    • ssbJobId
    • flinkJobId
    • clusterId
    • lastException
  8. Add HTTP Headers if needed by specifying the Header Name and Value.
  9. Click Save.
  1. Navigate to Streaming SQL Console.
    1. Go to your cluster in Cloudera Manager.
    2. Select SQL Stream Builder from the list of services.
    3. Click Streaming SQL Console.

      The Streaming SQL Console opens in a new window.

  2. Click Job Notifications from the main menu on Streaming SQL Console.
  3. Click Create > Group.

    The Create Group Notification window appears.

  4. Select from the already created Email and Webhook notifications that you would like to add to this group.
  5. Click Save.

Enabling notifications for a SQL job

After creating a job notification, you can enable it for a selected job from the Job Settings window.

  1. Select Job Settings on the Console page.
  2. Click Job Notifications.
  3. Search for the name that you have provided when creating the notification.
  4. Click on the notification.

    The selected notification is added to the job and the notification is sent with the information provided in the message template.