Homepage/Cloudera Streaming Analytics1.13.3

(Private Cloud)

Search Documentation

▶︎Cloudera
1. Reference Architectures
▶︎Cloudera Public Cloud
▼Cloudera Private Cloud
▶︎Cloudera Manager
1. Cloudera Manager
▶︎Applications
▶︎Legacy
▶︎

Data Services
Getting Started
Cloudera Manager
Management Console
Replication Manager
Data Catalog
Data Engineering
Data Warehouse
Data Warehouse Runtime
Cloudera AI
Base
Getting Started
Runtime
Upgrade
Storage
Flow Management
Streaming Analytics
Flow Management Operator
Streaming Analytics Operator
Streams Messaging Operator

«

Filter topics

Cloudera Streaming Analytics
▶︎Release Notes
▶︎Download
1. Download location of CSA
▶︎Apache Flink Overview
▶︎SQL Stream Builder Overview
▶︎Quick Start
▶︎Installation & Upgrade
▶︎Security
1. ▶︎Securing Apache Flink
2. ▶︎Securing Cloudera SQL Stream Builder
▼Using SQL Stream Builder
▶︎Using Apache Flink
▶︎Storm Flink Migration
1. ▶︎Comparing Storm and Flink
2. Migrating from Storm to Flink
SQL Stream Builder REST API Reference

Accessing the Flink Dashboard through Knox
Adding Catalogs
Adding Cloudera SQL Stream Builder as a Service
Adding data formats
Adding Flink as a Service
Adding Kafka Data Source
Adding new connectors
Adjusting logging configuration in Advanced Settings
After upgrading your cluster
ALTER Statements
Apache Flink Overview
Application development
Assigning Kafka keys in streaming queries
Atlas entities in Flink metadata collection
Authentication and encryption for Flink
Authentication in Cloudera SQL Stream Builder
Avro format
Before upgrading your cluster
Behavioral changes
Blackhole connector
CDC connectors
Choosing data sources
Cloudera Flink Tutorials
Cloudera SQL Stream Builder
Cloudera SQL Stream Builder architecture
Cloudera Streaming Analytics
Cluster service layout with Cloudera SQL Stream Builder
Cluster service layout with Flink
Collecting diagnostic data
Comparing Storm and Flink
Component support
Conceptual differences
Configuring custom Kerberos principal for Apache Flink
Configuring custom Kerberos principal for Cloudera SQL Stream Builder
Configuring Databases for Cloudera SQL Stream Builder
Configuring Flink application resources
Configuring Flink applications
Configuring Kafka tables
Configuring Materialized View database information
Configuring PyFlink applications
Configuring Ranger policies for Cloudera SQL Stream Builder
Configuring Ranger policies for Flink
Configuring Retention Time for Materialized Views
Configuring SQL job settings
Configuring state backend
Configuring state backend for Cloudera SQL Stream Builder
Configuring YARN queue for SQL jobs
Connectors
Converting DataStreams to Tables
Converting Tables to DataStreams
Core features of Flink
CREATE Statements
Creating a project
Creating and configuring the HBaseSinkFunction
Creating and naming SQL jobs
Creating Atlas entity type definitions for Flink
Creating Java User-defined functions
Creating Javascript User-defined Functions
Creating Materialized Views
Creating Python User-defined Functions
Creating Webhook tables
Creating widgets
CSV format
Customizing visualization types
Data Enrichment
Data formats
Data sources
Data Transformations tab
Data Types
Data types for Kafka connector
Datagen connector
DataStream API interoperability
DataStream connectors
Default ports for Flink and Cloudera SQL Stream Builder
Deployment scenarios
Deprecation notices in Cloudera Streaming Analytics 1.13.3
Deserialization tab
Developing JavaScript functions
Differences in architecture
Differences in data distribution
Download
Download location of CSA
DROP Statements
Dynamic SQL Hints
Enabling checkpoints for Flink applications
Enabling Flink DEBUG logging
Enabling Kerberos authentication
Enabling Knox authentication
Enabling Knox authentication for Flink Dashboard
Enabling Knox Auto Discovery for Flink
Enabling savepoints for Flink applications
Enabling security for Apache Flink
Enabling SPNEGO authentication for Flink Dashboard
Enabling TLS for database connection
Encryption in Cloudera SQL Stream Builder
Event Time tab
Example: joining Kafka and Kudu tables
Executing SQL jobs in production mode
Exporting SQL projects
Faker connector
File systems
Filesystem connector
Flink API Support
Flink application example
Flink application structure
Flink Dashboard
Flink DDL
Flink DML
Flink Project Template
Flink Queries
Flink SQL
Flink SQL tables
Flink Terminology
Functions
Getting Started
Governance
Governance
HBase sink with Flink
Hive catalog
Iceberg tables
Iceberg with Flink
Importing a project
Importing a project
INSERT Statements
Installation
Installation & Upgrade
Installing CSD and Parcel
JDBC connector
Job lifecycle
Joining streaming and bounded tables
JSON format
Kafka connector
Kafka connectors
Kafka Metrics Reporter
Kafka tables
Kafka with Flink
Key features of Cloudera SQL Stream Builder
Known issues and limitations
Kudu catalog
Kudu with Flink
Managing data source jobs
Managing member of a project
Managing session for SQL jobs
Managing teams in Streaming SQL Console
Managing time in Cloudera SQL Stream Builder
Managing widgets on the Dashboard
Masking information before using source control
Materialized View Pagination
Materialized views
Maven dependencies in Flink
Migrating Flink service to a different host
Migrating from Storm to Flink
Migrating SQL jobs
Migration
Monitoring
Monitoring
Navigating in a project
Notifications
Other supported statements
Performance & Scalability
Projects
Properties tab
Quick Start
Quick Start for Cloudera SQL Stream Builder
Quick Start for Flink
Reference
Release Notes
REST API
Resuming Flink applications
Resuming SQL jobs
Running a Flink job
Running a simple Flink application
Running SQL Stream jobs
Schema Definition tab
Schema Registry catalog
Schema Registry formats
Schema Registry with Flink
Securing Apache Flink
Securing Apache Flink jobs
Securing Cloudera SQL Stream Builder
Security
Setting parallelism and max parallelism
Setting the environment for a project
Setting the Java executable for the Flink client
Setting up MySQL/MariaDB database for Cloudera SQL Stream Builder
Setting up Oracle database for Cloudera SQL Stream Builder
Setting up PostgreSQL database for Cloudera SQL Stream Builder
Setting up Python for PyFlink
Setting up your HDFS Home directory
Source control of a project
Source, operator and sink in DataStream API
SQL and Table API
SQL and Table API supported features
SQL catalogs for Flink
SQL connectors for Flink
SQL Examples
SQL jobs
SQL Queries in Flink
SQL Statements in Flink
SQL Stream Builder Overview
Stopping Flink applications
Stopping SQL stream jobs
Storm Flink Migration
Streaming Analytics in Cloudera
Streams Messaging Manager integration
Support Matrix
Supported basic data types
Supported data types
System Requirements
Tables
Testing and validating Flink applications
Unsupported features
Updating Flink job dependencies
Updating SQL queries with PROCTIME function
Upgrade
Upgrading Cloudera Streaming Analytics artifacts and services
Uploading or unlocking your keytab
Using Apache Flink
Using Cloudera SQL Stream Builder with Cloudera Data Visualization
Using connectors with templates
Using Dynamic Materialized View Endpoints
Using EncryptTool for Flink properties
Using Flink CLI
Using SQL Stream Builder
Using System Functions
Verifying metadata collection
What is Apache Flink?
What is Cloudera SQL Stream Builder?
What's new in Cloudera Streaming Analytics
Widgets

«

Filter topics

SQL jobs

Getting Started
▶︎Projects
▶︎Data sources
1. Adding Kafka Data Source
2. Adding Catalogs
▶︎Connectors
▶︎Data formats
1. Adding data formats
▶︎Tables
▼SQL jobs
▶︎Functions
▶︎Materialized views
▶︎Widgets
Notifications
REST API
▶︎Monitoring
1. Collecting diagnostic data
Governance
▶︎Flink SQL
▶︎Data Enrichment

»

Using SQL Stream Builder

SQL jobs

Creating and naming SQL jobs
You need to create or select an already existing SQL job on the Streaming SQL Console to be able to submit SQL queries.
Running SQL Stream jobs
Every time you run an SQL statement in the SQL Stream console, it becomes a job and runs on the deployment as a Flink job. You can manage the running jobs using the Jobs tab on the UI.
Configuring SQL job settings
If you need to further customize your SQL Stream job, you can add more advanced features to configure the job restarting method and time, threads for parallelism, sample behavior, exactly once processing and restoring from savepoint.
Configuring state backend for Cloudera SQL Stream Builder
The same state backend configuration is applied to the SQL jobs in Cloudera SQL Stream Builder as configured for the Flink jobs. While Hashmap stores data as object on Java heap, RocksDB can be used to store larger state that does not fit easily in memory. The RocksDB state backend uses a combination of fast in-memory cache and optimized disk based lookups to manage state.
Managing session for SQL jobs
By default, the SQL Stream jobs are running in a session cluster. This means that multiple Flink jobs run in the same YARN session sharing the cluster, allocated resources, the Job Manager and Task Managers. The session starts when you open the Streaming SQL Console. You can reset the session, and set the properties of the session using the Streaming SQL Console.

© 2019, 2024 by Cloudera, Inc. All rights reserved.

This site uses cookies and related technologies, as described in our privacy policy, for purposes that may include site operation, analytics, enhanced user experience, or advertising. You may choose to consent to our use of these technologies, or

1.14.0
1.13.3
1.13.2
1.13.1
1.13.0
1.12.0
1.11.2
1.11.1
1.11.0
1.10.0
1.9.0
1.8.0
1.7.0
1.6
1.5
1.4
1. 1.4.1
2. 1.4.0
1.3.0
1.2.0
1.1.0