SQL Stream Builder Architecture
The SBB service is integrated on the Cloudera platform connected to Flink and its services: YARN, Kafka and Schema Registry. The primary point of user interaction for SQL Stream Builder is the Console component. When you submit a query using the UI, a Flink job is created on the cluster. The schema corresponding to the query is downloaded through Schema Registry. The Kafka topic is also populated by the Flink job submission. You are able to monitor and manage your Flink jobs using the YARN Resource Manager or the Flink Dashboard.
- SQL Stream Engine
- Streaming SQL Console
- Materialized View Engine
The primary point of user interaction for SQL Stream Builder is the Console component. When you submit a query using the Streaming SQL Console, a Flink job is automatically created in the background on the cluster. SSB also requires a Kafka service on the same cluster. This mandatory Kafka service is used to automatically populate topics for the websocket output. The websocket output is needed for sampling data to the Console, and when no virtual table sink is added to the SQL query.
When a Materialized View query is submitted, Flink generates the data to the Materialized View database from which the Materialized View Engine queries the required data. The Streaming SQL Console and the Materialized Views need databases where the metadata of SQL jobs are stored and from which the Materialized View Engine queries data to create the views. SSB supports MySQL/ MariaDB and PostgreSQL as databases. For the Streaming SQL Console, you can choose MySQL/MariaDB or PostgreSQL. However, you must install PostgreSQL to be able to create Materialized Views.
