What's New in Streaming Analytics

Learn about the new Streaming Analytics features in Cloudera DataFlow for Data Hub 7.2.16.

7.2.16.1

The following new features are introduced in Streaming Analytics CDF for Data Hub 7.2.16.1:

Confluent Schema Registry support: Confluent Schema Registry is added as a supported Catalog in SQL Stream Builder.
Configurable column indexing for Materialized Views: When creating Materialized Views, you can enable column indexing and select the columns that need to be indexed.

7.2.16

The following new features are introduced in Streaming Analytics CDF for Data Hub 7.2.16:

Apache Flink upgrade: Apache Flink 1.15.1 is supported in the Streaming Analytics 7.2.16 cluster definition.; For more information on what is included in the Apache Flink 1.15 version, see the Apache Flink 1.15.1 Release Post and the Apache Flink 1.15 Release Notes
Reworked Streaming SQL Console: The User Interface (UI) of SQL Stream Builder (SSB), the Streaming SQL Console has been reworked with new design elements.
Software Development Lifecycle (SDLC) support: Projects are introduced as an organizational element for SQL Stream Builder that allows you to create and collaborate on SQL jobs throughout the SDLC stages with source control. For more information, see the Project structure and development documentation.
note
The Source Control and Service Discovery features are in Technical Preview and not ready for production deployment. Cloudera encourages you to explore these features in non-production environments and provide feedback on your experiences through the Cloudera Community Forums.
Iceberg support: Support for Apache Iceberg table has been added as a connector to Flink. This allows you to create Iceberg tables in Streaming SQL Console.; For more information, see the Creating Iceberg tables documentation.
note
The integration of Apache Iceberg with Flink and SQL Stream Builder is in Technical Preview and not ready for production deployment. Cloudera encourages you to explore these features in non-production environments and provide feedback on your experiences through the Cloudera Community Forums.
Job notifications: You can enable notifications for SQL jobs to send information about job failures to a dedicated email group or through webhook.; For more information, see the Using SQL job notification documentation.
Diagnostic data collection: Data collection for diagnostic information is enabled for SQL Stream Builder to make troubleshooting and support processes easier.; For more information, see the Collecting diagnostic data documentation.
Materialized View pagination: You can set a limit and offset for the results of Materialized View queries to filter the query results more easier when accessing them through the Query API.; For more information, see the Materialized View Pagination documentation.
Configuring retention time for Materialized Views: You can configure how data should be retained for Materialized Views based on time or row count.; For more information, see the Configuring Retention Time for Materialized Views documentation.
SQL Server CDC Connector support: MS SQL Server Change Data Capture (CDC) connector is added to the list of supported connectors. The SQL Server CDC connector can be used with Flink SQL on the Streaming SQL Console either directly from the SQL Editor or using the DDL template.
JSON support for Schema Registry: JSON schemas are supported when using Cloudera Schema Registry with Kafka.