Understanding the use case

Learn how the Business Intelligence at Scale pattern empowers Ingest developers, Data engineers, Data Analysts, and Business stakeholders by using various Cloudera data services such as Streams Messaging Manager, DataFlow, Data Engineering, Data Warehouse, and Data Visualisation.

Business Intelligence at Scale is a common design pattern that you can implement in situations where business groups need insight from both enterprise and streaming data to make timely decisions. A traditional analytics solution would struggle with ingesting streaming data due to the technical challenges or experience needed in setting up a streaming ingest. This pattern guides Ingest Engineers to easily ingest streaming data, Data Engineers to model and curate data, and Business Analysts or Data Scientists to analyze, visualize, and share insights through an integrated self-service experience.

The objective is to allow business stake holders to quickly analyze all the data by using the integrated data lifecycle tools built into CDP. This service offers a smooth user experience for line of business decision makers to get answers to common questions through ready to use reports and dashboards, and an enhanced exploratory experience to line of business data analysts and power users to scrutinize data by building pipelines and querying the data sets.

Key problem this pattern solves

Customers can be more agile and better informed when making business decisions or creating ad-hoc reports based on new data as it arrives and becomes available.

Cloudera data lifecycle components

Business Intelligence at Scale pattern uses:

Cloudera's Streams Messaging Manager (SMM) to ingest external (non-enterprise) data by creating Kafka topics
Cloudera DataFlow Experience to convert and upload data to Amazon S3 (or any Cloud object store) in Avro format
Cloudera Data Engineering service to transform data into ORC or Parquet format using Spark jobs
Cloudera Data Warehouse service to create tables and materialized views from the datasets, run queries, and ad-hoc exploratory analysis
Cloudera Data Visualization to create reports and dashboards

This image shows various Cloudera data services that are used in the Business Intelligence at Scale pattern and key personas at play.

Using these Cloudera services:

Ingest developers can quickly develop data flows for real-time data and merge incoming data with existing data sets
Data engineers can transform, organize, and enrich data
Data Analysts and Data Scientists can query the data sets using Hue
Business stakeholders can derive insights using visualizations and dashboards