ReadyFlow: Kafka to Kudu
This use case shows you how you can move your data from a Kafka topic into Apache Kudu in your CDP Public Cloud Real-time Data Mart cluster. You can learn how to create such a data flow easily using the Kafka to Kudu ReadyFlow.
This ReadyFlow consumes JSON, CSV or Avro data from a source Kafka topic, parses the schema by looking up the schema name in the CDP Schema Registry and ingests it into a Kudu table. You can pick the Kudu operation (INSERT, INSERT_IGNORE, UPSERT, UPDATE, DELETE, UPDATE_IGNORE, DELETE_IGNORE) that fits best for your use case. Failed Kudu write operations are retried automatically to handle transient issues. Define a KPI on the failure_WriteToKudu connection to monitor failed write operations.
| Kafka to Kudu ReadyFlow details | |
|---|---|
| Source | Kafka topic |
| Source Format | JSON, CSV, Avro |
| Destination | Kudu |
| Destination Format | Kudu Table |
Time series use cases analyse data obtained during specified intervals, and enable you to improve performance based on available data. Examples include:
-
Optimizing yield or yield quality in a manufacturing plant
-
Dynamically optimizing network capacity during peak load of better telecommunications uptime and services
These use cases require that you store events at a high frequency, while providing ad-hoc query and record update abilities.
