CCP High Level Architecture

Cloudera Cybersecurity Platform (CCP) is primarily backed by Storm and Kafka.

CCP also leverages the following components:

ZooKeeper provides dynamic configuration updates to running Storm topologies. This enables CCP to push updates to our Storm topologies without restarting them.
CCP uses HBase primarily for enrichments. But HBase is also used it to store user state for our UIs.
HDFS uses HDFS for long term storage. Parsed and enriched messages land here, along with any reported exceptions or errors encountered along the way.
Solr and Elasticsearch (plus Kibana)
HDP uses Solr and Elasticsearch (plus Kibana) for real-time access. CCP provides out of the box compatibility with both Solr and Elasticsearch, and custom dashboards for data exploration in Kibana.
Zeppelin provides dashboards to perform custom analytics.
Information is pushed into Metron by setting up Kafka topics for parsers to read from. There are a variety of options for setting up Kafka topics, including, but not limited to:
  • Grok Kafka plugin
  • Fastcapa
  • NiFi