Enriching Telemetry Events Overview

After the raw security telemetry events have been parsed and normalized, you need to enrich the data elements of the normalized event.

Enrichments add external data from data stores (such as HBase). CCP uses a combination of HBase, Storm, and the telemetry messages in json format to enrich the data in real time to make it relevant and consumable. You can use this enriched information immediately rather than needing to hunt in different silos for the relevant information.

CCP supports two types of configurations: global and sensor specific. The sensor specific configuration configures the individual enrichments and threat intelligence enrichments for a given sensor type (for example, squid). This section describes sensor specific configurations.

CCP provides two types of enrichment:
  • Telemetry events
  • Threat intelligence information

CCP provides the following telemetry enrichment sources but you can add your own enrichment sources to suit your needs:

  • Asset

  • GeoIP

  • User

Prior to enabling an enrichment capability within CCP, the enrichment store (which for CCP is primarily HBase) must be loaded with enrichment data. The dataload utilities convert raw data sources to a primitive key (type, indicator) and value and place it in HBase.

CCP supports three types of enrichment loaders:
  • Bulk load from HDFS via MapReduce

  • Taxii Loader

  • Flat File ingestion

For simplicity's sake, we use the bulk loader to load enrichments: