Chapter 4. Enriching Telemetry Events
After the raw security telemetry events have been parsed and normalized, the next step is to enrich the data elements of the normalized event.
Enrichments add external data from data stores (such as HBase). Hortonworks Cybersecurity Platform (HCP) uses a combination of HBase, Storm, and the telemetry messages in json format to enrich the data in real time to make it relevant and consumable. You can use this enriched information immediately rather than needing to hunt in different silos for the relevant information.
HCP supports two types of configurations: global and sensor specific. The sensor specific configuration configures the individual enrichments and threat intelligence enrichments for a given sensor type (for example, squid). This section describes sensor specific configurations. For more information on global configuration, see Global Configuration.
HCP provides two types of enrichment:
Telemetry events
Threat intelligence information
For information about threat intelligence see Enriching Threat Intelligence Information.
HCP provides the following telemetry enrichment sources but you can add your own enrichment sources to suit your needs:
Asset
GeoIP
User
![[Note]](../common/images/admon/note.png) | Note |
|---|---|
The telemetry data sources for which HCP includes parsers (for example, Bro, Snort, and YAF) already include enrichment topologies. These topologies will become effective when you start the data sources in HCP. |
Prior to enabling an enrichment capability within HCP, the enrichment store (which for HCP is primarily HBase) must be loaded with enrichment data. The dataload utilities convert raw data sources to a primitive key (type, indicator) and value and place it in HBase.
HCP supports three types of enrichment loaders:
Bulk load from HDFS via MapReduce
Taxii Loader
Flat File ingestion
For simplicity's sake, we use the bulk loader to load enrichments:
| Task | Description |
|---|---|
| Bulk Loading Enrichment Information |
Bulk loading is used to load information that does not change frequently. |
| Mapping Fields to HBase Enrichments |
Now that you have data flowing into the HBase table, you need to ensure that the enrichment topology can be used to enrich the data flowing past. |
| OPTIONAL:Global Configuration |
Global enrichments are applied to all data sources as opposed to other enrichments that are applied at the field level to individual sensors. This type of enrichment can save you time by applying common enrichments to all of your sensors. |
| Verify Events are Enriched |
After you finish enriching your new data source, you should verify that the output matches your enrichment information. |