Run Book

Also available as:

PDF

Contents

loading table of contents...

Hortonworks Cybersecurity Package

Run Book

Copyright © 2012-2017 Hortonworks, Inc.

Except where otherwise noted, this document is licensed under Creative Commons Attribution ShareAlike 4.0 License

http://creativecommons.org/licenses/by-sa/4.0/legalcode

Hortonworks Cybersecurity Package (HCP) is a modern data application based on Apache Metron, powered by Apache Hadoop, Apache Storm, and related technologies.

HCP provides a framework and tools to enable greater efficiency in Security Operation Centers (SOCs) along with better and faster threat detection in real-time at massive scale. It provides ingestion, parsing and normalization of fully enriched, contextualized data, threat intelligence feeds, triage and machine learning based detection. It also provides end user near real-time dashboards.

Based on a strong foundation in the Hortonworks Data Platform (HDP) and Hortonworks DataFlow (HDF) stacks, HCP provides an integrated advanced platform for security analytics.

Please visit the Hortonworks Data Platform page for more information on Hortonworks technology. For more information on Hortonworks services, please visit either the Support or Training page. Feel free to Contact Us directly to discuss your specific needs.

Contents

1. Overview

2. Adding a New Telemetry Data Source

Prerequisites

Stream Data into HCP

OPTIONAL: Install the Squid Package
Install NiFi
Create a NiFi Flow to Stream Events to HCP

Parse the Squid Data Source to HCP

Parse the Squid Telemetry Event
Verify That the Events Are Indexed
Create an Index Template

Add New Data Source to the Metron Dashboard

Configuring a New Data Source Index in the Metron Dashboard
Reviewing the New Data Source Data

3. Transform the Squid Message

4. Enriching Telemetry Events

Bulk Loading Enrichment Information

OPTIONAL: Create a Mock Enrichment Source
Configuring an Extractor Configuration File
Configuring Element-to-Enrichment Mapping
Running the Enrichment Loader

Mapping Fields to HBase Enrichments

OPTIONAL: Global Configuration

Verify That the Events Are Enriched

5. Enriching Threat Intelligence Information

OPTIONAL: Create a Mock Threat Intel Feed Source
Configuring an Extractor Configuration File
Configuring Element-to-Threat Intel Feed Mapping
Run the Threat Intelligence Loader
Mapping Fields to HBase Enrichments
Verify That the Threat Intel Events Are Enriched

6. Prioritizing Threat Intelligence

Prerequisites

Performing Threat Triage

Viewing Triaged or Scored Alerts

Threat Triage Alert Panel
HCP Metron Dashboard View of Alerts

7. Configuring Indexing

Default Configuration
Specifying Index Parameters
Turning off HDFS Writer

8. Setting Up a Profile

Installing Profiler
Creating a Profile
Configuring the Profiler
Starting the Profiler
Developing Profiles
Testing

List of Figures

2.1. Add Processor Dialog Box
2.2. New TailFile Processor
2.3. Configure Processor Dialog Box Settings Tab
2.4. Configure Processor Dialog Box Properties Tab
2.5. Configure Processor
2.6. Configure Properties
2.7. Create Connection Dialog Box
2.8. Data Flow
2.9. NiFi Operate Panel
2.10. New Sensor Panel
2.11. Grok Validator Panel with Sample Text
2.12. Grok Validator Panel with Statement Information
2.13. Grok Validator Panel with First Element and Test Results
2.14. Grok Validator Panel with Second Element and Test Results
2.15. mm_grok_validator_squid_complete.png
2.16. Elasticsearch
2.17. Ambari Task List
2.18. Configure an Index Pattern
2.19. Discover Tab with Squid Elements
3.1. Squid Schema Panel
3.2. New Schema Information Panel
3.3. Populated Schema Information Panel
4.1. New Schema Information Panel
4.2. Populated New Schema Information Panel
4.3. Elasticsearch
5.1. New Schema Information Panel
5.2. Populated New Schema Information Panel
5.3. Elasticsearch
6.1. Threat Triage Rules Panel
6.2. Edit Rule Panel
6.3. Investigation Module Triaged Alert Panel
7.1. Management Module Advanced Panel