Analytics
Also available as:
PDF

Using Zeppelin to Analyze Data

Zeppelin enables you to analyze the enriched telemetry information Metron archives in HDFS.

Zeppelin includes tutorials that can help you learn how to use Zeppelin and start analyzing data:

In addition to creating your own notebooks, HCP provides several notebooks that you can use to analyze data and produce reports:

  • Metron - Connection Report

    This notebook enables you to determine the number of connections made between IPs. This notebook can be set up for Yaf, Bro, or Spark.

  • Metron - Connection Volume Report

    This notebook enables you to determine the number of connections filtered by a CIDR block. This notebook is set up for YAF.

  • Metron - YAF Telemetry

    This notebook enables you to obtain flow telemetry information for YAF, including:

    • Top talkers - internal and external

    • Flows by hour - internal and external

    • Top locations

    • Flow duration internal and external

  • Metron IP report

    This notebook enables you to produce a report for a given address that includes the following:

    • Most frequent connections (YAF, defaults to 24 hours)

    • Recent connections (YAF, defaults to 1 hour)

    • Top DNS queries (Bro, defaults to 24 hours)

    • All ports used (YAF, defaults to 24 hours)

    • HTTP user agents (Bro, defaults to 24 hours)

The following example guides you through using Zeppelin and the notebooks provided by HCP to perform different types of analyses on the Bro telemetry information to discover a potential issue.

  1. Bro produces data about a number of different network protocols. View which types of protocols exist in the data.

    Figure 4.2. Bro Protocols


  2. View when Bro telemetry information was received.

    You can check for any odd gaps and fluctuating periods of high and low activity.

    [Note]Note

    If there is not enough data for this visualization to be interesting, let Metron consume more data before continuing.

    Figure 4.3. Bro Telemetry Received


  3. List the most active Bro hosts.

    Figure 4.4. Most Active Hosts


  4. List any DNS servers running on non-standard ports.

    Figure 4.5. DNS Servers


  5. List any mime types that could be concerning.

    Figure 4.6. Mime Types


  6. Explode the HTTP records.

    Each HTTP record can contain multiple mime types. These need to be 'exploded' to work with them properly.

    Figure 4.7. Exploded HTTP Records


  7. Determine where application/x-dosexec originated.

    Figure 4.8. Suspicious xdosexc


  8. Take a look at the requests for x-dosexc.

    Figure 4.9. x-dosexc Requests


  9. Determine when the interactions with the suspicious host are occurring.

    Figure 4.10. When Interactions Occur


  10. Create an IP report in Zeppelin using the Metron IP Report notebook.

    For a given IP address, this notebook produces a report of:

    • Most frequent connections (YAF defaults to 24 hours)

    • Recent connections (Yaf, defaults to 1 hours)

    • Top DNS queries (Bro, defaults to 24 hours)

    • All ports used (Yaf, defaults to 24 hours)

    • HTTP user agents (Bro, defaults to 24 hours)

  11. Create traffic connection request report using the Connection Volume Report notebook.

    This notebook lets the user get connection counts filtered by a CIDR block. This notebook can be used for Bro, Yaf, and Snort.